Skip to content


Released: December 3, 2021

Language Enhancements

Support Complex Types in Latest

Scowl now allows you to select complex data types: map, struct, and array, in Latest queries.

For example, the following is now supported when policy_details is a struct:

Latest<carrier_cards_details_auto>(policy_details, carrier_name by card_id)

Maps, structs, and arrays may be arbitrarily nested. The following type from our fuzz testing was confirmed to work just fine:

{x_0: {x_0: map{string: {x_0: [time], x_1: {x_0: time, x_1: string, x_2: int}, x_2: map{string: int}}}, x_1: {x_0: {x_0: map{string: bool}, x_1: [int], x_2: {x_0: bool, x_1: time, x_2: time}}}}, x_1: map{string: {x_0: {x_0: map{string: int}, x_1: {x_0: float, x_1: bool, x_2: float}, x_2: map{string: string}}, x_1: map{string: [time]}, x_2: map{string: map{string: float}}}}, x_2: {x_0: map{string: [[float]]}, x_1: [map{string: {x_0: float, x_1: string}}], x_2: {x_0: map{string: map{string: bool}}, x_1: map{string: {x_0: time}}, x_2: [[string]]}}}

New Configuration Options

This release introduces a new parameter to the Terraform Sumatra module:

variable "es_volume_size" {
  description = "AWS OpenSearch volume size per node"
  type = number
  default = 10

This parameter configures the OpenSearch volume size per node.

Dev Configuration

The es_volume_size should be set to 10 for the sandbox instance to match the current settings. In the future, we may decide to bump up this value.

Prod Configuration

Based on current sizing, 100 is the proposed initial configuration value for Prod.

Other Enhancements

Warning on Feature Type Change

If the data type of a feature appearing in a Latest query changes type, a warning will be shown at publishing time.

To preclude the possibility of data corruption in DynamoDB, history of the aggregate will be reset and start accruing from scratch again.

For example, in the following query:

event impression
zip_code := $.data.zip_code as string

event query
user_zip := Latest<impression>(zip_code by user_id last 10 days)

if the type of impression.zip_code were changed from a string to an int, then user_zip may contain some null values starting at publishing time until 10 days pass to backfill the history.

UI Enhancements

  • "Snackbar" error reporting in Editor
  • Minor changes to publish branch view