Skip to content

Scowl Language Tutorial

Overview

Scowl is a language for creating scalable, context-aware, intelligent services. Events stream into the service; a user-defined topology processes each event; one or more decisions are returned.

Users define their logic declaratively. Scowl takes care of the rest: resolving dependencies, optimizing execution, and, most importantly, maintaining state.

Events and Topologies

An event is a named occurrence (e.g. signup, login, purchase), that happens at a particular time. For each event, a topology, written in Scowl, defines how the event will be processed in three stages:

  1. Extract - For ease of integration, Scowl allows clients to post data in a flexible JSON format. A topology's first step is to extract the desired data into a flat collection of fields, and to declare each field's data type.
  2. Enrich - The most interesting step is enriching the event with additional features through a collection of stateless and stateful transformations. Scowl's primary design goal is making this step both simple and powerful.
  3. Decide - Finally, rules define conditions that are suspicious or otherwise noteworthy. And decisions define what action(s) should be taken as a result.

Your First Scowl Topology

Let's create a trivial but instructive service to check if an email address is from a popular free email domain.

We expect the incoming JSON request to have a single top-level field called "email":

{
  "email": "hello@scowl.dev"
}

We will return a true/false decision about whether the email is from a free domain.

Declare Event

Every Scowl topology begins with the name of the event to be processed. The convention is to choose active names, e.g. login or create_comment. In this example, we will name our event verify_email:

event verify_email

Extract Data

Now inside this context, we can start to define our event's topology, beginning with the extract step.

To extract the email, we use Scowl's JSON extractor, which takes a JSONPath and returns the element at that path as the stated data type:

email := $.email as string

Our verify_email event schema now has a single string-type feature, called email.

Enrich Features

Next, we enrich our schema with a feature that captures the domain portion of the email address (e.g. "scowl.dev"). To do so, we will use the EmailDomain function from the standard library:

domain := EmailDomain(email)

Add a Decision

For our decide step, we define the conventional single decision, verdict. We specify the order in which rules are evaluated. The first true rule wins. By default the verdict will be Allow.

verdict := Decision(Allow, Block, Review default Allow)

Add a Rule

Finally, we add a rule that fires if the domain is from a free email service. Rules are defined like features, but specify the decision value set when the rule fires:

free_domain := Review when domain in ['gmail.com', 'yahoo.com', 'hotmail.com']

Obviously, in a production setting we would use a more robust domain list.

Full Topology

Here is the complete Scowl code for our simple topology:

event verify_email

email := $.email as string
domain := EmailDomain(email)
verdict := Decision(Allow, Block, Review default Allow)
free_domain := Review when domain in ['gmail.com', 'yahoo.com', 'hotmail.com']

Testing in the REPL

Scowl comes with a Read-Eval-Print-Loop (REPL) to develop and test topologies offline. If we run the REPL on the verify_email code above, we are dropped into a prompt to begin testing.

Set Up Test Data

First, we need to create an example request. In Scowl, $ represents the request JSON object. We can use the REPL's set command to set it:

scowl:verify_email> set $ {"email": "hello@scowl.dev"}

Evaluate Topology

To fetch our topology's decision, we can evaluate it by name:

scowl:verify_email> verdict
"Allow"

To see all of the intermediate feature values, we use the show features command:

scowl:verify_email> show features
{
 "domain": "scowl.dev",
 "email": "hello@scowl.dev",
 "free_domain": false,
 "verdict": "Allow"
}

Change Test Data

Now let's change the test input using reverse JSONPath syntax: set <json-path> <value>. We can see the new request by simply evaluating $:

scowl:verify_email> set $.email "spammer@hotmail.com"
scowl:verify_email> $
{
 "email": "spammer@hotmail.com"
}

For this new request, our topology evaluates verdict as "Review":

scowl:verify_email> show features
{
 "domain": "hotmail.com",
 "email": "spammer@hotmail.com",
 "free_domain": true,
 "verdict": "Review"
}

Explain Rule

One of Scowl's key design goals is to make all decisions easily interpretable. To explain any decision or feature, we use the REPL's explain command. This command outputs a tree view of every feature that influenced the given feature, which features influenced those, and so on:

scowl:verify_email> explain verdict
verdict: "Review"
 free_domain: true
  domain: "hotmail.com"
   email: "spammer@hotmail.com"

Error Handling

In the course of executing a topology, Scowl logs data errors and logic errors along with the features that generated them.

We can trigger an error by changing the input to a malformed email address:

scowl:verify_email> set $.email "not-an-email"

To display the errors, we use the REPL's show errors command:

scowl:verify_email> show errors
{
 "domain": "expected email address, got not-an-email"
}

Next Steps

The simple example in this tutorial covered Scowl's most basic functionality. Scowl supports the development of much more sophisticated logic. Some of the key functionality not covered so far includes:

  • Stateful aggregate features
  • Joining data from other event streams
  • Enriching data with service calls

To see some of these capabilities in action, check out our Recipes. For detailed information, review the Scowl documentation.