Connect

javascript

Available in: Self-Managed

Executes a JavaScript code block or file for each message.

Introduced in version 4.14.0.

# Configuration fields, showing default values
label: ""
javascript:
  code: "" # No default (optional)
  file: "" # No default (optional)
  global_folders: []

The execution engine behind this processor provides full ECMAScript 5.1 support (including regex and strict mode). Most of the ECMAScript 6 spec is implemented but this is a work in progress.

Imports via require should work similarly to NodeJS, and access to the console is supported which will print via the Redpanda Connect logger. More caveats can be found on GitHub.

This processor is implemented using the github.com/dop251/goja library.

Fields

code

An inline JavaScript program to run. You must specify a value for either the code or file field.

Type: string

file

A file containing a JavaScript program to run. You must specify a value for either the code or file field.

Type: string

global_folders[]

A list of directories to load modules from if the requested JavaScript module is not found elsewhere.

Type: array

Default: []

Examples

Simple mutation

In this example we define a simple function that performs a basic mutation against messages, treating their contents as raw strings.

pipeline:
  processors:
    - javascript:
        code: 'benthos.v0_msg_set_string(benthos.v0_msg_as_string() + "hello world");'

Structured mutation

In this example we define a function that performs basic mutations against a structured message. Note that we encapsulate the logic within an anonymous function that is called for each invocation, this is required in order to avoid duplicate variable declarations in the global state.

pipeline:
  processors:
    - javascript:
        code: |
          (() => {
            let thing = benthos.v0_msg_as_structured();
            thing.num_keys = Object.keys(thing).length;
            delete thing["b"];
            benthos.v0_msg_set_structured(thing);
          })();

Runtime

To optimize code execution, JavaScript runtimes are created on demand (in order to support parallel execution) and are reused across invocations. Therefore, it’s important to understand that the global state created by your programs will outlive individual invocations. For your programs to avoid failing after the first invocation, ensure that you do not define variables at the global scope.

Although technically possible, it is recommended that you do not rely on the global state for maintaining state across invocations as the pooling nature of the runtimes will prevent deterministic behavior. We aim to support deterministic strategies for mutating global state in the future.

Functions

benthos.v0_fetch

Executes an HTTP request synchronously and returns the result as an object of the form {"status":200,"body":"foo"}.

Parameters

url <string> The URL to fetch headers <object(string,string)> An object of string/string key/value pairs to add the request as headers. method <string> The method of the request. body <(optional) string> A body to send.

Examples

let result = benthos.v0_fetch("http://example.com", {}, "GET", "")
benthos.v0_msg_set_structured(result);

benthos.v0_msg_as_string

Obtain the raw contents of the processed message as a string.

Examples

let contents = benthos.v0_msg_as_string();

benthos.v0_msg_as_structured

Obtain the root of the processed message as a structured value. If the message is not valid JSON or has not already been expanded into a structured form this function will throw an error.

Examples

let foo = benthos.v0_msg_as_structured().foo;

benthos.v0_msg_exists_meta

Check that a metadata key exists.

Parameters

name <string> The metadata key to search for.

Examples

if (benthos.v0_msg_exists_meta("kafka_key")) {}

benthos.v0_msg_get_meta

Get the value of a metadata key from the processed message.

Parameters

name <string> The metadata key to search for.

Examples

let key = benthos.v0_msg_get_meta("kafka_key");

benthos.v0_msg_set_meta

Set a metadata key on the processed message to a value.

Parameters

name <string> The metadata key to set. value <anything> The value to set it to.

Examples

benthos.v0_msg_set_meta("thing", "hello world");

benthos.v0_msg_set_string

Set the contents of the processed message to a given string.

Parameters

value <string> The value to set it to.

Examples

benthos.v0_msg_set_string("hello world");

benthos.v0_msg_set_structured

Set the root of the processed message to a given value of any type.

Parameters

value <anything> The value to set it to.

Examples

benthos.v0_msg_set_structured({
  "foo": "a thing",
  "bar": "something else",
  "baz": 1234
});