# Configure Data Transforms

> For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [streaming-full.txt](https://docs.redpanda.com/streaming-full.txt)

---
title: Configure Data Transforms
latest-redpanda-tag: v25.3.11
latest-console-tag: v3.7.3
latest-operator-version: v26.1.4
# EOL = End-of-Life (support lifecycle status)
page-is-nearing-eol: "false"
page-is-past-eol: "false"
page-eol-date: November 19, 2026
latest-connect-version: 4.93.0
docname: data-transforms/configure
page-component-name: streaming
page-version: "25.3"
page-component-version: "25.3"
page-component-title: Streaming
page-relative-src-path: data-transforms/configure.adoc
page-edit-url: https://github.com/redpanda-data/docs/edit/v/25.3/modules/develop/pages/data-transforms/configure.adoc
description: Learn how to configure data transforms in Redpanda, including editing the <code>transform.yaml</code> file, environment variables, and memory settings. This topic covers both the configuration of transform functions and the WebAssembly (Wasm) engine's environment.
page-git-created-date: "2024-07-31"
page-git-modified-date: "2025-04-08"
support-status: supported
---

<!-- Source: https://docs.redpanda.com/streaming/25.3/develop/data-transforms/configure.md -->

Learn how to configure data transforms in Redpanda, including editing the `transform.yaml` file, environment variables, and memory settings. This topic covers both the configuration of transform functions and the WebAssembly (Wasm) engine’s environment.

## [](#configure-transform-functions)Configure transform functions

This section covers how to configure transform functions using the `transform.yaml` configuration file, command-line overrides, and environment variables.

### [](#config-file)Transform configuration file

When you [initialize](https://docs.redpanda.com/streaming/25.3/develop/data-transforms/build/#init) a data transforms project, a `transform.yaml` file is generated in the provided directory. You can use this configuration file to configure the transform function with settings, including input and output topics, the language used for the data transform, and any environment variables.

-   `name`: The name of the transform function.

-   `description`: A description of what the transform function does.

-   `input-topic`: The topic from which data is read.

-   `output-topics`: A list of up to eight topics to which the transformed data is written.

-   `language`: The language used for the transform function. The language is set to the one you defined during [initialization](https://docs.redpanda.com/streaming/25.3/develop/data-transforms/build/#init).

-   `env`: A dictionary of custom environment variables that are passed to the transform function. Do not prefix keys with `REDPANDA_`. Check the list of all [limitations](https://docs.redpanda.com/streaming/25.3/develop/data-transforms/how-transforms-work/#limitations).


Here is an example of a transform.yaml file:

```yaml
name: redpanda-example
description: |
  This transform function is an example to demonstrate how to configure data transforms in Redpanda.
input-topic: example-input-topic
output-topics:
  - example-output-topic-1
  - example-output-topic-2
language: tinygo-no-goroutines
env:
  DATA_TRANSFORMS_ARE_AWESOME: 'true'
```

### [](#cl)Override configurations with command-line options

You can set the name of the transform function, environment variables, and input and output topics on the command-line when you deploy the transform. These command-line settings take precedence over those specified in the `transform.yaml` file.

See [Deploy Data Transforms](https://docs.redpanda.com/streaming/25.3/develop/data-transforms/deploy/)

### [](#built-in)Built-In environment variables

As well as custom environment variables set in either the [command-line](#cl) or the [configuration file](#config-file), Redpanda makes some built-in environment variables available to your transform functions. These variables include:

-   `REDPANDA_INPUT_TOPIC`: The input topic specified.

-   `REDPANDA_OUTPUT_TOPIC_0..REDPANDA_OUTPUT_TOPIC_N`: The output topics in the order specified on the command line or in the configuration file. For example, `REDPANDA_OUTPUT_TOPIC_0` is the first variable, `REDPANDA_OUTPUT_TOPIC_1` is the second variable, and so on.


Transform functions are isolated from the broker’s internal environment variables to maintain security and encapsulation. Each transform function only uses the environment variables explicitly provided to it.

## [](#configure-the-wasm-engine)Configure the Wasm engine

This section covers how to configure the Wasm engine environment using Redpanda cluster configuration properties.

### [](#enable-transforms)Enable data transforms

To use data transforms, you must enable it for a Redpanda cluster using the [`data_transforms_enabled`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_enabled) property.

### [](#configure-memory-resources-for-data-transforms)Configure memory resources for data transforms

Redpanda reserves memory for each transform function within the broker. You need enough memory for your input record and output record to be in memory at the same time.

Set the following based on the number of functions you have and the amount of memory you anticipate needing.

-   [`data_transforms_per_core_memory_reservation`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_per_core_memory_reservation): Increase this setting if you plan to deploy a large number of data transforms or if your transforms are memory-intensive. Reducing it may limit the number of concurrent transforms.

-   [`data_transforms_per_function_memory_limit`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_per_function_memory_limit): Adjust this setting if individual transform functions require more memory to process records efficiently. Reducing it may cause memory errors in complex transforms.


The maximum number of functions that can be deployed to a cluster is equal to `data_transforms_per_core_memory_reservation` / `data_transforms_per_function_memory_limit`. When that limit is hit, Redpanda cannot allocate memory for the VM and the transforms stay in `errored` states.

### [](#binary-size)Configure maximum binary size

You can set the maximum size for a deployable Wasm binary that the broker can store using the [`data_transforms_binary_max_size`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_binary_max_size) property.

Increase this setting if your Wasm binaries are larger than the default limit. Setting it too low may prevent deployment of valid transform functions.

### [](#commit-interval)Configure commit interval

You can set the interval at which data transforms commit their progress using the [`data_transforms_commit_interval_ms`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_commit_interval_ms) property.

Adjust this setting to control how frequently the transform function’s progress is committed. Shorter intervals may provide more frequent progress updates but can increase load. Longer intervals reduce load but may delay progress updates.

### [](#log)Configure transform logging

The following properties configure logging for data transforms:

-   [`data_transforms_logging_buffer_capacity_bytes`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_logging_buffer_capacity_bytes): Increase this value if your transform logs are large or if you need to buffer more log data before flushing. Reducing this value may cause more frequent log flushing.

-   [`data_transforms_logging_flush_interval_ms`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_logging_flush_interval_ms): Adjust this value to control how frequently logs are flushed to the `transform_logs` topic. Shorter intervals provide more frequent log updates but can increase load. Longer intervals reduce load but may delay log updates.

-   [`data_transforms_logging_line_max_bytes`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_logging_line_max_bytes): Increase this value if your log messages are frequently truncated. Setting this value too low may truncate important log information.


### [](#runtime-limit)Configure runtime limits

You can set the maximum runtime for starting up a data transform and the time it takes for a single record to be transformed using the [`data_transforms_runtime_limit_ms`](https://docs.redpanda.com/streaming/25.3/reference/properties/cluster-properties/#data_transforms_runtime_limit_ms) property.

Adjust this value only if your transform functions need more time to process each record or to start up.

## [](#next-steps)Next steps

[Deploy Data Transforms](https://docs.redpanda.com/streaming/25.3/develop/data-transforms/deploy/)

## Suggested labs

-   [Flatten JSON Messages](https://docs.redpanda.com/labs/data-transforms/flatten-go/)
-   [Convert JSON Messages into Avro](https://docs.redpanda.com/labs/data-transforms/issdemo-go/)
-   [Filter Messages into a New Topic using a Regex](https://docs.redpanda.com/labs/data-transforms/regex-go/)
-   [Convert Timestamps using Rust](https://docs.redpanda.com/labs/data-transforms/ts-converter-rust/)
-   [Redact Information in JSON Messages](https://docs.redpanda.com/labs/data-transforms/redaction-go/)

[Search all labs](https://docs.redpanda.com/labs)