What’s New in Redpanda Connect

This topic includes new content added from version 4.29.0 onwards.

For a full list of product updates, and details of updates prior to version 4.29.0, see the Redpanda Connect release notes. For new Cloud-specific features, see What’s New in Redpanda Cloud.

Version 4.42.0

This release introduces a Bloblang playground where you can try out your Bloblang syntax and get immediate feedback. It also updates Redpanda Connect licensing checks for Redpanda Connect. Starting from version 4.42.0, you must add your Enterprise Edition license to Redpanda Connect before using any Enterprise connectors.

Version 4.41.0

This release adds the max_records_per_request field to the aws_sqs output.

Packaging update

Starting from version 4.41.0, the -cgo suffixed Docker image is no longer available due to low demand and the unacceptable cadence with which the image base (Debian) receives security updates.

To create your own CGO builds, run the following command:

CGO_ENABLED=1 make TAGS=x_benthos_extra redpanda-connect.

Version 4.40.0

This update includes a beta release of the pg_stream input for Redpanda Connect Self-Managed and Cloud, which captures data changes made to a PostgreSQL database in real time and streams them to Redpanda Connect.

Component updates

This release adds:

Other changes:

Deprecations

Deprecated field: kafka_timestamp metadata field

Affected components:

Replaced by: kafka_timestamp_ms metadata field

Deprecated field: timestamp

Affected components:

Replaced by: timestamp_ms field

Action required: Modify all systems that process these fields to ensure compatibility with the new field names.

Version 4.39.0

Highlights of this update include the beta release of three new components for Self-Managed and Cloud, including a new snowflake_streaming output. This release also includes additions to the redpanda family of components and introduces new features for Self-Managed.

Components for Cloud and Self-Managed (beta)

Additional redpanda components (beta)

  • redpanda (input and output): Consume message data from or send message data to Kafka brokers.

  • redpanda_common (input and output): Consume message data from or send message data to a Redpanda Kafka broker, using credentials from a common redpanda configuration block (Self-Managed only).

Features in Self-Managed

You can now:

Component updates

This release adds the following fields to the redpanda configuration service:

  • metadata_max_age

  • partitioner

  • idempotent_write

  • compression

  • broker_write_max_bytes

The url field of the aws_sqs output now supports interpolation functions.

Version 4.38.0

This update includes the beta release of two new outputs for Cloud and Self-Managed, and the introduction of a telemetry service, which collects and sends anonymized usage statistics to Redpanda.

New tooling

A new Helm chart for deploying Redpanda Connect in Kubernetes is now available. This chart simplifies the deployment and configuration of Redpanda Connect, allowing you to manage and scale Redpanda Connect instances in Kubernetes environments.

To get started with the Helm chart, see Get Started with the Redpanda Connect Helm Chart.

Or, see the release notes for the chart on GitHub.

New outputs (beta)

Component updates

This release adds:

Other changes:

  • The collection field of the mongodb output now supports interpolation functions.

  • All components that used the deprecated count function, with a default value for the path field, now use the counter function.

    If you have multiple components that execute a mapping using the deprecated count function with the original default value, and these counters need to cascade, there may be a change in behavior. To avoid this, explicitly define a value for the path field.

Version 4.37.0

Redpanda Connect has new bundled Redpanda Migrator components, more AI processors for Self-Managed and Cloud platforms, and three new outputs.

Redpanda Migrator components (beta)

You can now move workloads from any Apache Kafka system to Redpanda using the redpanda_migrator_bundle input and output.

For full details, see the Redpanda Migrator Cookbook for Self-Managed and Cloud.

AI processors (beta)

Some new additions to Redpanda Connect’s growing list of AI processors:

Certified and community outputs (beta)

Component updates

This release adds:

Version 4.36.0

You can now use the replication_factor_override and replication_factor fields of the redpanda_migrator input and output to set a replication factor for migrated topics.

Versions 4.35.1 to 4.35.3

Redpanda Connect is now fully integrated with the Redpanda Cloud platform. This release adds Azure and GCP components to the growing list of available components. For a full list of components supported in Redpanda Cloud, see the Connector Catalog.

Version 4.35.0

Redpanda Connect now includes beta versions of Redpanda Migrator components:

For full details, see the Redpanda Migrator Cookbook for Self-Managed and Cloud.

Component updates

This release adds:

Version 4.34.0

Redpanda Connect now includes beta versions of two new AI processors for Cloud and Self-Managed platforms:

Version 4.33.0

Redpanda Connect now includes beta versions of four new components.

For Self-Managed and Cloud platforms

For Self-Managed only

You can also use the new Bloblang vector method to create vector embeddings for SQL databases.

Component updates

This release adds:

  • The credentials_json field to all GCP components.

  • The content_md5 field to the aws_s3 output.

  • The send_ack field to the nats input.

Version 4.32.1

Redpanda Connect has a new batch of beta AI processors for Self-Managed and Cloud platforms.

For Self-Managed only

Version 4.31.0

Redpanda Connect now includes beta versions of two new components:

Component updates

This release adds:

Version 4.30.0 to 4.30.1

Redpanda Connect now includes a beta version of the splunk input.

Version 4.29.0

Redpanda Connect now separates packages into an enterprise bundle and a free bundle, which contains community and selected certified connectors. To learn more about licensing and support levels, see Connector Support Levels.

Component updates

This release adds: