Docs Connect Get Started What’s New What’s New in Redpanda Connect This topic includes new content added from version 4.29.0 onwards. For a full list of product updates, and details of updates prior to version 4.29.0, see the Redpanda Connect release notes. For new Cloud-specific features, see What’s New in Redpanda Cloud. Version 4.46.0 This release introduces a change data capture (CDC) input for MySQL, and enhancements to the Bloblang playground, making it easier to test and debug your pipeline configurations. The enhancements include: A Run button in Bloblang code snippets, which opens each snippet in the playground with a pre-filled input and mapping ready for testing. A Get help from AI button, which appears in the playground when an error occurs. Components for Cloud and Self-Managed Redpanda Connect (beta) mysql_cdc input Component updates This release adds: The instance_id field to the kafka, kafka_franz, ockam_kafka, redpanda, redpanda_common, and redpanda_migrator inputs. The rebalance_timeout, session_timeout, and heartbeat_interval fields to the kafka_franz, ockam_kafka, redpanda, redpanda_common, and redpanda_migrator inputs. The preserve_logical_types and raw_unions fields (for Avro messages), and a schema_id metadata field to the schema_registry_decode processor. The unchanged_toast_value field to the postgres_cdc input. A processors field to the schema_evolution object in the snowflake_streaming output. New default values to the max_message_bytes and broker_write_max_bytes fields in the kafka_franz, ockam_kafka, redpanda, redpanda_migrator, and redpanda_migrator_offsets outputs, using IEC instead of SI units. Other changes: The postgres_cdc input no longer supports PostgreSQL 10 or 11. Deprecations All changes are backward-compatible. No action is required if you are using an earlier version of Redpanda Connect. Deprecated fields Affected components new_column_type_mapping snowflake_streaming output Version 4.45.0 Highlights of this update include the beta release of two components for Self-Managed and Cloud Redpanda Connect, and a new cookbook that explains how to ingest data from a Redpanda topic into Snowflake using Snowpipe Streaming. You can also now share links to code snippets in the Bloblang playground using the Share link button. Components for Cloud and Self-Managed Redpanda Connect (beta) ollama_moderation processor (Self-Managed, Cloud) redpanda_migrator_offsets input (Self-Managed, Cloud) Component updates This release adds: The max_outstanding_messages and message_timeout fields to the aws_sqs input. The content_type field to the amqp_1 output. The offset_token field to the snowflake_streaming output. The save_prompt_metadata, max_tool_calls fields and tools object to the ollama_chat processor. The offset_topic, offset_group, offset_partition, offset_commit_timestamp and offset_metadata fields to the redpanda_migrator_offsets output. The topic_lag_refresh_period field to the redpanda and redpanda_common inputs. The redpanda_lag metric to the redpanda and redpanda_common inputs. The fetch_max_wait field to the kafka_franz, ockam_kafka, redpanda, redpanda_common, and redpanda_migrator inputs. The kafka_lag metadata field to the redpanda and redpanda_common inputs. The avro_schema and avro_schema_fingerprint metadata fields to the avro scanner. The label field to template test definitions. The bloblang scalar type to template fields. Redpanda Connect now supports: Interpolation functions for channel and table names in the snowflake_streaming output. Calls to external tools when using the ollama_chat processor. The execution of multiple database queries as transactions when using the sql_raw input, output and processor. Using the label metadata field within a template mapping to retrieve the template component label set in the corresponding configuration. The following Bloblang functions for error handling: error_source_name, error_source_label, and error_source_path. Deprecations All changes are backward-compatible. No action is required if you are using an earlier version of Redpanda Connect. Deprecated fields Affected components batching redpanda_migrator input, redpanda_migrator output kafka_key, max_in_flight redpanda_migrator_offsets output batch_size, multi_header, replication_factor, replication_factor_override, output_resource redpanda_migrator input Version 4.44.0 No documentation updates were made. Version 4.43.1 This release introduces support for trial licenses for Self-Managed Redpanda Connect. Version 4.43.0 This release renames the pg_stream input to postgres_cdc. The postgres_cdc input also no longer emits mode metadata, but sets the operation metadata field to read when a snapshot of a database is processed. Version 4.42.0 This release introduces a Bloblang playground where you can try out your Bloblang syntax and get immediate feedback. It also updates Redpanda Connect licensing checks for Redpanda Connect. Starting from version 4.42.0, you must add your Enterprise Edition license to Redpanda Connect before using any Enterprise connectors. Component updates Redpanda Connect now supports: A Google Cloud spanner driver for all SQL plugins. The following complex data types for the pg_stream input: JSONB, TEXT[], INET, TSVECTOR, TSRANGE, POINT, INTEGER[]. Parquet files for the bigquery output. This release also adds an exists operator to the cache processor. Version 4.41.0 This release adds the max_records_per_request field to the aws_sqs output. Packaging update Starting from version 4.41.0, the -cgo suffixed Docker image is no longer available due to low demand and the unacceptable cadence with which the image base (Debian) receives security updates. To create your own CGO builds, run the following command: CGO_ENABLED=1 make TAGS=x_benthos_extra redpanda-connect. Version 4.40.0 This update includes a beta release of: The pg_stream input, which captures data changes made to a PostgreSQL database in real time and streams them to Redpanda Connect. The benchmark processor, which provides message throughput statistics for a data pipeline. It also introduces a new Bloblang method for timestamps. Components for Cloud and Self-Managed Redpanda Connect (beta) pg_stream input benchmark processor Component updates This release adds: The metadata_max_age field to the redpanda_migrator_offsets output. The timestamp_ms field to the following outputs: ockam_kafka, kafka, kafka_franz, redpanda, redpanda_common, redpanda_migrator. The kafka_timestamp_ms metadata field, which uses a Unix timestamp with millisecond precision, to the following inputs: kafka, kafka_franz, redpanda, redpanda_common, redpanda_migrator. The translate_schema_ids and schema_registry_output_resource to the redpanda_migrator output. The backfill_dependencies and input_resource fields to the schema_registry output. Options to optimize the build of the data output from the snowflake_streaming output. Other changes: The following components now use the Franz Kafka Schema Registry client library: schema_registry input and output, and the schema_registry_encode and schema_registry_decode processors. The snowflake_streaming output can now automatically create tables in Snowflake, when the schema_evolution.enabled field is set to true. Deprecations Deprecated field: kafka_timestamp metadata field Affected components: redpanda input redpanda_common input Replaced by: kafka_timestamp_ms metadata field Deprecated field: timestamp Affected components: ockam_kafka output kafka output kafka_franz output redpanda output redpanda_common output redpanda_migrator output Replaced by: timestamp_ms field Action required: Modify all systems that process these fields to ensure compatibility with the new field names. Version 4.39.0 Highlights of this update include the beta release of three new components for Self-Managed and Cloud, including a new snowflake_streaming output. This release also includes additions to the redpanda family of components and introduces new features for Self-Managed. Components for Cloud and Self-Managed (beta) snowflake_streaming output timeplus input spicedb_watch input Additional redpanda components (beta) redpanda (input and output): Consume message data from or send message data to Kafka brokers. redpanda_common (input and output): Consume message data from or send message data to a Redpanda Kafka broker, using credentials from a common redpanda configuration block (Self-Managed only). Features in Self-Managed You can now: Create an allow list or deny list for a Redpanda Connect instance Use the rpk connect CLI flag --secrets to look up secrets on a remote system Use the rpk connect CLI flag --disable-telemetry to block the telemetry service Component updates This release adds: The follow_redirects field to the http processor. The following fields to the redpanda configuration service: metadata_max_age partitioner idempotent_write compression broker_write_max_bytes The metadata_max_age field to the kafka_franz output. The following fields to the kafka_franz and ockam_kafka inputs: fetch_max_bytes fetch_min_bytes fetch_max_partition_bytes The url field of the aws_sqs output now supports interpolation functions. Deprecations All changes are backward-compatible. No action is required if you are using an earlier version of Redpanda Connect. Deprecated fields Affected components auto_replay_nacks, sasl, metadata_max_age ockam_kafka input client_id, rack_id, sasl ockam_kafka output rack_id kafka_franz output Version 4.38.0 This update includes the beta release of two new outputs for Cloud and Self-Managed, and the introduction of a telemetry service, which collects and sends anonymized usage statistics to Redpanda. New tooling A new Helm chart for deploying Redpanda Connect in Kubernetes is now available. This chart simplifies the deployment and configuration of Redpanda Connect, allowing you to manage and scale Redpanda Connect instances in Kubernetes environments. To get started with the Helm chart, see Get Started with the Redpanda Connect Helm Chart. Or, see the release notes for the chart on GitHub. New outputs (beta) azure_data_lake_gen2 (certified) timeplus (community) Component updates This release adds: The checksum_algorithm to the aws_s3 output. The private_key field to the snowflake_put output. The nkey field to the following components: nats (input, output) nats_jetstream (input, output) nats_kv (input, output, processor) nats_stream (input, output) nats_request_reply (processor) Other changes: The collection field of the mongodb output now supports interpolation functions. All components that used the deprecated count function, with a default value for the path field, now use the counter function. If you have multiple components that execute a mapping using the deprecated count function with the original default value, and these counters need to cascade, there may be a change in behavior. To avoid this, explicitly define a value for the path field. Version 4.37.0 Redpanda Connect has new bundled Redpanda Migrator components, more AI processors for Self-Managed and Cloud platforms, and three new outputs. Redpanda Migrator components (beta) You can now move workloads from any Apache Kafka system to Redpanda using the redpanda_migrator_bundle input and output. For full details, see the Redpanda Migrator Cookbook for Self-Managed and Cloud. AI processors (beta) Some new additions to Redpanda Connect’s growing list of AI processors: aws_bedrock_embeddings cohere_chat cohere_embeddings gcp_vertex_ai_embeddings Certified and community outputs (beta) couchbase (community) cypher (community) questdb (certified) Component updates This release adds: The metadata_max_age field to kafka_franz and redpanda_migrator inputs. The fetch_in_order field to the schema_registry input. The encrypted_fields fields to the ockam_kafka input and output. Version 4.36.0 You can now use the replication_factor_override and replication_factor fields of the redpanda_migrator input and output to set a replication factor for migrated topics. Versions 4.35.1 to 4.35.3 Redpanda Connect is now fully integrated with the Redpanda Cloud platform. This release adds Azure and GCP components to the growing list of available components. For a full list of components supported in Redpanda Cloud, see the Connector Catalog. Version 4.35.0 Redpanda Connect now includes beta versions of Redpanda Migrator components: redpanda_migrator input and output redpanda_migrator_bundle input and output redpanda_migrator_offsets output For full details, see the Redpanda Migrator Cookbook for Self-Managed and Cloud. Component updates This release adds: OAuth, basic authentication, and JSON Web Token fields to the schema_registry input and output. The job_project field to the gcp_bigquery output. The broker_write_max_bytes field to the kafka_franz and ockam_kafka outputs. Version 4.34.0 Redpanda Connect now includes beta versions of two new AI processors for Cloud and Self-Managed platforms: aws_bedrock_chat gcp_vertex_ai_chat Version 4.33.0 Redpanda Connect now includes beta versions of four new components. For Self-Managed and Cloud platforms schema_registry input and output qdrant output For Self-Managed only ockam_kafka input and output You can also use the new Bloblang vector method to create vector embeddings for SQL databases. Component updates This release adds: The credentials_json field to all GCP components. The content_md5 field to the aws_s3 output. The send_ack field to the nats input. Version 4.32.1 Redpanda Connect has a new batch of beta AI processors for Self-Managed and Cloud platforms. For Self-Managed and Cloud platforms openai_chat_completion openai_embeddings openai_image_generation openai_speech openai_transcription openai_translation For Self-Managed only ollama_chat ollama_embeddings Version 4.31.0 Redpanda Connect now includes beta versions of two new components: pinecone output - Redpanda Connect’s first AI component redpanda_data_transform processor Component updates This release adds: Custom TLS fields for the splunk input and splunk_hec output. The timestamp field to kafka and kafka_franz outputs. The status_topic field to the redpanda component. Version 4.30.0 to 4.30.1 Redpanda Connect now includes a beta version of the splunk input. Version 4.29.0 Redpanda Connect now separates packages into an enterprise bundle and a free bundle, which contains community and selected certified connectors. To learn more about licensing and support levels, see Connector Support Levels. Component updates This release adds: The auth.oauth2.scope field to the pulsar input and output. The subscription_initial_position field to the pulsar input. Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Introduction to Redpanda Connect Quickstarts