Cloud

Kafka Connect

Use Kafka Connect to integrate your Redpanda data with different data systems. As managed solutions, connectors offer a simpler way to integrate your data than manually creating a solution with the Kafka API. You can set up and manage these connectors for BYOC and Dedicated clusters in the Redpanda Cloud UI or Cloud API.

To enable this feature, contact Redpanda Support. To disable this feature, see Disable Kafka Connect.
Redpanda Support does not manage or monitor Kafka Connect. For fully-supported connectors, consider Redpanda Connect.
When Kafka Connect is enabled, there is a dedicated node running even when no connectors are deployed.

Each connector is either a source or a sink:

A source connector imports data from a source system into a Redpanda cluster. The source connector’s main task is to fetch data from these sources and convert them into a format suitable for Redpanda.
A sink connector exports data from a Redpanda cluster and pushes it into a target system. Sink connectors read the data from Redpanda and transform it into a format that the target system can use.

These sources and sinks work together to create a data pipeline that can move and transform data from one system to another.

Modifying the properties of topics that are created and managed by Redpanda applications can cause unexpected errors. This may lead to connector and cluster failures.

Converters and Serialization

Use converters to handle the serialization and deserialization of data between a Redpanda topic and an external system with Kafka Connect.

Monitor Kafka Connect

Use metrics to monitor the health of Kafka Connect.

Disable Kafka Connect

Learn how to disable Kafka Connect using the Cloud API.

Single Message Transforms

Single Message Transforms (SMTs) let you modify the data and its characteristics as it passes through a connector.

Sizing Connectors

How to choose number of tasks to set for a connector.

Create an S3 Sink Connector

Use the Redpanda Cloud UI to create an AWS S3 Sink Connector.