Kafka Connect

Use Kafka Connect to integrate your Redpanda data with different data systems. As managed solutions, connectors offer a simpler way to integrate your data than manually creating a solution with the Kafka API. You can set up and manage these connectors in Redpanda Console. All connectors are managed by Redpanda.

Each connector is either a source or a sink:

• A source connector imports data from a source system into a Redpanda cluster. The source connector’s main task is to fetch data from these sources and convert them into a format suitable for Redpanda.

• A sink connector exports data from a Redpanda cluster and pushes it into a target system. Sink connectors read the data from Redpanda and transform it into a format that the target system can use.

These sources and sinks work together to create a data pipeline that can move and transform data from one system to another.

• Converters and Serialization

  Use converters to handle the serialization and deserialization of data between a Redpanda topic and a managed connector.

• Monitor Connectors

  Use metrics to monitor the health of your Redpanda managed connectors.

• Single Message Transforms

  Single Message Transforms (SMTs) let you modify the data and its characteristics as it passes through a connector.

• Sizing Connectors

  How to choose number of tasks to set for a connector.

• Create an S3 Sink Connector

  Use the Redpanda Cloud UI to create an AWS S3 Sink Connector.

• Create a Google BigQuery Sink Connector

  Use the Redpanda Cloud UI to create a Google BigQuery Sink Connector.

• Create a GCS Sink Connector

  Use the Redpanda Cloud UI to create a GCS Sink Connector.

• Create an Iceberg Sink Connector

  Use the Redpanda Cloud UI to create an Iceberg Sink Connector.

• Create a JDBC Sink Connector

  Use the Redpanda Cloud UI to create a JDBC Sink Connector.

• Create a JDBC Source Connector

  Use the Redpanda Cloud UI to create a JDBC Source Connector.

• Create a MirrorMaker2 Source Connector

  Use the Redpanda Cloud UI to create a MirrorMaker2 Source Connector.

• Create a MirrorMaker2 Checkpoint Connector

  Use the Redpanda Cloud UI to create a MirrorMaker2 Checkpoint Connector.

• Create a MirrorMaker2 Heartbeat Connector

  Use the Redpanda Cloud UI to create a MirrorMaker2 Heartbeat Connector.

• Create a MongoDB Sink Connector

  Use the Redpanda Cloud UI to create a MongoDB Sink Connector.

• Create a MongoDB Source Connector

  Use the Redpanda Cloud UI to create a MongoDB Source Connector.

• Create a MySQL (Debezium) Source Connector

  Use the Redpanda Cloud UI to create a MySQL (Debezium) Source Connector.

• Create a PostgreSQL (Debezium) Source Connector

  Use the Redpanda Cloud UI to create a PostgreSQL (Debezium) Source Connector.

• Create a SQL Server (Debezium) Source Connector

  Use the Redpanda Cloud UI to create a SQL Server (Debezium) Source Connector.

• Create a Snowflake Sink Connector

  Use the Redpanda Cloud UI to create a Snowflake Sink Connector.