Kafka Connect

    Use Kafka Connect to integrate your Redpanda data with different data systems. As managed solutions, connectors offer a simpler way to integrate your data than manually creating a solution with the Kafka API. You can set up and manage these connectors for BYOC and Dedicated clusters in the Redpanda Cloud UI. Note that you remain responsible for monitoring, alerting, and restarting Kafka Connect connectors, as these tasks are not managed or monitored by Redpanda support.

    Kafka Connect is automatically enabled on AWS and GCP clusters, which means there is a node running for Kafka Connect even if connectors are not used. To enable Kafka Connect on Azure clusters, see Enable Kafka Connect.

    Each connector is either a source or a sink:

    • A source connector imports data from a source system into a Redpanda cluster. The source connector’s main task is to fetch data from these sources and convert them into a format suitable for Redpanda.

    • A sink connector exports data from a Redpanda cluster and pushes it into a target system. Sink connectors read the data from Redpanda and transform it into a format that the target system can use.

    These sources and sinks work together to create a data pipeline that can move and transform data from one system to another.

  • Converters and Serialization

    Use converters to handle the serialization and deserialization of data between a Redpanda topic and a managed connector.

  • Monitor Connectors

    Use metrics to monitor the health of your Redpanda managed connectors.

  • Single Message Transforms

    Single Message Transforms (SMTs) let you modify the data and its characteristics as it passes through a connector.

  • Sizing Connectors

    How to choose number of tasks to set for a connector.

  • Create an S3 Sink Connector

    Use the Redpanda Cloud UI to create an AWS S3 Sink Connector.

  • Create a Google BigQuery Sink Connector

    Use the Redpanda Cloud UI to create a Google BigQuery Sink Connector.

  • Create a GCS Sink Connector

    Use the Redpanda Cloud UI to create a GCS Sink Connector.

  • Create an Iceberg Sink Connector

    Use the Redpanda Cloud UI to create an Iceberg Sink Connector.

  • Create a JDBC Sink Connector

    Use the Redpanda Cloud UI to create a JDBC Sink Connector.

  • Create a JDBC Source Connector

    Use the Redpanda Cloud UI to create a JDBC Source Connector.

  • Create a MirrorMaker2 Source Connector

    Use the Redpanda Cloud UI to create a MirrorMaker2 Source Connector.

  • Create a MirrorMaker2 Checkpoint Connector

    Use the Redpanda Cloud UI to create a MirrorMaker2 Checkpoint Connector.

  • Create a MirrorMaker2 Heartbeat Connector

    Use the Redpanda Cloud UI to create a MirrorMaker2 Heartbeat Connector.

  • Create a MongoDB Sink Connector

    Use the Redpanda Cloud UI to create a MongoDB Sink Connector.

  • Create a MongoDB Source Connector

    Use the Redpanda Cloud UI to create a MongoDB Source Connector.

  • Create a MySQL (Debezium) Source Connector

    Use the Redpanda Cloud UI to create a MySQL (Debezium) Source Connector.

  • Create a PostgreSQL (Debezium) Source Connector

    Use the Redpanda Cloud UI to create a PostgreSQL (Debezium) Source Connector.

  • Create a SQL Server (Debezium) Source Connector

    Use the Redpanda Cloud UI to create a SQL Server (Debezium) Source Connector.

  • Create a Snowflake Sink Connector

    Use the Redpanda Cloud UI to create a Snowflake Sink Connector.