Managed Connectors

Redpanda Connectors provide a way to integrate your Redpanda data with different data systems. As managed solutions, connectors offer a simpler way to integrate your data than manually creating a solution with the Kafka API. You can set up and manage these connectors in Redpanda Console. All connectors are managed by Redpanda.

Each connector is either a source or a sink:

• A source connector imports data from a source system into a Redpanda cluster. The source could be a cloud storage system such as S3 or GCS. The source connector's main task is to fetch data from these sources and convert them into a format suitable for Redpanda.

• A sink connector exports data from a Redpanda cluster and pushes it into a target system. The target could be a cloud storage system such as S3 or GCS. Sink connectors read the data from Redpanda and transform it into a format that the target system can use.

These sources and sinks work together to create a data pipeline that can move and transform data from one system to another.

• Amazon S3 Sink Connector

  The Amazon S3 Sink connector stores Apache Kafka messages in an AWS S3 bucket.

• Google BigQuery Sink Connector

  The Google BigQuery Sink connector enables you to pipe any structured data from Redpanda to BigQuery tables for analytics and decision-making.

• Google Cloud Storage Sink Connector

  The Google Cloud Storage (GCS) Sink connector stores Redpanda messages in a Google Cloud Storage bucket.

• HTTP Source Connector

  The HTTP Source connector enables change data capture (CDC) from JSON/HTTP APIs into Redpanda.

• JDBC Sink Connector

  The JDBC Sink Connector enables you to transfer structured data from Redpanda into a relational database.

• JDBC Source Connector

  The JDBC Source Connector enables you to transfer data from a relational database into Redpanda topics.

• MirrorMaker2 Source Connector

  The MirrorMaker2 Source connector enables you to import messages from another Kafka cluster.

• MirrorMaker2 Checkpoint Source Connector

  The MirrorMaker2 Checkpoint connector enables you to import consumer group offsets from other Kafka clusters.

• MirrorMaker2 Heartbeat Source Connector

  The MirrorMaker2 Heartbeat Source connector enables you to generate and send heartbeat messages to a local cluster's heartbeat topic.

• MongoDB Sink Connector

  The MongoDB Sink connector enables you to store Redpanda structured data in MongoDB.

• MongoDB Source Connector

  The MongoDB Source Connector imports collections from MongoDB database.

• MySQL (Debezium) Source Connector

  The MySQL (Debezium) Source connector enables you to import a stream of changes from MySQL, AmazonRDS, and Amazon Aurora relational databases.

• PostgreSQL (Debezium) Source Connector

  The PostgreSQL (Debezium) Source connector enables you to import updates to Redpanda from PostgreSQL.

• Snowflake Sink Connector

  The Snowflake Sink connector ingests and stores Redpanda structured data into a Snowflake database for analytics and decision-making.