Connect

Components Catalog

Use the following table to search for available inputs, outputs, and processors.

For a description of the support levels, see Support levels.

Name Connector Type Support Level Enterprise Licensed Available in Cloud

a2a_message

Processor

Certified

Yes

Yes

amqp_0_9RabbitMQ AMQP

Input, Output

Certified

No

Yes

amqp_1RabbitMQ AMQP Apache Qpid

Input, Output

Community

No

No

arc

Output

Community

No

Yes

archiveZIP TAR GZIP

Processor

Certified

No

Yes

avro

Processor, Scanner

Community

No

Yes

awk

Processor

Community

No

No

aws_bedrock_chatAmazon AWS Bedrock Chat

Processor

Certified

No

Yes

aws_bedrock_embeddingsAmazon AWS Bedrock Embeddings

Processor

Certified

No

Yes

aws_cloudwatch

Metric

Community

No

No

aws_cloudwatch_logsAWS CloudWatch Logs Amazon CloudWatch Logs

Input

Community

No

Yes

aws_dynamodbAWS DynamoDB Amazon DynamoDB DynamoDB

Cache, Output

Community

No

Yes

aws_dynamodb_cdcAmazon DynamoDB CDC

Input

Certified

Yes

Yes

aws_dynamodb_partiqlAmazon AWS DynamoDB PartiQL

Processor

Certified

No

Yes

aws_kinesisAWS Kinesis Amazon Kinesis Kinesis

Input, Output

Certified

No

Yes

aws_kinesis_firehoseAWS Kinesis Firehose Amazon Kinesis Firehose Kinesis Firehose

Output

Certified

No

Yes

aws_lambdaAWS Lambda Amazon Lambda Lambda

Processor

Certified

No

Yes

aws_s3AWS S3 Amazon S3 S3 Simple Storage Service

Cache, Input, Output

Certified

No

Yes

aws_snsAWS SNS Amazon SNS SNS Simple Notification Service

Output

Community

No

Yes

aws_sqsAWS SQS Amazon SQS SQS Simple Queue Service

Input, Output

Certified

No

Yes

azure_blob_storageAzure Blob Storage Microsoft Azure Storage

Input, Output

Certified

No

Yes

azure_cosmosdbMicrosoft Azure Azure

Input, Output, Processor

Certified

No

Yes

azure_data_lake_gen2Microsoft Azure Azure

Output

Certified

No

Yes

azure_queue_storageAzure Queue Storage Microsoft Azure Queue

Input, Output

Certified

No

Yes

azure_table_storageAzure Table Storage Microsoft Azure Table

Input, Output

Certified

No

Yes

batched

Input

Certified

No

Yes

beanstalkd

Input, Output

Community

No

No

benchmark

Processor

Certified

No

Yes

bloblang

Processor

Certified

No

Yes

bounds_check

Processor

Certified

No

Yes

branch

Processor

Certified

No

Yes

broker

Input, Output

Certified

No

Yes

cache

Output, Processor

Certified

No

Yes

cached

Processor

Certified

No

Yes

cassandraApache Cassandra

Input, Output

Community

No

No

catch

Processor

Certified

No

Yes

chunker

Scanner

Certified

No

Yes

cockroachdb_changefeed

Input

Community

No

No

cohere_chat

Processor

Certified

No

Yes

cohere_embeddings

Processor

Certified

No

Yes

cohere_rerank

Processor

Certified

No

Yes

command

Processor

Certified

No

No

compress

Processor

Certified

No

Yes

couchbase

Cache, Output, Processor

Community

No

No

crash

Processor

Certified

No

No

csvComma-Separated Values

Input, Scanner

Certified

No

Yes

cyborgdb

Output

Community

No

Yes

cypher

Output

Community

No

No

decompress

Processor, Scanner

Certified

No

Yes

dedupe

Processor

Certified

No

Yes

discord

Input, Output

Community

No

No

drop

Output

Certified

No

Yes

drop_on

Output

Certified

No

Yes

dynamic

Input, Output

Community

No

No

elasticsearch_v8

Output

Certified

No

Yes

elasticsearch_v9

Output

Community

No

No

fallback

Output

Certified

No

Yes

ffiForeign Function Interface

Processor

Certified

No

No

file

Cache, Input, Output

Certified

No

No

for_each

Processor

Certified

No

Yes

gateway

Input

Certified

Yes

Yes

gcp_bigqueryGCP BigQuery Google BigQuery BigQuery

Output

Certified

No

Yes

gcp_bigquery_selectGCP BigQuery Google Cloud GCP

Input, Processor

Certified

No

Yes

gcp_bigquery_write_apiGCP BigQuery

Output

Certified

Yes

Yes

gcp_cloud_storageGCP Cloud Storage Google Cloud Storage GCS

Cache, Input, Output

Certified

No

Yes

gcp_cloudtraceGCP Cloud Trace

Tracer

Certified

No

Yes

gcp_pubsubGCP PubSub Google Cloud Pub/Sub GCP Pub/Sub Google Pub/Sub

Input, Output

Certified

No

Yes

gcp_spanner_cdcGoogle Cloud GCP

Input

Certified

Yes

Yes

gcp_vertex_ai_chatGCP Vertex AI Google Cloud GCP

Processor

Certified

No

Yes

gcp_vertex_ai_embeddingsGoogle Cloud GCP

Processor

Certified

No

Yes

generate

Input

Certified

No

Yes

git

Input

Certified

No

Yes

google_drive_download

Processor

Certified

Yes

Yes

google_drive_list_labels

Processor

Certified

Yes

Yes

google_drive_search

Processor

Certified

Yes

Yes

grok

Processor

Community

No

No

group_by

Processor

Certified

No

Yes

group_by_value

Processor

Certified

No

Yes

hdfs

Input, Output

Community

No

No

http

Processor

Certified

No

Yes

http_clientHTTP REST API REST

Input, Output

Certified

No

Yes

http_serverHTTP REST API REST Gateway

Input, Output

Certified

No

Yes

icebergApache Iceberg Apache Polaris AWS Glue Databricks Unity Catalog

Output

Certified

Yes

Yes

influxdb

Metric

Community

No

No

inproc

Input, Output

Certified

No

Yes

insert_part

Processor

Certified

No

Yes

jaeger

Tracer

Community

No

No

javascript

Processor

Certified

No

No

jira

Processor

Certified

No

Yes

jmespath

Processor

Certified

No

Yes

jq

Processor

Certified

No

Yes

json_api

Metric

Certified

No

No

json_array

Scanner

Community

No

Yes

json_documents

Scanner

Certified

No

Yes

json_schemaJSON Schema

Processor

Certified

No

Yes

kafkaApache Kafka

Input, Output

Certified

No

Yes

kafka_franzApache Kafka Kafka

Input, Output

Certified

No

Yes

lines

Scanner

Certified

No

Yes

local

Rate_limit

Certified

No

Yes

log

Processor

Certified

No

Yes

logger

Metric

Certified

No

No

lru

Cache

Community

No

Yes

mapping

Processor

Certified

No

Yes

memcached

Cache

Community

No

Yes

memory

Buffer, Cache

Certified

No

Yes

metric

Processor

Certified

No

Yes

microsoft_sql_server_cdc

Input

Certified

Yes

Yes

mongodbMongo

Cache, Input, Output, Processor

Certified

No

Yes

mongodb_cdcMongoDB CDC

Input

Certified

Yes

Yes

mqtt

Input, Output

Certified

No

Yes

msgpack

Processor

Community

No

No

multilevel

Cache

Certified

No

Yes

mutation

Processor

Certified

No

Yes

mysql_cdc

Input

Certified

Yes

Yes

nanomsg

Input, Output

Community

No

No

natsNATS.io

Input, Output

Certified

No

Yes

nats_jetstreamNATS JetStream NATS

Input, Output

Certified

No

Yes

nats_kvNATS KV

Cache, Input, Output, Processor

Certified

No

Yes

nats_request_replyNATS Request Reply

Processor

Certified

No

Yes

nats_streamNATS Stream NATS Streaming NATS

Input, Output

Community

No

No

none

Buffer, Metric, Tracer

Certified

No

Yes

noop

Cache, Processor

Certified

No

Yes

nsq

Input, Output

Community

No

No

ockam_kafka

Input, Output

Community

No

No

ollama_chat

Processor

Certified

No

No

ollama_embeddings

Processor

Certified

No

No

ollama_moderation

Processor

Certified

No

No

open_telemetry_collectorOpenTelemetry

Metric, Tracer

Certified

Yes

Yes

openai_chat_completion

Processor

Certified

No

Yes

openai_embeddings

Processor

Certified

No

Yes

openai_image_generation

Processor

Certified

No

Yes

openai_speech

Processor

Certified

No

Yes

openai_transcription

Processor

Certified

No

Yes

openai_translation

Processor

Certified

No

Yes

opensearch

Output

Certified

No

Yes

oracledb_cdcOracle CDC OracleDB CDC Oracle Database CDC

Input

Certified

Yes

Yes

otlp_grpcOpenTelemetry OTLP OTel gRPC

Input, Output

Certified

Yes

Yes

otlp_httpOpenTelemetry OTLP OTel

Input, Output

Certified

Yes

Yes

parallel

Processor

Certified

No

Yes

parquet

Input, Processor

Certified: input

Community: processor

No

No

parquet_decode

Processor

Certified

No

Yes

parquet_encode

Processor

Certified

No

Yes

parse_log

Processor

Community

No

Yes

pg_stream

Certified

Yes

Yes

pinecone

Output

Certified

No

Yes

postgres_cdc

Input

Certified

Yes

Yes

processors

Processor

Certified

No

Yes

prometheus

Metric

Certified

No

Yes

protobuf

Processor

Certified

No

No

pulsarApache Pulsar

Input, Output

Community

No

No

pusher

Output

Community

No

No

qdrant

Output, Processor

Certified

No

Yes

questdb

Output

Certified

No

Yes

rate_limit

Processor

Certified

No

Yes

re_match

Scanner

Certified

No

Yes

read_until

Input

Certified

No

Yes

redis

Cache, Processor, Rate_limit

Certified

No

Yes

redis_hashRedis Hash Redis

Output

Certified

No

Yes

redis_listRedis List Redis Lists Redis

Input, Output

Certified

No

Yes

redis_pubsubRedis PubSub Redis Pub/Sub Redis

Input, Output

Certified

No

Yes

redis_scanRedis

Input

Certified

No

Yes

redis_scriptRedis Script

Processor

Certified

No

Yes

redis_streamsRedis Streams Redis

Input, Output

Certified

No

Yes

redpanda

Cache, Input, Output, Tracer

Certified

No

Yes

redpanda_common

Input, Output

Certified

Yes

Yes

redpanda_data_transform

Processor

Certified

No

No

redpanda_migrator

Input, Output

Certified

No

Yes

reject

Output

Certified

No

Yes

reject_errored

Output

Certified

No

Yes

resource

Input, Output, Processor

Certified

No

Yes

retry

Output, Processor

Certified

No

Yes

ristretto

Cache

Community

No

Yes

salesforce

Input

Certified

Yes

Yes

salesforce_cdcSalesforce Salesforce CDC

Input

Certified

Yes

Yes

salesforce_graphqlSalesforce Salesforce GraphQL

Input

Certified

Yes

Yes

salesforce_sinkSalesforce Salesforce Sink

Output

Certified

Yes

Yes

schema_registry

Input, Output

Certified

No

Yes

schema_registry_decode

Processor

Certified

No

Yes

schema_registry_encode

Processor

Certified

No

Yes

select_parts

Processor

Certified

No

Yes

sentry_capture

Processor

Community

No

No

sequence

Input

Certified

No

Yes

sftp

Input, Output

Certified

No

Yes

skip_bom

Scanner

Certified

No

Yes

slack

Input

Certified

Yes

Yes

slack_postSlack Post

Output

Certified

Yes

Yes

slack_reactionSlack Reaction

Output

Certified

Yes

Yes

slack_threadSlack Thread

Processor

Certified

Yes

Yes

slack_usersSlack Users

Input

Certified

Yes

Yes

sleep

Processor

Certified

No

Yes

snowflake_putSnowflake

Output

Certified

Yes

Yes

snowflake_streamingSnowflake Streaming

Output

Certified

Yes

Yes

socket

Input, Output

Certified

No

No

socket_server

Input

Certified

No

No

spicedb_watch

Input

Community

No

Yes

split

Processor

Certified

No

Yes

splunk

Input

Certified

Yes

Yes

splunk_hecSplunk

Output

Certified

Yes

Yes

sql

Cache, Output, Processor

Certified: cache

Community: output, processor

No

Yes

sql_driver_clickhouseClickHouse

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

Yes

sql_driver_gocosmosAzure Cosmos DB

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

No

sql_driver_mssqlMicrosoft SQL Server

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

No

sql_driver_mysqlMYSQL

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

Yes

sql_driver_oracleOracle

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

Yes

sql_driver_postgresPostgreSQL

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

Yes

sql_driver_snowflakeSnowflake

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

No

sql_driver_sqliteSQLite

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

Yes

sql_driver_trinoTrino

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

No

sql_insertSQL PostgreSQL MySQL Microsoft SQL Server ClickHouse Trino

Output, Processor

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

Yes

sql_rawSQL PostgreSQL MySQL Microsoft SQL Server ClickHouse Trino

Input, Output, Processor

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

Yes

sql_selectSQL PostgreSQL MySQL Microsoft SQL Server ClickHouse Trino

Input, Processor

Certified: MYSQL, Oracle, PostgreSQL, SQLite
Community: ClickHouse, Azure Cosmos DB, Microsoft SQL Server, Snowflake, Trino

No

Yes

sqlite

Buffer

Community

No

No

statsd

Metric

Certified

No

No

stdin

Input

Certified

No

No

stdout

Output

Certified

No

No

string_split

Processor

Certified

No

Yes

subprocess

Input, Output, Processor

Community

No

No

switch

Output, Processor, Scanner

Certified

No

Yes

sync_response

Output, Processor

Certified

No

Yes

system_window

Buffer

Certified

No

Yes

tar

Scanner

Certified

No

Yes

text_chunker

Processor

Certified

No

Yes

tigerbeetle_cdc

Input

Certified

No

No

timeplus

Input, Output

Community

No

Yes

to_the_end

Scanner

Certified

No

Yes

try

Processor

Certified

No

Yes

ttlru

Cache

Community

No

Yes

twitter_search

Input

Community

No

No

unarchiveZIP TAR GZIP Archive

Processor

Certified

No

Yes

wasm

Processor

Community

No

No

websocket

Input, Output

Certified

No

No

while

Processor

Certified

No

Yes

workflow

Processor

Certified

No

Yes

xml

Processor

Community

No

Yes

zmq4

Input, Output

Community

No

No

About Components

Every Redpanda Connect pipeline has at least one input, an optional buffer, an output and any number of processors:

input:
  kafka:
    addresses: [ TODO ]
    topics: [ foo, bar ]
    consumer_group: foogroup

buffer:
  type: none

pipeline:
  processors:
  - mapping: |
      message = this
      meta.link_count = links.length()

output:
  aws_s3:
    bucket: TODO
    path: '${! meta("kafka_topic") }/${! json("message.id") }.json'

These are the main components within Redpanda Connect and they provide the majority of useful behavior.

Observability components

There are also the observability components: http, logger, metrics, and tracing, which allow you to specify how Redpanda Connect exposes observability data.

http:
  address: 0.0.0.0:4195
  enabled: true
  debug_endpoints: false

logger:
  format: json
  level: WARN

metrics:
  statsd:
    address: localhost:8125
    flush_period: 100ms

tracer:
  jaeger:
    agent_address: localhost:6831

Resource components

Finally, there are caches and rate limits. These are components that are referenced by core components and can be shared.

input:
  http_client: # This is an input
    url: TODO
    rate_limit: foo_ratelimit # This is a reference to a rate limit

pipeline:
  processors:
    - cache: # This is a processor
        resource: baz_cache # This is a reference to a cache
        operator: add
        key: '${! json("id") }'
        value: "x"
    - mapping: root = if errored() { deleted() }

rate_limit_resources:
  - label: foo_ratelimit
    local:
      count: 500
      interval: 1s

cache_resources:
  - label: baz_cache
    memcached:
      addresses: [ localhost:11211 ]

It’s also possible to configure inputs, outputs and processors as resources which allows them to be reused throughout a configuration with the resource input, resource output and resource processor respectively.

For more information about any of these component types check out their sections:

Support levels

Each connector has a corresponding level of support, which determines whether it is suitable for production environments or community-driven projects.

  • Certified: Production-ready and available to purchase with Redpanda support.

  • Community: Not considered production-ready and available only for community support in Slack.

Anyone can use the Certified and Community connectors. However, to use Enterprise connectors, you need an Enterprise Edition license.