For the complete documentation index, see llms.txt. Component-specific: streaming-full.txt

Redpanda Streaming
  • Docs
  • Data Platform
  • Self-Managed
  • Streaming
Support Community
Labs
Get Help
Support Community
Labs

What do you think of this page?

Let us know more:

Let us contact you about your feedback:

Please complete the “I'm not a robot” checkbox before submitting.

Redpanda
Redpanda
Documentation
Switch product
Open Redpanda Cloud
    • Overview
Streaming
v26.1 v25.3 v25.2 v25.1 (EOL) v24.3 (EOL)
v24.2 (EOL) v24.1 (EOL) v23.3 (EOL)
    • Overview
    • Get Started
      • What’s New
        • Redpanda
        • Operator
        • Helm Charts
      • Introduction to Redpanda
      • How Redpanda Works
      • Introduction to Redpanda Console
      • Quickstarts
        • Redpanda Streaming
        • Redpanda CLI Quickstart
        • Docker Compose Labs
      • Redpanda Licensing
        • Editions and Enterprise Features
        • Add License to Redpanda
          • Linux
          • Kubernetes
          • Use Redpanda Console
        • Add License to Redpanda Console
        • Check Status
          • Use rpk
          • Use Redpanda Operator
          • Use Redpanda Console
          • Use Metrics
        • Disable Enterprise Features
      • Redpanda CLI
        • Introduction to rpk
        • Install or Update rpk
        • rpk Profiles
        • Specify Broker Addresses for rpk
        • Specify Admin API Addresses for rpk
      • Partner Integrations
    • Develop
      • Kafka Compatibility
      • Benchmark Redpanda
      • Use Redpanda with the HTTP Proxy API
      • Topics
        • Manage Topics
        • Manage Cloud Topics
          • Configure Producers for Cloud Topics
      • Edit Topic Configuration
      • Produce Data
        • Configure Producers
        • Idempotent Producers
        • Configure Leader Pinning
      • Consume Data
        • Consumer Offsets
        • Follower Fetching
        • Filter Messages
        • Deserialize Messages
        • Paginate Messages in Redpanda Console
      • Data Transforms
        • Overview
        • Get Started
          • Linux
          • Kubernetes
        • Build
        • Configure
        • Deploy
        • Test
        • Monitor
        • Manage in Redpanda Console
        • Upgrade
        • Versioning and Compatibility
        • Examples
      • Transactions
    • Deploy
      • Redpanda
        • Kubernetes
          • Overview
          • Get Started
            • Azure AKS
            • Amazon EKS
            • Google GKE
            • Local (kind and minikube)
          • Deployment Workflow
          • Requirements and Recommendations
          • Tune Worker Nodes
          • Deploy Redpanda
          • Production Readiness Checklist
          • High Availability
        • Linux
          • Hardware and Software Requirements
          • Deployment Options
            • Deploy for Development
            • Deploy for Production: Automated
            • Deploy for Production: Manual
            • Production Readiness Checklist
          • Sizing Use Cases
          • Sizing Guidelines
          • System Tuning
      • Redpanda Console
        • Kubernetes
          • Requirements
          • Deploy on Kubernetes
        • Linux
          • Requirements
          • Deploy on Linux
      • Redpanda Connect
      • Kafka Connect
        • Deploy on Kubernetes
        • Deploy on Docker
    • Manage
      • Kubernetes
        • Customize the Helm Chart
        • Cluster Properties
        • Manage Topics
        • Cloud Topics
        • Manage Kafka Connect
        • Storage
          • Volume Types
          • Configure Storage
            • PersistentVolume
            • hostPath
            • emptyDir
          • Expand PersistentVolumes
          • Delete PersistentVolumes
        • Tiered Storage
          • Use Tiered Storage
          • Fast Commission and Decommission Brokers in Kubernetes
          • Remote Read Replicas
          • Topic Recovery
          • Whole Cluster Restore
        • Networking and Connectivity
          • Overview
          • Connect to Redpanda
          • Configure Listeners
          • External Access
            • Use a NodePort Service
            • Use LoadBalancer Services
            • Use Custom Services
        • Security
          • TLS Encryption
            • Use cert-manager
            • Use Secrets
          • Authentication and Authorization
            • Enable Authentication
            • Manage Users and ACLs (Operator)
            • Manage Roles and ACLs (Operator)
            • Manage Groups and ACLs (Operator)
            • Manage Schema Registry ACLs (Operator)
          • Audit Logging
        • Rack Awareness
        • Remote Read Replicas
        • Shadowing
          • Configure Shadowing
          • Monitor
          • Failover Runbook
        • Manage Pod Resources
        • Scale
        • Cluster Maintenance
          • Node Pools
          • Enable the PVCUnbinder
          • Decommission Brokers
          • Recovery Mode
          • Rolling Restart
        • Monitor
          • Redpanda
          • Kafka Connect
        • Resilience Testing
      • Cluster Maintenance
        • Configure Cluster Properties
        • Configure Broker Properties
        • Configure Topic Properties
        • Cluster Balancing
        • Continuous Data Balancing
        • Decommission Brokers
        • Maintenance Mode
        • Rolling Restart
        • Audit Logging
          • Sample Audit Log Messages
        • Manage Disk Space
        • About Client Throughput Quotas
        • Manage Throughput
        • Compaction Settings
        • Configure Client Connections
        • Forced Partition Recovery
        • Node-wise Partition Recovery
      • Security
        • Authentication
        • Authorization
          • Role-Based Access Control (RBAC)
          • Group-Based Access Control (GBAC)
          • Access Control Lists (ACLs)
        • FIPS Compliance
        • Configure Kafka TLS Encryption
        • Configure Listeners
        • IAM Roles
      • Tiered Storage
        • Use Tiered Storage
        • Fast Commission and Decommission Brokers
        • Mountable Topics
      • Iceberg
        • About Iceberg Topics
        • Specify Iceberg Schema
        • Use Iceberg Catalogs
        • Integrate with REST Catalogs
          • AWS Glue
          • Databricks Unity Catalog
          • GCP BigLake
          • Snowflake and Open Catalog
        • Query Iceberg Topics
        • Migrate to Iceberg Topics
        • Tune Iceberg Performance
        • Troubleshoot Iceberg Topics
      • Schema Registry
        • Overview
        • Use Schema Registry
          • API
          • Kubernetes
        • Schema Registry Authorization
        • Schema Registry Contexts
        • Server-Side Schema ID Validation
        • Manage in Redpanda Console
      • High Availability
      • Disaster Recovery
        • Shadowing
          • Overview
          • Configure Shadowing
          • Monitor Shadowing
          • Failover
          • Failover Runbook
        • Whole Cluster Restore
        • Topic Recovery
      • Remote Read Replicas
      • Recovery Mode
      • Rack Awareness
      • Raft Group Reconfiguration
      • Optimize I/O
      • Redpanda Console
        • Configure Redpanda Console
        • Add an Enterprise License
        • Connect to Redpanda
        • Security
          • Authentication
          • Authorization
          • TLS Termination
        • HTTP Path Rewrites
        • Deserialization
        • Topic Documentation
        • Telemetry
        • Kafka Connect
      • Use the Admin API
      • Monitor Redpanda
    • Upgrade
      • Upgrade Redpanda in Linux
      • Upgrade Redpanda in Kubernetes
      • Upgrade the Redpanda Operator
      • Kubernetes Compatibility
      • Migrate Node Pools
      • Migrate Node Pools Manually
      • Deprecated Features
      • Iceberg Schema Changes in v25.3
    • Migrate
      • Migrate to Redpanda Console v3.0.x
      • Migrate Data to Redpanda with MirrorMaker 2
      • Migrate from the Redpanda Helm chart
      • Migrate from Strimzi to Redpanda Operator
    • Troubleshoot
      • Cluster Diagnostics
        • Linux
        • Kubernetes
      • Debug Bundles
        • Overview
        • Configure
          • Linux
          • Kubernetes
        • Generate
          • Linux
          • Kubernetes
          • Redpanda Console
        • Inspect
      • Resolve Errors
        • Linux
        • Kubernetes
    • Reference
      • Properties
        • Broker Configuration Properties
        • Cluster Configuration Properties
        • Object Storage Properties
        • Topic Configuration Properties
      • Release Notes
        • Redpanda
        • Redpanda Console
        • Redpanda Helm Chart
        • Redpanda Operator
      • API Reference
      • Data Transforms SDKs
        • Golang
        • Rust
        • JavaScript
          • Data Transforms API
          • Schema Registry API
      • Kubernetes
        • Kubernetes Helm Chart Specifications
          • Redpanda
          • Redpanda Operator
          • Redpanda Console
          • Kafka Connect
          • Redpanda Connect
        • Kubernetes Custom Resource Definitions
          • cluster.redpanda.com/v1alpha2
      • Monitoring Metrics
        • Public Metrics
        • Internal Metrics
      • rpk Commands
        • rpk
        • rpk -X
        • rpk cluster
          • rpk cluster config
            • rpk cluster config edit
            • rpk cluster config export
            • rpk cluster config force-reset
            • rpk cluster config get
            • rpk cluster config import
            • rpk cluster config list
            • rpk cluster config lint
            • rpk cluster config set
            • rpk cluster config status
          • rpk cluster connections
            • rpk cluster connections list
          • rpk cluster health
          • rpk cluster license
            • rpk cluster license info
            • rpk cluster license set
          • rpk cluster logdirs
            • rpk cluster logdirs describe
          • rpk cluster maintenance
            • rpk cluster maintenance disable
            • rpk cluster maintenance enable
            • rpk cluster maintenance status
          • rpk cluster info
          • rpk cluster partitions
            • rpk cluster partitions balance
            • rpk cluster partitions disable
            • rpk cluster partitions enable
            • rpk cluster partitions list
            • rpk cluster partitions balancer-status
            • rpk cluster partitions move
              • rpk cluster partitions move-cancel
              • rpk cluster partitions move-status
            • rpk cluster partitions transfer-leadership
            • rpk cluster partitions unsafe-recover
          • rpk cluster quotas
            • rpk cluster quotas alter
            • rpk cluster quotas describe
            • rpk cluster quotas import
          • rpk cluster self-test
            • rpk cluster self-test start
            • rpk cluster self-test status
            • rpk cluster self-test stop
          • rpk cluster storage
            • rpk cluster storage mount/unmount
              • rpk cluster storage cancel mount
              • rpk cluster storage list mount
              • rpk cluster storage list-mountable
              • rpk cluster storage mount
              • rpk cluster storage status mount
              • rpk cluster storage unmount
            • rpk cluster storage restore
            • rpk cluster storage restore start
            • rpk cluster storage restore status
          • rpk cluster txn
            • rpk cluster txn describe
            • rpk cluster txn describe-producers
            • rpk cluster txn list
        • rpk connect
          • rpk connect agent
            • rpk connect agent init
            • rpk connect agent run
          • rpk connect blobl server
          • rpk connect create
          • rpk connect dry-run
          • rpk connect echo
          • rpk connect install
          • rpk connect lint
          • rpk connect list
          • rpk connect plugin
            • rpk connect plugin init
          • rpk connect run
          • rpk connect streams
          • rpk connect studio pull
          • rpk connect studio sync-schema
          • rpk connect template lint
          • rpk connect test
          • rpk connect uninstall
          • rpk connect upgrade
        • rpk container
          • rpk container
          • rpk container purge
          • rpk container start
          • rpk container status
          • rpk container stop
        • rpk debug
          • rpk debug bundle
          • rpk debug remote-bundle cancel
          • rpk debug remote-bundle download
          • rpk debug remote-bundle start
          • rpk debug remote-bundle status
          • rpk debug remote-bundle
        • rpk generate
          • rpk generate app
          • rpk generate grafana-dashboard
          • rpk generate license
          • rpk generate prometheus-config
          • rpk generate shell-completion
        • rpk group
          • rpk group delete
          • rpk group offset-delete
          • rpk group describe
          • rpk group list
          • rpk group seek
        • rpk help
        • rpk iotune
        • rpk plugin
          • rpk plugin list
          • rpk plugin uninstall
          • rpk plugin install
        • rpk profile
          • rpk profile clear
          • rpk profile create
          • rpk profile current
          • rpk profile delete
          • rpk profile edit
          • rpk profile edit-globals
          • rpk profile list
          • rpk profile print
          • rpk profile print-globals
          • rpk profile prompt
          • rpk profile rename-to
          • rpk profile set
          • rpk profile set-globals
          • rpk profile use
        • rpk registry
          • rpk registry compatibility-level
            • rpk registry compatibility-level get
            • rpk registry compatibility-level set
          • rpk registry context
            • rpk registry context delete
            • rpk registry context list
          • rpk registry mode
            • rpk registry mode get
            • rpk registry mode reset
            • rpk registry mode set
          • rpk registry schema
            • rpk registry schema check-compatibility
            • rpk registry schema create
            • rpk registry schema delete
            • rpk registry schema get
            • rpk registry schema list
            • rpk registry schema references
          • rpk registry subject
            • rpk registry subject delete
            • rpk registry subject list
        • rpk redpanda
          • rpk redpanda admin
            • rpk redpanda admin brokers
              • rpk redpanda admin brokers decommission
              • rpk redpanda admin brokers decommission-status
              • rpk redpanda admin brokers list
              • rpk redpanda admin brokers recommission
            • rpk redpanda admin config
              • rpk redpanda admin config log-level set
              • rpk redpanda admin config log-level
              • rpk redpanda admin config print
            • rpk redpanda config print
            • rpk redpanda admin partitions
              • rpk redpanda admin partitions
              • rpk redpanda admin partitions list
          • rpk redpanda check
          • rpk redpanda config
            • rpk redpanda config bootstrap
            • rpk redpanda config init
            • rpk redpanda config set
          • rpk redpanda mode
          • rpk redpanda start
          • rpk redpanda stop
          • rpk redpanda tune
            • rpk redpanda tune
            • rpk redpanda tune help
            • rpk redpanda tune list
        • rpk security
          • rpk security acl
            • rpk security acl create
            • rpk security acl delete
            • rpk security acl list
          • rpk security role
            • rpk security role assign
            • rpk security role create
            • rpk security role delete
            • rpk security role describe
            • rpk security role list
            • rpk security role unassign
          • rpk security user
            • rpk security user create
            • rpk security user delete
            • rpk security user update
            • rpk security user list
        • rpk shadow
          • rpk shadow config generate
          • rpk shadow create
          • rpk shadow delete
          • rpk shadow describe
          • rpk shadow failover
          • rpk shadow list
          • rpk shadow status
          • rpk shadow update
        • rpk topic
          • rpk topic add-partitions
          • rpk topic alter-config
          • rpk topic analyze
          • rpk topic consume
          • rpk topic create
          • rpk topic delete
          • rpk topic describe
          • rpk topic describe-storage
          • rpk topic list
          • rpk topic produce
          • rpk topic trim-prefix
        • rpk transform
          • rpk transform build
          • rpk transform delete
          • rpk transform deploy
          • rpk transform init
          • rpk transform list
          • rpk transform logs
          • rpk transform pause
          • rpk transform resume
        • rpk version
      • Glossary
Connect
    • Overview
    • Get Started
      • Introduction to Redpanda Connect
      • What’s New
      • Quickstarts
        • CLI
        • Helm Chart
      • Licensing
      • Versioning
    • Install and Upgrade
      • CLI
      • Helm Chart
      • Prebuilt Binary
      • Build from Source
    • Configuration
      • Resources
      • Allow and Deny Lists
      • Message Batching
      • Windowed Processing
      • Metadata
      • Error Handling
      • Interpolation
      • Secrets
      • Field Paths
      • Processing Pipelines
      • Unit Testing
      • Templating
      • Dynamic Inputs and Outputs
      • Using CUE
      • Telemetry
    • Components Catalog
      • HTTP
      • Inputs
        • amqp_0_9
        • amqp_1
        • aws_cloudwatch_logs
        • aws_dynamodb_cdc
        • aws_kinesis
        • aws_s3
        • aws_sqs
        • azure_blob_storage
        • azure_cosmosdb
        • azure_queue_storage
        • azure_table_storage
        • batched
        • beanstalkd
        • broker
        • cassandra
        • cockroachdb_changefeed
        • csv
        • discord
        • dynamic
        • file
        • gcp_bigquery_select
        • gcp_cloud_storage
        • gcp_pubsub
        • gcp_spanner_cdc
        • generate
        • git
        • hdfs
        • http_client
        • http_server
        • inproc
        • kafka
        • kafka_franz
        • mongodb
        • mongodb_cdc
        • mqtt
        • mysql_cdc
        • nanomsg
        • nats
        • nats_jetstream
        • nats_kv
        • nats_stream
        • nsq
        • ockam_kafka
        • oracledb_cdc
        • otlp_http
        • otlp_grpc
        • parquet
        • postgres_cdc
        • pulsar
        • read_until
        • redis_list
        • redis_pubsub
        • redis_scan
        • redis_streams
        • redpanda
        • redpanda_common
        • redpanda_migrator
        • microsoft_sql_server_cdc
        • resource
        • salesforce
        • salesforce_cdc
        • salesforce_graphql
        • schema_registry
        • sequence
        • sftp
        • slack
        • slack_users
        • socket
        • socket_server
        • spicedb_watch
        • splunk
        • sql_raw
        • sql_select
        • stdin
        • subprocess
        • tigerbeetle_cdc
        • timeplus
        • twitter_search
        • websocket
        • zmq4
      • Scanners
        • avro
        • chunker
        • csv
        • decompress
        • json_array
        • json_documents
        • lines
        • re_match
        • skip_bom
        • switch
        • tar
        • to_the_end
      • Processors
        • archive
        • avro
        • awk
        • aws_bedrock_chat
        • aws_bedrock_embeddings
        • aws_dynamodb_partiql
        • aws_lambda
        • azure_cosmosdb
        • benchmark
        • bloblang
        • bounds_check
        • branch
        • cache
        • cached
        • cohere_chat
        • cohere_embeddings
        • cohere_rerank
        • catch
        • command
        • compress
        • couchbase
        • crash
        • decompress
        • dedupe
        • ffi
        • for_each
        • gcp_bigquery_select
        • gcp_vertex_ai_chat
        • gcp_vertex_ai_embeddings
        • google_drive_download
        • google_drive_list_labels
        • google_drive_search
        • grok
        • group_by
        • group_by_value
        • http
        • insert_part
        • javascript
        • jira
        • jmespath
        • jq
        • json_schema
        • log
        • mapping
        • metric
        • mongodb
        • msgpack
        • mutation
        • nats_kv
        • nats_request_reply
        • noop
        • ollama_chat
        • ollama_embeddings
        • ollama_moderation
        • openai_chat_completion
        • openai_embeddings
        • openai_image_generation
        • openai_speech
        • openai_transcription
        • openai_translation
        • parallel
        • parquet
        • parquet_decode
        • parquet_encode
        • parse_log
        • processors
        • protobuf
        • qdrant
        • rate_limit
        • redpanda_data_transform
        • redis
        • redis_script
        • resource
        • retry
        • schema_registry_decode
        • schema_registry_encode
        • select_parts
        • sentry_capture
        • slack_thread
        • sleep
        • split
        • sql
        • sql_insert
        • sql_raw
        • sql_select
        • string_split
        • subprocess
        • switch
        • sync_response
        • text_chunker
        • try
        • unarchive
        • wasm
        • while
        • workflow
        • xml
      • Outputs
        • amqp_0_9
        • amqp_1
        • arc
        • aws_dynamodb
        • aws_kinesis
        • aws_kinesis_firehose
        • aws_s3
        • aws_sns
        • aws_sqs
        • azure_blob_storage
        • azure_cosmosdb
        • azure_data_lake_gen2
        • azure_queue_storage
        • azure_table_storage
        • beanstalkd
        • broker
        • cache
        • cassandra
        • couchbase
        • cyborgdb
        • cypher
        • discord
        • drop
        • drop_on
        • dynamic
        • elasticsearch_v8
        • elasticsearch_v9
        • fallback
        • file
        • gcp_bigquery
        • gcp_bigquery_write_api
        • gcp_cloud_storage
        • gcp_pubsub
        • hdfs
        • http_client
        • http_server
        • iceberg
        • inproc
        • kafka
        • kafka_franz
        • mongodb
        • mqtt
        • nanomsg
        • nats
        • nats_jetstream
        • nats_kv
        • nats_stream
        • nsq
        • ockam_kafka
        • opensearch
        • otlp_http
        • otlp_grpc
        • pinecone
        • pulsar
        • pusher
        • qdrant
        • questdb
        • redis_hash
        • redis_list
        • redis_pubsub
        • redis_streams
        • redpanda
        • redpanda_common
        • redpanda_migrator
        • reject
        • reject_errored
        • resource
        • retry
        • salesforce_sink
        • schema_registry
        • sftp
        • slack_post
        • slack_reaction
        • snowflake_put
        • snowflake_streaming
        • socket
        • splunk_hec
        • sql
        • sql_insert
        • sql_raw
        • stdout
        • subprocess
        • switch
        • sync_response
        • timeplus
        • websocket
        • zmq4
      • Caches
        • aws_dynamodb
        • aws_s3
        • couchbase
        • file
        • gcp_cloud_storage
        • lru
        • memcached
        • memory
        • mongodb
        • multilevel
        • nats_kv
        • noop
        • redpanda
        • redis
        • ristretto
        • sql
        • ttlru
      • Rate Limits
        • local
        • redis
      • Buffers
        • memory
        • sqlite
        • none
        • system_window
      • Metrics
        • aws_cloudwatch
        • logger
        • open_telemetry_collector
        • statsd
        • influxdb
        • none
        • json_api
        • prometheus
      • Tracers
        • gcp_cloudtrace
        • none
        • jaeger
        • open_telemetry_collector
        • redpanda
      • Redpanda
      • Logger
    • Plugins
    • Bloblang
      • Walkthrough
      • Functions
      • Methods
      • Arithmetic
      • Advanced
      • Playground
    • Guides
      • Monitoring
      • Message Delivery Semantics
      • Performance Tuning
      • Synchronous Responses
      • Cloud Credentials
        • Amazon Web Services
        • Authenticate to Amazon Aurora using IAM roles
        • Google Cloud Platform
      • Serverless
        • Lambda
      • Streams Mode
        • Streams Via Config Files
        • Streams Via REST API
        • Streams API
      • Migration
        • Migrate to the Unified Redpanda Migrator
        • Migrating to Version 4
        • Migrating to Version 3
        • Migrating to Version 2
    • Cookbooks
      • DynamoDB CDC Patterns
      • Oracle Database CDC Patterns
      • Enrichment Workflows
      • Filtering and Sampling
      • Message Routing Patterns
      • Work with Jira Issues
      • Joining Streams
      • Ingest data into Snowflake
      • Redpanda Migrator
      • Retrieval-Augmented Generation (RAG)
      • Custom Metrics
      • Create a Discord Bot
    • Reference
      • Helm Chart Specification
Streaming • v26.1
Streaming releases 8 versions
Latest 0
v26.1 Latest
Released Mar 2026 Until March 31, 2027
Latest
Supported 0
v25.3
Released Nov 2025 Until November 19, 2026
Supported
v25.2
Released Jul 2025 Until July 31, 2026
Supported
End of life 0
v25.1
Released Apr 2025 Ended April 7, 2026
End of life
v24.3
Released Dec 2024 Ended December 3, 2025
End of life
v24.2
Released Jul 2024 Ended July 31, 2025
End of life
v24.1
Released Apr 2024 Ended April 30, 2025
End of life
v23.3
Released Dec 2023 Ended December 22, 2024
End of life
What's new Upgrade guide Support policy

Redpanda Streaming Documentation

Kafka-compatible event streaming with 10x lower latency and no JVM dependencies. Deploy Redpanda on your infrastructure for maximum performance and control.

Get started with Streaming

Quick start

Set up a local Redpanda cluster in Docker and start streaming in minutes.

Get started

Kubernetes

Deploy production-ready clusters with the Redpanda Operator.

Get started

Linux installation

Install Redpanda on bare metal or VMs for maximum performance.

Get started

Popular topics

Configuration

Tune Redpanda for your workload with cluster and broker properties.

rpk commands

Command-line tool for managing clusters, topics, and data.

Tiered Storage

Archive data to object storage for cost-effective long-term retention.

Schema Registry

Manage schemas for Avro, Protobuf, and JSON with built-in registry.

Need a hand?

Ask Redpanda AI, search the docs, or talk to a real human in Slack.

Join the community
Redpanda
© 2026 Redpanda Data, Inc. All rights reserved.
Privacy policy Terms Status Trust Cookie preferences
Redpanda AI
Ask about any doc
AI answers may be imprecise. Always verify in the docs.