Cluster Configuration Properties
Cluster configuration properties are the same for all nodes in a cluster. They can be set at the cluster level.
For information on how to edit cluster properties, see Configuring Cluster Properties. Some properties require the cluster to be restarted in order to be applied; see a specific property's reference for whether restart is required.
Admin properties
admin_api_require_auth
Whether admin API clients must provide HTTP basic authentication headers.
Default: false
Restart required: no
superusers
List of superuser usernames.
Default: null
Restart required: no
Cloud storage properties
cloud_storage_access_key
AWS or GCP access key.
Default: null
Restart required: yes
cloud_storage_api_endpoint
Optional API endpoint.
Default: null
Restart required: yes
cloud_storage_api_endpoint_port
TLS port override.
Default: 443
Restart required: yes
cloud_storage_bucket
AWS or GCS bucket that should be used to store data.
Default: null
Restart required: yes
cloud_storage_cache_size
Max size of archival cache.
Units: bytes
Default: 21474836480 (20 GiB)
Restart required: yes
cloud_storage_credentials_source
The source of credentials used to connect to cloud services.
Default: config_file
Valid values: config_file, aws_instance_metadata, sts, gcp_instance_metadata
Restart required: yes
cloud_storage_disable_tls
Disable TLS for all S3 or GCS connections.
Type: boolean
Default: false
Restart required: yes
cloud_storage_enabled
Enable archival storage. Must be enabled true in order to use Tiered Storage or Remote Read Replicas.
Type: boolean
Default: false
Restart required: yes
cloud_storage_max_connections
Max number of simultaneous connections to S3 per shard. Includes connections used for both uploads and downloads.
Units: number of simultaneous connections
Default: 20
Restart required: yes
cloud_storage_region
AWS or GCP region that houses the bucket used for storage.
Type: string
Default: null
Restart required: yes
cloud_storage_secret_key
AWS or GCP secret key.
Type: string
Default: null
Restart required: yes
cloud_storage_trust_file
Path to certificate that should be used to validate server certificate during TLS handshake.
Type: string
Default: null
Restart required: yes
Cluster management properties
cluster_id
Cluster identifier.
Type: string
Default: null
Restart required: no
enable_auto_rebalance_on_node_add
Enable automatic partition rebalancing when new nodes are added.
Type: boolean
Default: false
Restart required: no
enable_leader_balancer
Enable automatic leadership rebalancing.
Type: boolean
Default: true
Restart required: no
enable_rack_awareness
Enable rack-aware replica assignment.
Type: boolean
Default: false
Restart required: no
Kafka API properties
enable_idempotence
Enable idempotent producers.
Type: boolean
Default: true
Restart required: yes
enable_sasl
Enable SASL authentication for Kafka connections.
Type: boolean
Default: false
Restart required: no
fetch_max_bytes
Maximum number of bytes returned in a fetch request.
Units: bytes
Default: 57671680 (55 MiB)
Restart required: no
group_max_session_timeout_ms
The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.
Units: milliseconds
Default: 300000 (300 sec)
Restart required: no
group_min_session_timeout_ms
The minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating which can overwhelm broker resources.
Units: milliseconds
Default: 6000 (6 sec)
Restart required: no
kafka_connection_rate_limit
Maximum connections per second for one core. If null (the default), the number of connections per second is unlimited.
Units: number of connections per second, per core
Default: null
Range: [1, ...]
Restart required: yes
kafka_connection_rate_limit_overrides
Overrides the maximum connections per second for one core for the specified IP addresses (for example, ['127.0.0.1:90', '50.20.1.1:40'])
Type: string
Default: null
Restart required: no
kafka_connections_max
Maximum number of Kafka client connections per broker. If null, the property is disabled.
Units: number of Kafka client connections per broker
Default: null
Restart required: no
kafka_connections_max_overrides
A list of IP addresses for which Kafka client connection limits are overridden and don't apply. For example, (['127.0.0.1:90', '50.20.1.1:40']).
Default: {} (empty list)
Restart required: no
kafka_connections_max_per_ip
Maximum number of Kafka client connections per IP address, per broker. If null, the property is disabled.
Units: number of Kafka client connections per IP address, per broker
Default: null
Restart required: no
kafka_group_recovery_timeout_ms
Kafka group recovery timeout.
Units: milliseconds
Default: 30000 (30 sec)
Restart required: no
kafka_qdc_enable
Enable Kafka queue depth control.
Type: boolean
Default: false
Restart required: yes
kafka_qdc_max_latency_ms
Maximum latency threshold for Kafka queue depth control depth tracking.
Units: milliseconds
Default: 80
Restart required: yes
rm_sync_timeout_ms
Resource manager's synchronization timeout. Maximum time for this node to wait for internal state machine to catch up with all events written by previous leaders before rejecting a request.
Units: milliseconds
Default: 10000 (10 sec)
Restart required: yes
rpc_server_listen_backlog
Maximum TCP connection queue length for Kafka server and internal RPC server. If null (the default value), no queue length is set.
Units: number of queue entries
Default: null
Range: [1, ...]
Restart required: yes
rpc_server_tcp_recv_buf
Internal RPC TCP receive buffer size. If null (the default value), no buffer size is set by Redpanda.
Units: bytes
Default: null
Range: [32 KiB, ...], aligned to 4096 bytes
Restart required: yes
rpc_server_tcp_send_buf
Internal RPC TCP send buffer size. If null (the default value), no buffer size is set by Redpanda.
Units: bytes
Default: null
Range: [32 KiB, ...], aligned to 4096 bytes
Restart required: yes
target_quota_byte_rate
Target quota byte rate.
Units: bytes per second
Default: 2147483648 (2 GiB)
Range: [1048576 (1 MiB), ...]
Restart required: no
Metrics properties
aggregate_metrics
Enable aggregation of metrics returned by the /metrics endpoint. Metric aggregation is performed by summing the values of samples by labels and is done when it makes sense by the shard and/or partition labels.
Type: boolean
Default: false
Restart required: yes
disable_metrics
Disable registering metrics exposed on the internal metrics endpoint.
Type: boolean
Default: false
Restart required: yes
disable_public_metrics
Disable registering metrics exposed on the public metrics endpoint.
Type: boolean
Default: false
Restart required: yes
enable_metrics_reporter
Enable the cluster metrics reporter. If true, the metrics reporter collects and exports to Redpanda Data a set of customer usage metrics at the interval set by metrics_reporter_report_interval.
The cluster metrics of the metrics reporter are different from monitoring metrics.
- The metrics reporter exports customer usage metrics for consumption by Redpanda Data.
- Monitoring metrics are exported for consumption by Redpanda users to monitor their system's health.
Type: boolean
Default: true
Restart required: no
Raft properties
raft_learner_recovery_rate
Raft learner recovery rate limit. Throttles the rate of data communicated to nodes (learners) that need to catch up to leaders.
Units: bytes per second
Default: 104857600 (100 MB/sec)
Restart required: no
Storage properties
delete_retention_ms
Delete segments that are older than this age.
Units: milliseconds
Default: 604800000 (1 week)
Restart required: no
log_cleanup_policy
Bit flags setting the default topic cleanup policy: compact and/or delete.
Default: delete
Valid Values: compact, delete, compact,delete, none
Restart required: no
log_compaction_interval_ms
How often to trigger background compaction.
Units: milliseconds
Default: 10000 (10 sec)
Restart required: no
log_compression_type
Default topic compression type (gzip, snappy, lz4, zstd, producer, or none).
Default: producer
Valid values: gzip, snappy, lz4, zstd, producer, none
Restart required: no
log_message_timestamp_type
Default timestamp type for topic messages (CreateTime or LogAppendTime).
Default: CreateTime
Valid values: CreateTime, LogAppendTime
Restart required: no
storage_strict_data_init
Requires that an empty file named .redpanda_data_dir be present in the data directory. If set true, Redpanda will refuse to start if the file is not found in the data directory.
Default: false
Restart required: no
Support properties
metrics_reporter_url
URL of the cluster metrics reporter.
Default: https://m.rp.vectorized.io/v2
Topic and partition properties
auto_create_topics_enabled
Allow automatic topic creation.
Type: boolean
Default: false
Restart required: no
default_topic_partitions
Default number of partitions per topic.
Units: number of partitions per topic
Default: 1
Restart required: no
default_topic_replications
Default replication factor for new topics.
Units: number of replicas per topic
Default: 1
Range: [1, ...], must be odd
Restart required: no
internal_topic_replication_factor
Target replication factor for internal topics.
Units: number of replicas per topic
Default: 3
Restart required: no
retention_bytes
Default maximum number of bytes per partition on disk before triggering deletion of the oldest messages. If null (the default value), no limit is applied.
Units: bytes per partition
Default: null
Restart required: no
rm_violation_recovery_policy
Describes how to recover from an invariant violation on the partition level.
Default: 0
Restart required: yes
Transaction properties
enable_transactions
Enable transactions (atomic writes).
Type: boolean
Default: true
Restart required: yes
seq_table_min_size
The minimum threshold number of sessions to keep in the seq table. Not affected by compaction.
Default: 1000
Restart required: yes
tm_sync_timeout_ms
Transaction manager's synchronization timeout. Maximum time to wait for internal state machine to catch up before rejecting a request.
Units: milliseconds
Default: 10000 (10 sec)
Restart required: yes
tm_violation_recovery_policy
Describes how to recover from an invariant violation at the transaction coordinator level.
Default: crash
Restart required: yes
transaction_coordinator_cleanup_policy
Cleanup policy for a transaction coordinator topic.
Default: delete
Valid Values: compact, delete, compact,delete, none
Restart required: no
transaction_coordinator_delete_retention_ms
Delete segments older than this age. To ensure transaction state is retained as long as the longest-running transaction, make sure this is no less than transactional_id_expiration_ms.
Units: milliseconds
Default: 604800000 (1 week)
Restart required: no
transactional_id_expiration_ms
Expiration time of producer IDs. Measured starting from the time of the last write until now for a given ID.
Units: milliseconds
Default: 604800000 (1 week)
Restart required: yes
tx_timeout_delay_ms
Delay before scheduling the next check for timed out transactions.
Units: milliseconds
Default: 1000
Restart required: yes
Wasm properties
Redpanda Data Transforms, built on top of Wasm, is a lab feature and is not supported for production deployments.
enable_coproc
Enable coprocessing mode.
Type: boolean
Default: false
Restart required: yes
Suggested reading
- Fast distributed transactions with Redpanda article