Skip to main content
Version: 22.3

Cluster Configuration Properties

Cluster configuration properties are the same for all nodes in a cluster. They can be set at the cluster level. For information on how to edit cluster properties, see Configuring Cluster Properties.

Cluster properties are grouped into the following categories:

Admin

PropertyDescriptionRestart Required
admin_api_require_authWhether admin API clients must provide HTTP basic authentication headers.

Default: false
No
superusersList of superuser usernames.

Default: Null
No

Cloud storage

PropertyDescriptionRestart Required
cloud_storage_access_keyAWS or GCP access key.

Default: Null
Yes
cloud_storage_api_endpointOptional API endpoint.

Default: Null
Yes
cloud_storage_api_endpoint_portTLS port override.

Default: 443
Yes
cloud_storage_bucketAWS or GCS bucket that should be used to store data.

Default: Null
Yes
cloud_storage_cache_sizeMax size of archival cache.

Default: 20971520000 (20 GB)
Yes
cloud_storage_credentials_sourceThe source of credentials used to connect to cloud services.

Default: config_file
Yes
cloud_storage_disable_tlsDisable TLS for all S3 or GCS connections.

Default: false
Yes
cloud_storage_enabledEnable archival storage.

This property must be enabled in order to use Tiered Storage or Remote Read Replicas.

Default: false
Yes
cloud_storage_max_connectionsMax number of simultaneous uploads to S3 or GCS.

Default: 20
Yes
cloud_storage_regionAWS or GCP region that houses the bucket used for storage.

Default: Null
Yes
cloud_storage_secret_keyAWS or GCP secret key.

Default: Null
Yes
cloud_storage_trust_filePath to certificate that should be used to validate server certificate during TLS handshake.

Default: Null
Yes

Cluster management

PropertyDescriptionRestart Required
cluster_idCluster identifier.

Default: Null
No
enable_auto_rebalance_on_node_addEnable automatic partition rebalancing when new nodes are added.

Default: false
No
enable_leader_balancerEnable automatic leadership rebalancing.

Default: true
No
enable_rack_awarenessEnable rack-aware replica assignment.

Default: false
No

Kafka API (consumers, producers, servers)

PropertyDescriptionRestart Required
enable_idempotenceEnable idempotent producers.

Default: true
Yes
enable_saslEnable SASL authentication for Kafka connections.

Default: false
No
fetch_max_bytesMaximum number of bytes returned in fetch request.

Default: 57671680 (55 MB)
No
group_max_session_timeout_msThe maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.

Default: 300000
No
group_min_session_timeout_msThe minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating which can overwhelm broker resources.

Default: 6000
No
kafka_connection_rate_limitMaximum connections per second for one core.

Default: Null
Minimum = 1
Yes
kafka_connection_rate_limit_overridesOverrides for specific IP addresses for maximum connections per second for one core.

Default: Null
No
kafka_group_recovery_timeout_msKafka group recovery timeout expressed in milliseconds (ms).

Default: 30000
No
kafka_qdc_enableEnable Kafka queue depth control.

Default: false
Yes
kafka_qdc_max_latency_msMaximum latency threshold for Kafka queue depth control depth tracking.

Default: 80
Yes
rm_sync_timeout_msTime to wait state catch up before rejecting a request.

Default: 10000
Yes
rpc_server_listen_backlogTCP connection queue length for Kafka server and internal RPC server.

Default: nil
Minimum = 1
Yes
rpc_server_tcp_recv_bufTCP receive buffer size in bytes.

Default: nil
Minimum = 32768 (32 KB)
Yes
rpc_server_tcp_send_bufTCP transmit buffer size in bytes.

Default: nil
Minimum = 32768 (32 KB)
Yes
target_quota_byte_rateTarget quota byte rate, in bytes per second.

Default: 2097152000 (2 GB)
Minimum = 1048576 (1 MB)
No

Metrics

PropertyDescriptionRestart Required
disable_metricsDisable registering metrics exposed on the internal metrics endpoint (/metrics).

Default: false
Yes
disable_public_metricsDisable registering metrics exposed on the public metrics endpoint (/public_metrics).

Default: false
Yes
enable_metrics_reporterEnable cluster metrics reporter.

Default: true
No

Raft

PropertyDescriptionRestart Required
raft_learner_recovery_rateRaft learner recovery rate limit in bytes per sec.

Default: 104857600 (100 MB)
Yes

Storage

PropertyDescriptionRestart Required
delete_retention_msDelete segments older than this age, expressed in milliseconds (ms).

Default: 604800000 (1 week)
No
log_cleanup_policyDefault topic cleanup policy: delete or compact or both (delete,compact).

Default: delete
No
log_compaction_interval_msHow often to trigger background compaction.

Default: 10000
No
log_compression_typeDefault topic compression type. Possible types: gzip, snappy, lz4, zstd, producer, and none.

Default: Producer
No
log_message_timestamp_typeDefault timestamp type for topic messages: CreateTime or LogAppendTime.

Default: CreateTime
No

Support

PropertyDescriptionRestart Required
metrics_reporter_urlURL of the cluster metrics reporter.

Default: https://m.rp.vectorized.io/v2
N/A

Topics and partitions

PropertyDescriptionRestart Required
auto_create_topics_enabledAllow topic auto creation.

Default: false
No
default_topic_partitionsDefault number of partitions per topic.

Default: 1
No
default_topic_replicationsDefault replication factor for new topics.

Default: 1
No
internal_topic_replication_factorTarget replication factor for internal topics.

Default: 3
No
retention_bytesDefault maximum number of bytes per partition on disk before triggering a compaction.

Default: Null
No
rm_violation_recovery_policyDescribes how to recover from an invariant violation on the partition level: crash or best_effort.

Default: 0

This lab feature is not supported in production environments.
Yes

Transactions

PropertyDescriptionRestart Required
enable_transactionsEnable transactions.

Default: true
Yes
seq_table_min_sizeMinimum size of the seq table that is not affected by compaction.

Default: 1000
tm_sync_timeout_msTime to wait state catch up before rejecting a request, expressed in milliseconds (ms).

Default: 10000
Yes
tm_violation_recovery_policyDescribes how to recover from an invariant violation at the transaction coordinator level: crash or best_effort.

Default: crash
Yes
transaction_coordinator_cleanup_policyCleanup policy for a transaction coordinator topic: compact or delete.

Default: delete
No
transaction_coordinator_delete_retention_msDelete segments older than this age (in ms).

To ensure transaction state is retained as long as the longest-running transaction, make sure this is at least as high as transactional_id_expiration_ms. If you increase transactional_id_expiration_ms, you must increase this by at least the same amount.

Default: 604800000 (1 week)
No
transactional_id_expiration_msProducer IDs expire after this amount of time has elapsed after the last write with the given producer ID.

Default: 604800000
Yes
tx_timeout_delay_msDelay before scheduling the next check for timed out transactions.

Default: 1000

Wasm

PropertyDescriptionRestart Required
enable_coprocEnable coprocessing mode.

Default: false

This lab feature is in is not supported in production environments.
Yes

Suggested reading

  • Fast distributed transactions with Redpanda article