Skip to main content
Version: 22.1

Cluster configuration properties

Cluster configuration properties are properties that are the same for all nodes in a cluster, and can be set at the cluster level. For information on how to edit cluster properties, refer to Configuring cluster properties.

All cluster properties are presented in tables that are grouped into the following categories:

Admin

PropertyDescriptionRestart Required
admin_api_require_authWhether admin API clients must provide HTTP Basic authentication headers.

Default: false
No
superusersList of superuser usernames.

Default: Null
No

Cloud Storage

PropertyDescriptionRestart Required
cloud_storage_access_keyAWS or GCP access key.

Default: Null
Yes
cloud_storage_api_endpointOptional API endpoint.

Default: Null
Yes
cloud_storage_api_endpoint_portTLS port override.

Default: 443
Yes
cloud_storage_bucketAWS or GCS bucket that should be used to store data.

Default: Null
Yes
cloud_storage_cache_sizeMax size of archival cache.

Default: 20971520000 (20 GB)
Yes
cloud_storage_disable_tlsDisable TLS for all S3 or GCS connections.

Default: false
Yes
cloud_storage_enabledEnable archival storage. This property must be enabled in order to use Shadow Indexing.

Default: false
Yes
cloud_storage_max_connectionsMax number of simultaneous uploads to S3 or GCS.

Default: 20
Yes
cloud_storage_regionAWS or GCP region that houses the bucket used for storage.

Default: Null
Yes
cloud_storage_secret_keyAWS or GCP secret key.

Default: Null
Yes
cloud_storage_trust_filePath to certificate that should be used to validate server certificate during TLS handshake.

Default: Null
Yes

Cluster management

PropertyDescriptionRestart Required
cluster_idCluster identifier.

Default: Null
No
enable_auto_rebalance_on_node_addEnable automatic partition rebalancing when new nodes are added.

Default: false
No
enable_leader_balancerEnable automatic leadership rebalancing.

Default: true
No
enable_rack_awarenessEnable rack-aware replica assignment.

Default: false
No

Kafka API (consumers, producers, and servers)

PropertyDescriptionRestart Required
enable_idempotenceEnable idempotent producer.

Default: true
Yes
enable_saslEnable SASL authentication for Kafka connections.

Default: false
No
fetch_max_bytesMaximum number of bytes returned in fetch request.

Default: 57671680 (55 MB)
No
group_max_session_timeout_msThe maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.

Default: 300000
No
group_min_session_timeout_msThe minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating which can overwhelm broker resources.

Default: 6000
No
kafka_connection_rate_limitMaximum connections per second for one core.

Default: Null
Minimum = 1
Yes
kafka_connection_rate_limit_overridesOverrides for specific IP addresses for maximum connections per second for one core.

Default: Null
No
kafka_group_recovery_timeout_msKafka group recovery timeout expressed in milliseconds (ms).

Default: 30000
No
kafka_qdc_enableEnable kafka queue depth control.

Default: false
Yes
kafka_qdc_max_latency_msMaximum latency threshold for kafka queue depth control depth tracking.

Default: 80
Yes
rm_sync_timeout_msTime to wait state catch up before rejecting a request.

Default: 10000
Yes
rpc_server_listen_backlogTCP connection queue length for Kafka server and internal RPC server.

Default: nil
Minimum = 1
Yes
rpc_server_tcp_recv_bufTCP receive buffer size in bytes.

Default: nil
Minimum = 32768 (32 KB)
Yes
rpc_server_tcp_send_bufTCP transmit buffer size in bytes.

Default: nil
Minimum = 32768 (32 KB)
Yes
target_quota_byte_rateTarget quota byte rate, in bytes per second.

Default: 2097152000 (2 GB)
Minimum = 1048576 (1 MB)
No

Metrics

PropertyDescriptionRestart Required
disable_metricsDisable registering metrics.

Default: false
Yes
enable_metrics_reporterEnable cluster metrics reporter.

Default: true
No

Raft

PropertyDescriptionRestart Required
raft_learner_recovery_rateRaft learner recovery rate limit in bytes per sec.

Default: 104857600 (100 MB)
Yes

Storage (logs, segments, and any information written to disk)

PropertyDescriptionRestart Required
delete_retention_msDelete segments older than this age, expressed in milliseconds (ms).

Default: 604800000 (1 week)
No
log_cleanup_policyDefault topic cleanup policy.

Default: delete
No
log_compaction_interval_msHow often we trigger background compaction.

Default: 10000
No
log_compression_typeDefault topic compression type. Possible types are gzip, snappy, lz4, zstd, producer, and none.

Default: Producer
No
log_message_timestamp_typeDefault timestamp type for topic messages. Possible types are CreateTime and LogAppendTime.

Default: CreateTime
No

Support

PropertyDescriptionRestart Required
metrics_reporter_urlThe URL of the cluster metrics reporter.

Default: https://m.rp.vectorized.io/v2
N/A

Topics and Partitions

PropertyDescriptionRestart Required
auto_create_topics_enabledAllow topic auto creation.

Default: false
No
default_topic_partitionsDefault number of partitions per topic.

Default: 1
No
default_topic_replicationsDefault replication factor for new topics.

Default: 1
No
id_allocator_replicationReplication factor for an id allocator topic.

Default: 1
No
internal_topic_replication_factorTarget replication factor for internal topics.

Default: 3
No
retention_bytesDefault maximum number of bytes per partition on disk before triggering a compaction.

Default: Null
No
rm_violation_recovery_policyDescribes how to recover from an invariant violation on the partition level. Choices are crash and best_effort.

Default: 0

This feature is in tech preview and is not supported in production environments.
Yes

Transactions

PropertyDescriptionRestart Required
enable_transactionsEnable transactions.

Default: false
Yes
seq_table_min_sizeMinimum size of the seq table that is not affected by compaction.

Default: 1000
tm_sync_timeout_msTime to wait state catch up before rejecting a request, expressed in milliseconds (ms).

Default: 10000
Yes
tm_violation_recovery_policyDescribes how to recover from an invariant violation at the transaction coordinator level. Choices are crash and best_effort.

Default: crash
Yes
transaction_coordinator_cleanup_policyCleanup policy for a transaction coordinator topic. Choices are compact and delete.

Default: delete
No
transaction_coordinator_delete_retention_msDelete segments older than this age (in ms).

Default: 604800000 (1 week)
No
transaction_coordinator_replicationReplication factor for a transaction coordinator topic.

Default: 1
No
transactional_id_expiration_msProducer IDs expire once this amount of time has elapsed after the last write with the given producer id.

Default: 604800000
Yes
tx_timeout_delay_msDelay before scheduling the next check for timed out transactions.

Default: 1000

Wasm

PropertyDescriptionRestart Required
enable_coprocEnable coprocessing mode.

Default: false

This feature is in tech preview and is not supported in production environments.
Yes

Suggested reading

  • Fast distributed transactions with Redpanda article