Cluster Configuration Properties

Cluster configuration properties are properties that are the same for all nodes in a cluster, and can be set at the cluster level. For information on how to edit cluster properties, refer to Configuring Cluster Properties.

All cluster properties are presented in tables that are grouped into the following categories:

Admin

Property Description Restart Required

admin_api_require_auth

Whether admin API clients must provide HTTP Basic authentication headers.

Default: false

No

superusers

List of superuser usernames.

Default: Null

No

Cloud Storage

Property Description Restart Required

cloud_storage_access_key

AWS or GCP access key.

Default: Null

Yes

cloud_storage_api_endpoint

Optional API endpoint.

Default: Null

Yes

cloud_storage_api_endpoint_port

TLS port override.

Default: 443

Yes

cloud_storage_bucket

AWS or GCS bucket that should be used to store data.

Default: Null

Yes

cloud_storage_cache_size

Max size of archival cache.

Default: 20971520000 (20 GB)

Yes

cloud_storage_credentials_source

The source of credentials used to connect to cloud services.

Default: config_file

Yes

cloud_storage_disable_tls

Disable TLS for all S3 or GCS connections.

Default: false

Yes

cloud_storage_enabled

Enable archival storage. This property must be enabled in order to use Tiered Storage or Remote Read Replicas.

Default: false

Yes

cloud_storage_max_connections

Max number of simultaneous uploads to S3 or GCS.

Default: 20

Yes

cloud_storage_region

AWS or GCP region that houses the bucket used for storage.

Default: Null

Yes

cloud_storage_secret_key

AWS or GCP secret key.

Default: Null

Yes

cloud_storage_trust_file

Path to certificate that should be used to validate server certificate during TLS handshake.

Default: Null

Yes

Cluster management

Property Description Restart Required

cluster_id

Cluster identifier.

Default: Null

No

 Deprecated red enable_auto_rebalance_on_node_add

Enable automatic partition rebalancing when new nodes are added.

Default: false

No

enable_leader_balancer

Enable automatic leadership rebalancing.

Default: true

No

enable_rack_awareness

Enable rack-aware replica assignment.

Default: false

No

Kafka API (consumers, producers, and servers)

Property Description Restart Required

enable_idempotence

Enable idempotent producer.

Default: true

Yes

enable_sasl

Enable SASL authentication for Kafka connections.

Default: false

No

fetch_max_bytes

Maximum number of bytes returned in fetch request.

Default: 57671680 (55 MB)

No

group_max_session_timeout_ms

The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.

Default: 300000

No

group_min_session_timeout_ms

The minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating which can overwhelm broker resources.

Default: 6000

No

kafka_connection_rate_limit

Maximum connections per second for one core.

Default: Null
Minimum = 1

Yes

kafka_connection_rate_limit_overrides

Overrides for specific IP addresses for maximum connections per second for one core.

Default: Null

No

kafka_group_recovery_timeout_ms

Kafka group recovery timeout expressed in milliseconds (ms).

Default: 30000

No

kafka_qdc_enable

Enable kafka queue depth control.

Default: false

Yes

kafka_qdc_max_latency_ms

Maximum latency threshold for kafka queue depth control depth tracking.

Default: 80

Yes

rm_sync_timeout_ms

Time to wait state catch up before rejecting a request.

Default: 10000

Yes

rpc_server_listen_backlog

TCP connection queue length for Kafka server and internal RPC server.

Default: nil
Minimum = 1

Yes

rpc_server_tcp_recv_buf

TCP receive buffer size in bytes.

Default: nil
Minimum = 32768 (32 KB)

Yes

rpc_server_tcp_send_buf

TCP transmit buffer size in bytes.

Default: nil
Minimum = 32768 (32 KB)

Yes

target_quota_byte_rate

Target quota byte rate, in bytes per second.

Default: 2097152000 (2 GB)
Minimum = 1048576 (1 MB)

No

Metrics

Property Description Restart Required

disable_metrics

Disable registering metrics exposed on the internal metrics endpoint (/metrics).

Default: false

Yes

disable_public_metrics

Disable registering metrics exposed on the public metrics endpoint (/public_metrics).

Default: false

Yes

enable_metrics_reporter

Enable cluster metrics reporter. If true, the metrics reporter collects and exports to Redpanda Data a set of customer usage metrics.

Default: true

No

Raft

Property Description Restart Required

raft_learner_recovery_rate

Raft learner recovery rate limit in bytes per sec.

Default: 104857600 (100 MB)

Yes

Storage (logs, segments, and any information written to disk)

Property Description Restart Required

delete_retention_ms

Delete segments older than this age, expressed in milliseconds (ms).

Default: 604800000 (1 week)

No

log_cleanup_policy

Default topic cleanup policy.

Default: delete

No

log_compaction_interval_ms

How often we trigger background compaction.

Default: 10000

No

log_compression_type

Default topic compression type. Possible types are gzip, snappy, lz4, zstd, producer, and none.

Default: Producer

No

log_message_timestamp_type

Default timestamp type for topic messages. Possible types are CreateTime and LogAppendTime.

Default: CreateTime

No

Support

Property Description Restart Required

metrics_reporter_url

The URL of the cluster metrics reporter.

Default: https://m.rp.vectorized.io/v2

N/A

Topics and Partitions

Property Description Restart Required

auto_create_topics_enabled

Allow topic auto creation.

Default: false

No

default_topic_partitions

Default number of partitions per topic.

Default: 1

No

default_topic_replications

Default replication factor for new topics.

Default: 1

No

id_allocator_replication

Replication factor for an id allocator topic.

Default: 1

No

internal_topic_replication_factor

Target replication factor for internal topics.

Default: 3

No

retention_bytes

Default maximum number of bytes per partition on disk before triggering a compaction.

Default: Null

No

rm_violation_recovery_policy

Describes how to recover from an invariant violation on the partition level. Choices are crash and best_effort.

Default: 0

This feature is in tech preview and is not supported in production environments.

Yes

Transactions

Property Description Restart Required

enable_transactions

Enable transactions.

Default: false

Yes

seq_table_min_size

Minimum size of the seq table that is not affected by compaction.

Default: 1000

tm_sync_timeout_ms

Time to wait state catch up before rejecting a request, expressed in milliseconds (ms).

Default: 10000

Yes

tm_violation_recovery_policy

Describes how to recover from an invariant violation at the transaction coordinator level. Choices are crash and best_effort.

Default: crash

Yes

transaction_coordinator_cleanup_policy

Cleanup policy for a transaction coordinator topic. Choices are compact and delete.

Default: delete

No

transaction_coordinator_delete_retention_ms

Delete segments older than this age (in ms).

Default: 604800000 (1 week)

No

transaction_coordinator_replication

Replication factor for a transaction coordinator topic.

Default: 1

No

transactional_id_expiration_ms

Producer IDs expire once this amount of time has elapsed after the last write with the given producer id.

Default: 604800000

Yes

tx_timeout_delay_ms

Delay before scheduling the next check for timed out transactions.

Default: 1000

Suggested reading

  • Fast distributed transactions with Redpanda article