Cluster configuration properties
Cluster configuration properties are properties that are the same for all nodes in a cluster, and can be set at the cluster level. For information on how to edit cluster properties, refer to Configuring cluster properties.
All cluster properties are presented in tables that are grouped into the following categories:
- Admin
- Cloud Storage
- Cluster management
- Kafka API (consumers, producers, and servers)
- Metrics
- Raft
- Storage (logs, segments, and any information written to disk)
- Support
- Topics and Partitions
- Transactions
- Wasm
Admin
| Property | Description | Restart Required |
|---|---|---|
| admin_api_require_auth | Whether admin API clients must provide HTTP Basic authentication headers. Default: false | No |
| superusers | List of superuser usernames. Default: Null | No |
Cloud Storage
| Property | Description | Restart Required |
|---|---|---|
| cloud_storage_access_key | AWS or GCP access key. Default: Null | Yes |
| cloud_storage_api_endpoint | Optional API endpoint. Default: Null | Yes |
| cloud_storage_api_endpoint_port | TLS port override. Default: 443 | Yes |
| cloud_storage_bucket | AWS or GCS bucket that should be used to store data. Default: Null | Yes |
| cloud_storage_cache_size | Max size of archival cache. Default: 20971520000 (20 GB) | Yes |
| cloud_storage_disable_tls | Disable TLS for all S3 or GCS connections. Default: false | Yes |
| cloud_storage_enabled | Enable archival storage. This property must be enabled in order to use Shadow Indexing. Default: false | Yes |
| cloud_storage_max_connections | Max number of simultaneous uploads to S3 or GCS. Default: 20 | Yes |
| cloud_storage_region | AWS or GCP region that houses the bucket used for storage. Default: Null | Yes |
| cloud_storage_secret_key | AWS or GCP secret key. Default: Null | Yes |
| cloud_storage_trust_file | Path to certificate that should be used to validate server certificate during TLS handshake. Default: Null | Yes |
Cluster management
| Property | Description | Restart Required |
|---|---|---|
| cluster_id | Cluster identifier. Default: Null | No |
| enable_auto_rebalance_on_node_add | Enable automatic partition rebalancing when new nodes are added. Default: false | No |
| enable_leader_balancer | Enable automatic leadership rebalancing. Default: true | No |
| enable_rack_awareness | Enable rack-aware replica assignment. Default: false | No |
Kafka API (consumers, producers, and servers)
| Property | Description | Restart Required |
|---|---|---|
| enable_idempotence | Enable idempotent producer. Default: true | Yes |
| enable_sasl | Enable SASL authentication for Kafka connections. Default: false | No |
| fetch_max_bytes | Maximum number of bytes returned in fetch request. Default: 57671680 (55 MB) | No |
| group_max_session_timeout_ms | The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures. Default: 300000 | No |
| group_min_session_timeout_ms | The minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating which can overwhelm broker resources. Default: 6000 | No |
| kafka_connection_rate_limit | Maximum connections per second for one core. Default: Null Minimum = 1 | Yes |
| kafka_connection_rate_limit_overrides | Overrides for specific IP addresses for maximum connections per second for one core. Default: Null | No |
| kafka_group_recovery_timeout_ms | Kafka group recovery timeout expressed in milliseconds (ms). Default: 30000 | No |
| kafka_qdc_enable | Enable kafka queue depth control. Default: false | Yes |
| kafka_qdc_max_latency_ms | Maximum latency threshold for kafka queue depth control depth tracking. Default: 80 | Yes |
| rm_sync_timeout_ms | Time to wait state catch up before rejecting a request. Default: 10000 | Yes |
| rpc_server_listen_backlog | TCP connection queue length for Kafka server and internal RPC server. Default: nil Minimum = 1 | Yes |
| rpc_server_tcp_recv_buf | TCP receive buffer size in bytes. Default: nil Minimum = 32768 (32 KB) | Yes |
| rpc_server_tcp_send_buf | TCP transmit buffer size in bytes. Default: nil Minimum = 32768 (32 KB) | Yes |
| target_quota_byte_rate | Target quota byte rate, in bytes per second. Default: 2097152000 (2 GB) Minimum = 1048576 (1 MB) | No |
Metrics
| Property | Description | Restart Required |
|---|---|---|
| disable_metrics | Disable registering metrics. Default: false | Yes |
| enable_metrics_reporter | Enable cluster metrics reporter. If true, the metrics reporter collects and exports to Redpanda Data a set of customer usage metrics. Default: true | No |
Raft
| Property | Description | Restart Required |
|---|---|---|
| raft_learner_recovery_rate | Raft learner recovery rate limit in bytes per sec. Default: 104857600 (100 MB) | Yes |
Storage (logs, segments, and any information written to disk)
| Property | Description | Restart Required |
|---|---|---|
| delete_retention_ms | Delete segments older than this age, expressed in milliseconds (ms). Default: 604800000 (1 week) | No |
| log_cleanup_policy | Default topic cleanup policy. Default: delete | No |
| log_compaction_interval_ms | How often we trigger background compaction. Default: 10000 | No |
| log_compression_type | Default topic compression type. Possible types are gzip, snappy, lz4, zstd, producer, and none. Default: Producer | No |
| log_message_timestamp_type | Default timestamp type for topic messages. Possible types are CreateTime and LogAppendTime. Default: CreateTime | No |
Support
| Property | Description | Restart Required |
|---|---|---|
| metrics_reporter_url | The URL of the cluster metrics reporter. Default: https://m.rp.vectorized.io/v2 | N/A |
Topics and Partitions
| Property | Description | Restart Required |
|---|---|---|
| auto_create_topics_enabled | Allow topic auto creation. Default: false | No |
| default_topic_partitions | Default number of partitions per topic. Default: 1 | No |
| default_topic_replications | Default replication factor for new topics. Default: 1 | No |
| id_allocator_replication | Replication factor for an id allocator topic. Default: 1 | No |
| internal_topic_replication_factor | Target replication factor for internal topics. Default: 3 | No |
| retention_bytes | Default maximum number of bytes per partition on disk before triggering a compaction. Default: Null | No |
| rm_violation_recovery_policy | Describes how to recover from an invariant violation on the partition level. Choices are crash and best_effort. Default: 0 This feature is in tech preview and is not supported in production environments. | Yes |
Transactions
| Property | Description | Restart Required |
|---|---|---|
| enable_transactions | Enable transactions. Default: false | Yes |
| seq_table_min_size | Minimum size of the seq table that is not affected by compaction. Default: 1000 | |
| tm_sync_timeout_ms | Time to wait state catch up before rejecting a request, expressed in milliseconds (ms). Default: 10000 | Yes |
| tm_violation_recovery_policy | Describes how to recover from an invariant violation at the transaction coordinator level. Choices are crash and best_effort. Default: crash | Yes |
| transaction_coordinator_cleanup_policy | Cleanup policy for a transaction coordinator topic. Choices are compact and delete. Default: delete | No |
| transaction_coordinator_delete_retention_ms | Delete segments older than this age (in ms). Default: 604800000 (1 week) | No |
| transaction_coordinator_replication | Replication factor for a transaction coordinator topic. Default: 1 | No |
| transactional_id_expiration_ms | Producer IDs expire once this amount of time has elapsed after the last write with the given producer id. Default: 604800000 | Yes |
| tx_timeout_delay_ms | Delay before scheduling the next check for timed out transactions. Default: 1000 |
Wasm
| Property | Description | Restart Required |
|---|---|---|
| enable_coproc | Enable coprocessing mode. Default: false This feature is in tech preview and is not supported in production environments. | Yes |
Suggested reading
- Fast distributed transactions with Redpanda article