Tunable Configuration Properties
Tunable properties are cluster properties that are specifically designed for tuning a cluster.
For information on how to edit tunable properties, see Configuring Cluster Properties.
Cloud storage properties
cloud_storage_cache_check_interval
Timeout to check if cache eviction should be triggered.
Units: milliseconds
Default: 30000 (30 seconds)
Restart required: yes
cloud_storage_enable_compacted_topic_reupload
Enable re-upload of data for compacted topics.
Type: boolean
Default: true
Restart required: no
cloud_storage_enable_remote_read
Default remote read configuration value for new topics.
Type: boolean
Default: false
Restart required: no
cloud_storage_enable_remote_write
Default remote write configuration value for new topics.
Type: boolean
Default: false
Restart required: no
cloud_storage_housekeeping_interval_ms
Interval for cloud storage housekeeping tasks.
Units: milliseconds
Default: 300000 (5 minutes)
Restart required: no
cloud_storage_initial_backoff_ms
Initial backoff time for exponential backoff algorithm.
Units: milliseconds
Default: 100
Restart required: yes
cloud_storage_manifest_upload_timeout_ms
Manifest upload timeout.
Units: milliseconds
Default: 10000 (10 seconds)
Restart required: yes
cloud_storage_max_connection_idle_time_ms
Maximum HTTPS connection idle time.
Units: milliseconds
Default: 5000
Restart required: yes
cloud_storage_max_materialized_segments_per_shard
Maximum number of concurrent readers of remote data per shard (CPU core). If null, the value of topic_partitions_per_shard multiplied by two is used.
Type: unsigned integer
Units: number of concurrent readers
Default: null
Restart required: no
cloud_storage_max_readers_per_shard
Maximum concurrent readers of remote data per shard (CPU core). If null, the value of topic_partitions_per_shard is used, with one reader per partition if the shard is at its maximum partition capacity.
Type: unsigned integer
Units: number of concurrent readers
Default: null
Restart required: no
cloud_storage_max_segments_pending_deletion_per_partition
The maximum limit per partition for the number of segments pending deletion from the cloud. Segments can be deleted due to retention or compaction. If this limit is breached and deletion fails, then segments are orphaned in the cloud and must be removed manually.
Type: unsigned integer
Units: number of segments
Default: 5000
Restart required: no
cloud_storage_metadata_sync_timeout_ms
Timeout for shadow indexing (SI) metadata synchronization.
Units: milliseconds
Default: 10000 (10 seconds)
Restart required: no
cloud_storage_readreplica_manifest_sync_timeout_ms
Timeout to check if new data is available for a partition in S3 for read replica.
Units: milliseconds
Default: 30000 (30 seconds)
Restart required: no
cloud_storage_reconciliation_interval_ms
Interval at which the archival service runs reconciliation.
Units: milliseconds
Default: 1000
Restart required: yes
cloud_storage_segment_max_upload_interval_sec
Duration that a segment can be kept in local storage without uploading it to remote storage. If null, defaults to infinite duration.
Units: seconds
Default: null (infinite duration)
Restart required: yes
cloud_storage_segment_upload_timeout_ms
Log segment upload timeout.
Units: milliseconds
Default: 30000 (30 seconds)
Restart required: yes
cloud_storage_upload_ctrl_d_coeff
Derivative coefficient of the upload PID controller.
Type: double
Default: 0.0
Restart required: yes
cloud_storage_upload_ctrl_max_shares
Maximum number of IO and CPU shares that the archival upload can use.
Units: number of IO and CPU shares
Default: 1000
Restart required: yes
cloud_storage_upload_ctrl_min_shares
Minimum number of IO and CPU shares that the archival upload can use.
Units: number of IO and CPU shares
Default: 100
Restart required: yes
cloud_storage_upload_ctrl_p_coeff
Proportional coefficient of the upload PID controller.
Type: double
Default: -2.0
Restart required: yes
cloud_storage_upload_ctrl_update_interval_ms
Sampling interval of the upload PID controller.
Units: milliseconds
Default: 60000 (1 minute)
Restart required: yes
Cluster management properties
controller_backend_housekeeping_interval_ms
Interval between iterations of the controller backend’s housekeeping loop.
Units: milliseconds
Default: 1000
Restart required: yes
controller_log_accummulation_rps_capacity_acls_and_users_operations
Maximum capacity of accumulated requests for the cluster controller’s access control list (ACL) operations and user operations. If null, no maximum capacity is applied.
Type: unsigned integer
Units: number of requests
Default: null
Restart required: no
Related properties: rps_limit_acls_and_users_operations
controller_log_accummulation_rps_capacity_configuration_operations
Maximum capacity of accumulated requests for the cluster controller’s configuration operations. If null, no maximum capacity is applied.
Type: unsigned integer
Units: number of requests
Default: null
Restart required: no
Related properties: rps_limit_configuration_operations
controller_log_accummulation_rps_capacity_move_operations
Maximum capacity of accumulated requests for the cluster controller’s move operations. If null, no maximum capacity is applied.
Type: unsigned integer
Units: number of requests
Default: null
Restart required: no
Related properties: rps_limit_move_operations
controller_log_accummulation_rps_capacity_node_management_operations
Maximum capacity of accumulated requests for the cluster controller’s node management operations. If null, no maximum capacity is applied.
Type: unsigned integer
Units: number of requests
Default: null
Restart required: no
Related properties: rps_limit_node_management_operations
health_manager_tick_interval
Period at which the health manager runs.
Units: milliseconds
Default: 180000 (3 minutes)
Restart required: yes
health_monitor_max_metadata_age
Maximum duration that metadata is cached in the health monitor of a non-controller node.
Units: milliseconds
Default: 10000 (10 seconds)
Restart required: no
health_monitor_tick_interval
Period at which the health monitor refreshes cluster state.
Units: milliseconds
Default: 10000 (10 seconds)
Restart required: no
id_allocator_batch_size
Size of one batch message (one log record) of an ID allocator log.
Type: integer
Units: bytes
Default: 1000
Restart required: yes
id_allocator_log_capacity
Maximum number of messages (batches) of an ID allocator log. Once the log reaches this limit, it is compacted.
Type: integer
Units: number of log messages (batches)
Default: 100
Restart required: yes
join_retry_timeout_ms
Duration between cluster join retries.
Units: milliseconds
Default: 5000 (5 seconds)
Restart required: yes
leader_balancer_idle_timeout
Timeout at which to run the leader balancer when in an idle state.
Units: milliseconds
Default: 120000 (2 minutes)
Restart required: no
leader_balancer_mute_timeout
Timeout used to mute groups, where muted groups are removed from consideration as leadership rebalancing targets.
Units: milliseconds
Default: 300000 (5 minutes)
Restart required: no
leader_balancer_node_mute_timeout
Timeout used to mute a node, where a muted node is removed from consideration as leadership rebalancing target.
Units: milliseconds
Default: 20000 (20 seconds)
Restart required: no
leader_balancer_transfer_limit_per_shard
Maximum limit per shard (CPU core) on the number of in-progress leadership transfers.
Type: unsigned integer
Units: number of in-progress leadership transfers per shard
Default: 512
Range: [1, 2048]
Restart required: no
max_concurrent_producer_ids
Maximum number of producer IDs that the resource manager state machine caches in internal state.
Type: 64-bit unsigned integer
Units: number of IDs
Default: maximum value of a 64-bit unsigned integer
Range: [1, maximum value of a 64-bit unsigned integer]
Restart required: no
members_backend_retry_ms
Duration between members backend reconciliation loop retries.
Units: milliseconds
Default: 5000
Restart required: yes
metadata_dissemination_interval_ms
Interval for metadata dissemination batching.
Units: milliseconds
Default: 3000
Restart required: yes
metadata_dissemination_retries
Number of attempts looking up a topic’s metadata, like shard, before failing a request.
Units: number of retries
Default: 30
Restart required: yes
metadata_dissemination_retry_delay_ms
Delay before retrying a topic-lookup in a shard or other meta-tables.
Units: milliseconds
Default: 500
Restart required: yes
metadata_status_wait_timeout_ms
Maximum duration to wait in metadata request for cluster health to be refreshed.
Units: milliseconds
Default: 2000
Restart required: yes
node_management_operation_timeout_ms
Timeout for executing node management operations.
Units: milliseconds
Default: 5000
Restart required: yes
node_status_interval
Time interval between two node status messages, which establish liveness status outside of the Raft protocol.
Units: milliseconds
Default: 100
Restart required: no
partition_autobalancing_concurrent_moves
Number of partitions that can be reassigned at once by the partition auto-balancer.
Type: unsigned integer
Units: number of partitions
Default: 50
Restart required: no
partition_autobalancing_tick_interval_ms
Tick interval of the partition auto-balancer.
Units: milliseconds
Default: 30000 (30 seconds)
Restart required: no
partition_autobalancing_movement_batch_size_bytes
Total size of partitions that the partition auto-balancer moves in one batch.
Type: unsigned integer
Units: bytes
Default: 5368709120 (5 GiB)
Restart required: no
rps_limit_acls_and_users_operations
Request-per-second (rps) rate limit for the cluster controller’s access control list (ACL) operations and user operations.
Type: unsigned integer
Units: requests per second
Default: 1000
Restart required: no
rps_limit_configuration_operations
Request-per-second (rps) rate limit for the cluster controller’s configuration operations.
Type: unsigned integer
Units: requests per second
Default: 1000
Restart required: no
rps_limit_move_operations
Request-per-second (rps) rate limit for the cluster controller’s move operations.
Type: unsigned integer
Units: requests per second
Default: 1000
Restart required: no
Kafka API properties
default_num_windows
Default number of quota tracking windows.
Units: number of windows
Default: 10
Restart required: no
default_window_sec
Default duration of quota tracking window.
Units: milliseconds
Default: 1000
Restart required: no
fetch_reads_debounce_timeout
Duration to wait for the next read in fetch request when minimum number of requested bytes wasn’t read.
Units: milliseconds
Default: 1
Restart required: no
fetch_session_eviction_timeout_ms
Minimum time before an unused session gets evicted. The maximum time after which an inactive session is deleted is twice the given configuration value cache.
Units: milliseconds
Default: 60000 (1 minute)
Restart required: yes
group_initial_rebalance_delay
A delay at the start of a consumer group rebalancing event to wait for new members.
Units: milliseconds
Default: 3000
Range: [0, …]
Restart required: no
group_new_member_join_timeout
Timeout to wait for new members to join a consumer group.
Units: milliseconds
Default: 30000
Range: [0, …]
Restart required: no
group_topic_partitions
Number of partitions in the internal group membership topic.
Type: integer
Units: number of partitions
Default: 16
Restart required: no
kafka_batch_max_bytes
Maximum size of a batch processed by the server. If the batch is compressed, this limit applies to the compressed batch size.
Type: unsigned integer
Units: bytes
Default: 1048576 (1 MiB)
Restart required: no
kafka_max_bytes_per_fetch
Maximum amount of data a consumer receives when fetching a record.
Type: integer
Units: bytes
Default: 67108864 (64 MiB)
Restart required: no
kafka_qdc_depth_alpha
Smoothing factor for queue depth control depth tracking.
Type: double
Default: 0.8
Restart required: yes
kafka_qdc_depth_update_ms
The update frequency of the queue depth control algorithm.
Units: milliseconds
Default: 7000 (7 seconds)
Restart required: yes
kafka_qdc_idle_depth
Queue depth of the queue depth control algorithm when in an idle state.
Units: number of queue entries
Default: 10
Restart required: no
kafka_qdc_latency_alpha
Smoothing parameter for queue depth control algorithm.
Type: double
Default: 0.002
Restart required: yes
kafka_qdc_max_depth
Maximum queue depth of the queue depth control algorithm.
Units: number of queue entries
Default: 100
Range: >= kafka_qdc_min_depth
Restart required: yes
kafka_qdc_min_depth
Minimum queue depth of the queue depth control algorithm.
Units: number of queue entries
Default: 1
Range: <= kafka_qdc_max_depth
Restart required: yes
kafka_qdc_window_count
Number of time windows used by the queue depth control algorithm.
Units: number of time windows
Default: 12
Restart required: yes
kafka_qdc_window_size_ms
Duration of a time window used by the queue depth control algorithm.
Units: milliseconds
Default: 1500
Restart required: yes
kafka_request_max_bytes
Maximum size of a request processed by a Kafka API.
Type: unsigned integer
Units: bytes
Default: 104857600 (100 MiB)
Restart required: no
kafka_rpc_server_stream_recv_buf
Maximum size of the user-space receive buffer. If null, this limit is not applied.
Type: unsigned integer
Units: bytes
Default: null
Range: [512, 512 KiB]. Must be 4096-byte aligned.
Restart required: yes
max_kafka_throttle_delay_ms
Fail-safe maximum throttle delay on Kafka requests.
Units: milliseconds
Default: 60000 (1 minute)
Restart required: no
Metrics properties
Raft properties
full_raft_configuration_recovery_pattern
Recover raft configurations on start for NTPs that match this pattern.
Type: string, or list of strings
Default: {} (empty)
Restart required: yes
raft_heartbeat_disconnect_failures
Number of consecutive failed heartbeat responses before forcibly closing an unresponsive TCP connection. Disabled when set to 0.
Type: unsigned integer
Units: number of consecutive failed heartbeat responses
Default: 3
Range: [0, …]
Restart required: yes
raft_heartbeat_interval_ms
Period of raft leader heartbeat.
Units: milliseconds
Default: 150
Restart required: no
raft_heartbeat_timeout_ms
Timeout duration of raft leader heartbeat RPC.
Units: milliseconds
Default: 3000
Restart required: no
raft_io_timeout_ms
Timeout duration for raft I/O.
Units: milliseconds
Default: 10000 (10 seconds)
Restart required: yes
raft_max_concurrent_append_requests_per_follower
Maximum number of concurrent requests per follower.
Type: unsigned integer
Units: number of concurrent requests per follower
Default: 16
Restart required: yes
raft_max_recovery_memory
Maximum amount of memory available for raft follower recovery. If null, defaults to 10% of total memory.
Type: unsigned integer
Units: bytes
Default: null
Range: [32 MiB, …]
Restart required: no
raft_recovery_default_read_size
Default size of a read issued during raft follower recovery.
Type: unsigned integer
Units: bytes
Default: 524288 (512 KiB)
Range: [128, 5 MiB]
Restart required: no
raft_replicate_batch_window_size
Maximum size of a batch for raft replication.
Type: unsigned integer
Units: bytes
Default: 1048576 (1 MiB)
Restart required: yes
raft_smp_max_non_local_requests
Maximum number of non-local requests that can execute concurrently on a CPU core (seastar 'shard'). If null, defaults to the value determined by the following formula:
topic_partitions_per_shard * (266) * (<number of CPU cores> - 1)
For details, see seastar::smp_service_group_config.
Type: unsigned integer
Units: number of requests
Default: null
Restart required: yes
raft_timeout_now_timeout_ms
Timeout of the raft timeout now RPC request.
Units: milliseconds
Default: 1000
Restart required: no
Storage properties
abort_index_segment_size
Capacity of an abort index segment.
Type: unsigned integer
Units: number of transactions
Default: 50000
Restart required: yes
append_chunk_size
Size of direct-write operations to disk.
Type: unsigned integer
Units: bytes
Default: 16384 (16 KiB)
Range: [4096, 32 MiB]. Must be 4096-byte aligned.
Restart required: yes
compacted_log_segment_size
Size of a compacted log segment.
Type: unsigned 64-bit integer
Units: bytes
Default: 268435456 (256 MiB)
Range: [1 MiB, …]
Restart required: no
compaction_ctrl_backlog_size
Size of the compaction backlog of the backlog controller. If null (empty), defaults to 10% of disk capacity.
Type: integer
Units: bytes
Default: null
Range: [1 MiB, …]
Restart required: yes
compaction_ctrl_d_coeff
Derivative coefficient for compaction PID controller.
Type: double
Default: 0.2
Restart required: yes
compaction_ctrl_i_coeff
Integral coefficient for compaction PID controller.
Type: double
Default: 0.0
Range: [0.0, …]
Restart required: yes
compaction_ctrl_max_shares
Maximum number of I/O and CPU shares that the compaction process can use.
Units: short
Units: number of shares
Default: 1000
Restart required: yes
compaction_ctrl_min_shares
Minimum number of I/O and CPU shares that the compaction process can use.
Type: short
Units: number of shares
Default: 10
Restart required: yes
compaction_ctrl_p_coeff
Proportional coefficient for compaction PID controller. Must be negative to decrease the compaction backlog when compaction shares increase.
Type: double
Default: -12.5
Range: […, 0.0)
Restart required: yes
compaction_ctrl_update_interval_ms
Sampling interval of the compaction PID controller.
Units: milliseconds
Default: 30000 (30 seconds)
Restart required: yes
disable_batch_cache
Flag to disable the batch cache in log manager.
Type: boolean
Default: false
Restart required: yes
kvstore_flush_interval
Flush interval of the key-value store.
Units: milliseconds
Default: 10
Restart required: no
kvstore_max_segment_size
Maximum size of a segment of the key-value store.
Type: unsigned integer
Units: bytes
Default: 16777216 (16 MiB)
Restart required: yes
log_segment_size
Default size of log segment. Per topic, it is superseded by topic property segment.bytes.
Type: unsigned 64-bit integer
Units: bytes
Default: 1073741824 (1 GiB)
Range: [1 MiB, …]
Restart required: no
log_segment_size_max
Upper limit of topic segment.bytes. Higher values are clamped to this limit. If null, no limit is applied.
Type: unsigned 64-bit integer
Units: bytes
Default: null (no limit applied)
Restart required: no
log_segment_size_min
Lower limit of topic segment.bytes. Lower values are clamped to this limit. If null, no limit is applied.
Type: unsigned 64-bit integer
Units: bytes
Default: 1048576 (1 MiB)
Restart required: no
log_segment_size_jitter_percent
Configures the amount of size variation added to a segment used for each partition: a percentage value in the range [0, log_segment_size_jitter_percent] is randomly chosen and that percentage of the original segment size limit is added.
Applying segment size jitter is a good defensive measure to space out the overhead created from segment rolls over time.
Type: unsigned short
Units: percent of segment size that
Default: 5
Range: [0, 99]
Restart required: yes
max_compacted_log_segment_size
Max compacted segment size after consolidation.
Type: unsigned integer
Units: bytes
Default: 5368709120 (5 GiB)
Restart required: no
readers_cache_eviction_timeout_ms
Duration after which inactive readers are evicted from the cache of readers.
Units: milliseconds
Default: 30000 (30 seconds)
Restart required: yes
reclaim_batch_cache_min_free
Minimum amount of free memory maintained by the batch cache.
Type: unsigned integer
Units: bytes
Default: 67108864 (64 MiB)
Restart required: yes
reclaim_growth_window
Starting from the last point in time when memory was reclaimed from the batch cache, this is the duration during which the amount of memory to reclaim grows at a significant rate, based on heuristics about the amount of available memory.
Units: milliseconds
Default: 3000
Range: <= reclaim_stable_window
Restart required: yes
reclaim_max_size
Maximum batch cache reclaim size.
Units: bytes
Default: 4194304 (4 MiB)
Restart required: yes
reclaim_min_size
Minimum batch cache reclaim size.
Units: bytes
Default: 131072 (128 KiB)
Restart required: yes
reclaim_stable_window
If the duration since the last time memory was reclaimed has been longer than this property, the memory usage of the batch cache is considered stable, so only the minimum size (reclaim_min_size) is set to be reclaimed.
Units: milliseconds
Default: 10000 (10 seconds)
Range: >= reclaim_growth_window
Restart required: yes
release_cache_on_segment_roll
Flag for whether to release cache when a full segment is rolled.
Type: boolean
Default: false
Restart required: no
replicate_append_timeout_ms
Timeout for append entry’s RPCs issued while replicating entries.
Units: milliseconds
Default: 3000
Restart required: yes
segment_appender_flush_timeout_ms
Maximum duration until buffered data is written.
Units: milliseconds
Default: 1000
Restart required: no
segment_fallocation_step
Size of a file allocation (f allocation) step of a segment.
Type: unsigned integer
Units: bytes
Default: 33554432 (32 MiB)
Range: [4096, 1 GiB]. Must be 4096-byte aligned.
Restart required: no
storage_compaction_index_memory
Maximum number of bytes that may be used on each shard (CPU core) by compaction index writers.
Type: unsigned integer
Units: bytes
Default: 134217728 (128 MiB)
Range: [16 MiB, 100 GiB]
Restart required: no
storage_max_concurrent_replay
Maximum number of partition logs that are replayed concurrently on startup or flushed concurrently on shutdown.
Type: unsigned integer
Units number of concurrently replayed logs
Default: 1024
Range: [128, …]
Restart required: no
storage_min_free_bytes
Threshold of minimum amount of free space before rejecting producers.
Type: unsigned integer
Units: bytes
Default: 5368709120 (5 GiB)
Range: [10 MiB, …]
Restart required: no
storage_read_buffer_size
Size of a read buffer (one per in-flight read, one per log segment).
Type: unsigned integer
Units: bytes
Default: 131072 (128 KiB)
Restart required: yes
storage_read_readahead_count
Number of read-ahead reads when reading from a segment.
Type: short
Units: number of reads
Default: 10
Restart required: yes
storage_space_alert_free_threshold_bytes
The threshold of free storage (in bytes of free storage) below which a low storage-space alert is logged.
Type: unsigned integer
Units: bytes of free storage
Default: 0
Range: [0, …]
Restart required: no
storage_space_alert_free_threshold_percent
The threshold of free storage (in percentage of total storage that is free) below which a low storage-space alert is logged.
Type: unsigned integer
Units: percentage of total storage that is free
Default: 5
Range: [0, 50]
Restart required: no
Topic and partition properties
alter_topic_cfg_timeout_ms
Time to wait for entries replication in controller log when executing alter topic configuration request.
Units: milliseconds
Default: 5000
Restart required: no
create_topic_timeout_ms
Timeout to wait for new topic creation.
Units: milliseconds
Default: 2000
Restart required: no
topic_fds_per_partition
The number of file descriptors reserved for topics per partition.
Type: integer
Units: number of file descriptors
Default: 5
Range: [1, 1000]
Restart required: no
topic_memory_per_partition
The amount of memory reserved for topics per partition.
Type: unsigned integer
Units: bytes
Default: 1048576 (1 MiB)
Range: [1, 100 MiB]
Restart required: no
topic_partitions_per_shard
Maximum number of partitions which may be allocated per shard (CPU core).
Type: unsigned integer
Units: number of partitions
Default: 7000
Range: [16, 131072]
Restart required: no
topic_partitions_reserve_shard0
Reserved partition slots on shard (CPU core) 0 on each node. If this is >= topic_partitions_per_shard, no data partitions are scheduled on shard 0.
Type: unsigned integer
Units: number of partitions
Default: 2
Range: [0, 131072]
Restart required: no