Cluster Configuration Properties

Capacity (in number of txns) of an abort index segment.

Each partition tracks the aborted transaction offset ranges to help service client requests. If the number transactions increase beyond this threshold, they are flushed to disk to easy memory pressure. Then they’re loaded on demand. This configuration controls the maximum number of aborted transactions before they are flushed to disk.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: 50000

Interval, in milliseconds, at which Redpanda looks for inactive transactions and aborts them.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10000 (10s)

Whether Admin API clients must provide HTTP basic authentication headers.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

Enable aggregation of metrics returned by the /metrics endpoint. Aggregation can simplify monitoring by providing summarized data instead of raw, per-instance metrics. Metric aggregation is performed by summing the values of samples by labels and is done when it makes sense by the shard and/or partition labels.

Requires restart: No

Optional: Yes

Visibility: None

Type: boolean

Default: false

The amount of time since the last broker status heartbeat. After this time, a broker is considered offline and not alive.

Unit: milliseconds

Requires restart: No

Nullable: No

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 5000

The duration, in milliseconds, that Redpanda waits for the replication of entries in the controller log when executing a request to alter topic configurations. This timeout ensures that configuration changes are replicated across the cluster before the alteration request is considered complete.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 5000 (5s)

Size of direct write operations to disk in bytes. A larger chunk size can improve performance for write-heavy workloads, but increase latency for these writes as more data is collected before each write operation. A smaller chunk size can decrease write latency, but potentially increase the number of disk I/O operations.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 16384

Defines the number of bytes allocated by the internal audit client for audit messages. When changing this, you must disable audit logging and then re-enable it for the change to take effect. Consider increasing this if your system generates a very large number of audit records in a short amount of time.

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Default: 16777216

Enables or disables audit logging. When you set this to true, Redpanda checks for an existing topic named _redpanda.audit_log. If none is found, Redpanda automatically creates one for you.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

List of strings in JSON style identifying the event types to include in the audit log. This may include any of the following: management, produce, consume, describe, heartbeat, authenticate, schema_registry, admin.

Requires restart: No

Optional: Yes

Visibility: user

Type: array

Default: [management, authenticate, admin]

List of user principals to exclude from auditing.

Requires restart: No

Optional: Yes

Visibility: user

Type: array

Default: null

List of topics to exclude from auditing.

Requires restart: No

Optional: Yes

Visibility: user

Type: array

Default: null

Defines the number of partitions used by a newly-created audit topic. This configuration applies only to the audit log topic and may be different from the cluster or other topic configurations. This cannot be altered for existing audit log topics.

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-2147483648, 2147483647]

Default: 12

Defines the replication factor for a newly-created audit log topic. This configuration applies only to the audit log topic and may be different from the cluster or other topic configurations. This cannot be altered for existing audit log topics. Setting this value is optional. If a value is not provided, Redpanda will use the value specified for internal_topic_replication_factor.

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [-32768, 32767]

Default: null

Interval, in milliseconds, at which Redpanda flushes the queued audit log messages to the audit log topic. Longer intervals may help prevent duplicate messages, especially in high throughput scenarios, but they also increase the risk of data loss during shutdowns where the queue is lost.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 500

Defines the maximum amount of memory in bytes used by the audit buffer in each shard. Once this size is reached, requests to log additional audit messages will return a non-retryable error. Limiting the buffer size per shard helps prevent any single shard from consuming excessive memory due to audit log messages.

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: integer

Default: 1048576

Allow automatic topic creation. To prevent excess topics, this property is not supported on Redpanda Cloud BYOC and Dedicated clusters. You should explicitly manage topic creation for these Redpanda Cloud clusters.

If you produce to a topic that doesn’t exist, the topic will be created with defaults if this property is enabled.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

Cluster identifier.

Requires restart: No

Optional: No

Visibility: None

Type: string

Default: null

Size (in bytes) for each compacted log segment.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: 268435456

Target backlog size for compaction controller. If not set the max backlog size is configured to 80% of total disk space available.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Derivative coefficient for compaction PID controller.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: number

Default: 0.2

Integral coefficient for compaction PID controller.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: number

Default: 0.0

Maximum number of I/O and CPU shares that compaction process can use.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-32768, 32767]

Default: 1000

Minimum number of I/O and CPU shares that compaction process can use.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-32768, 32767]

Default: 10

Proportional coefficient for compaction PID controller. This must be negative, because the compaction backlog should decrease when the number of compaction shares increases.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: number

Default: -12.5

Interval between iterations of controller backend housekeeping loop.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 1000 (1s)

Maximum capacity of rate limit accumulation in controller ACLs and users operations limit.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Maximum capacity of rate limit accumulation in controller configuration operations limit.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Maximum capacity of rate limit accumulation in controller move operations limit.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Maximum capacity of rate limit accumulation in controller node management operations limit.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Maximum capacity of rate limit accumulation in controller topic operations limit.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Maximum amount of time before Redpanda attempts to create a controller snapshot after a new controller command appears.

Unit: seconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: 60

If set to true, move partitions between cores in runtime to maintain balanced partition distribution.

Requires restart: No

Optional: No

Visibility: user

Type: boolean

Default: false

Interval, in milliseconds, between trigger and invocation of core balancing.

Unit: milliseconds

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10000

If set to true, and if after a restart the number of cores changes, Redpanda will move partitions between cores to maintain balanced partition distribution.

Requires restart: No

Optional: No

Visibility: user

Type: boolean

Default: true

Enables CPU profiling for Redpanda.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

The sample period for the CPU profiler.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 100

Timeout, in milliseconds, to wait for new topic creation.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 2000

The maximum size for a deployable WebAssembly binary that the broker can store.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 10485760

The commit interval at which data transforms progress.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 3000

Enables WebAssembly-powered data transforms directly in the broker. When data_transforms_enabled is set to true, Redpanda reserves memory for data transforms, even if no transform functions are currently deployed. This memory reservation ensures that adequate resources are available for transform functions when they are needed, but it also means that some memory is allocated regardless of usage.

Requires restart: Yes

Optional: Yes

Visibility: user

Type: boolean

Default: false

Buffer capacity for transform logs, per shard. Buffer occupancy is calculated as the total size of buffered log messages; that is, logs emitted but not yet produced.

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: integer

Default: 512000

Flush interval for transform logs. When a timer expires, pending logs are collected and published to the transform_logs topic.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 500

Transform log lines truncate to this length. Truncation occurs after any character escaping.

Unit: bytes

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 1024

The amount of memory to reserve per core for data transform (Wasm) virtual machines. Memory is reserved on boot. The maximum number of functions that can be deployed to a cluster is equal to data_transforms_per_core_memory_reservation / data_transforms_per_function_memory_limit.

Requires restart: Yes

Optional: Yes

Visibility: user

Type: integer

Default: 20971520

The amount of memory to give an instance of a data transform (Wasm) virtual machine. The maximum number of functions that can be deployed to a cluster is equal to data_transforms_per_core_memory_reservation / data_transforms_per_function_memory_limit.

Requires restart: Yes

Optional: Yes

Visibility: user

Type: integer

Default: 2097152

The percentage of available memory in the transform subsystem to use for read buffers.

Requires restart: Yes

Optional: No

Visibility: tunable

Type: integer

Default: 45

The maximum amount of runtime to start up a data transform, and the time it takes for a single record to be transformed.

Unit: milliseconds

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 3000

The percentage of available memory in the transform subsystem to use for write buffers.

Requires restart: Yes

Optional: No

Visibility: tunable

Type: integer

Default: 45

The recursion depth after which debug logging is enabled automatically for the log reader.

Requires restart: No

Nullable: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: null

Default number of quota tracking windows.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-32768, 32767]

Default: 10

Default number of partitions per topic.

Unit: number of partitions per topic

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-2147483648, 2147483647]

Default: 1

Default replication factor for new topics.

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-32768, 32767]

Default: 1

Default quota tracking window size in milliseconds.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 1000

Disable batch cache in log manager.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

Disables the cluster recovery loop. This property is used to simplify testing and should not be set in production.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

Disable registering the metrics exposed on the internal /metrics endpoint.

Requires restart: No

Optional: Yes

Visibility: None

Type: boolean

Default: false

Disable registering the metrics exposed on the /public_metrics endpoint.

Requires restart: No

Optional: Yes

Visibility: None

Type: boolean

Default: false

The percentage of total disk capacity that Redpanda will avoid using. This applies both when cloud cache and log data share a disk, as well as when cloud cache uses a dedicated disk.

It is recommended to not run disks near capacity to avoid blocking I/O due to low disk space, as well as avoiding performance issues associated with SSD garbage collection.

Unit: percentage of total disk size.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: number

Default: 25.0

Enables cluster metadata uploads. Required for whole cluster restore.

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: boolean

Default: true

Limits the write rate for the controller log.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

Enable idempotent producers.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Enable automatic leadership rebalancing. Mode is set by leader_balancer_mode.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Enable the cluster metrics reporter. If true, the metrics reporter collects and exports to Redpanda Data a set of customer usage metrics at the interval set by metrics_reporter_report_interval.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Enable Redpanda extensions for MPX.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

Enable PID file. You should not need to change.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: true

Enable rack-aware replica assignment.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

Enable SASL authentication for Kafka connections. Authorization is required to modify this property. See also kafka_enable_authorization.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

Mode to enable server-side schema ID validation.

Related topics:

Requires restart: No

Optional: Yes

Visibility: user

Accepted Values:

Default: none

Enable transactions (atomic writes).

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Enables the usage tracking mechanism, storing windowed history of kafka/cloud_storage metrics over time.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

Whether new feature flags auto-activate after upgrades (true) or must wait for manual activation via the Admin API (false).

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: true

Maximum number of bytes returned in a fetch request.

Unit: bytes

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Default: 57671680

The strategy used to fulfill fetch requests.

Requires restart: No

Optional: Yes

Visibility: tunable

Accepted Values: polling, non_polling, non_polling_with_debounce

Default: non_polling

Time to wait for the next read in fetch requests when the requested minimum bytes was not reached.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 1

Time duration after which the inactive fetch session is removed from the fetch session cache. Fetch sessions are used to implement the incremental fetch requests where a consumer does not send all requested partitions to the server but the server tracks them for the consumer.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 60000

Delay added to the rebalance phase to wait for new members.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 3000

The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: None

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 300000

The minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating, which can overwhelm broker resources.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: None

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 6000

Timeout for new member joins.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 30000

Frequency rate at which the system should check for expired group offsets.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 600000 (10min)

Consumer group offset retention seconds. To disable offset retention, set this to null.

Unit: seconds

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: 604800 (one week)

Number of partitions in the internal group membership topic.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-2147483648, 2147483647]

Default: 16

How often the health manager runs.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 180000 (3min)

Maximum age of the metadata cached in the health monitor of a non-controller broker.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10000

A list of supported HTTP authentication mechanisms.

Requires restart: No

Optional: Yes

Visibility: user

Type: array

Accepted Values: BASIC, OIDC

Default: [basic]

The ID allocator allocates messages in batches (each batch is a one log record) and then serves requests from memory without touching the log until the batch is exhausted.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-32768, 32767]

Default: 1000

Capacity of the id_allocator log in number of batches. After it reaches id_allocator_stm, it truncates the log’s prefix.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-32768, 32767]

Default: 100

Initial local retention size target for partitions of topics with Tiered Storage enabled. If no initial local target retention is configured, then all locally-retained data will be delivered to learner when joining the partition replica set.

Unit: bytes

Requires restart: No

Optional: No

Visibility: user

Type: integer

Default: null

Initial local retention time target for partitions of topics with Tiered Storage enabled. If no initial local target retention is configured, then all locally-retained data will be delivered to learner when joining the partition replica is set.

Unit: milliseconds

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: null

Target replication factor for internal topics.

Unit: number of replicas per topic.

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-2147483648, 2147483647]

Default: 3

Time between cluster join retries in milliseconds.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 5000

Maximum size of a batch processed by the server. If the batch is compressed, the limit applies to the compressed batch size.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: 1048576

Maximum connections per second for one core. If null (the default), then the number of connections per second is unlimited.

Unit: number of connections per second, per core

Related topics:

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [-9223372036854775808, 9223372036854775807]

Default: null

Overrides the maximum connections per second for one core for the specified IP addresses (for example, ['127.0.0.1:90', '50.20.1.1:40'])

Related topics:

Requires restart: No

Optional: Yes

Visibility: user

Type: array

Default: null

Maximum number of Kafka client connections per broker. If null, the property is disabled.

Unit: number of Kafka client connections per broker

Default: null

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [0, 4294967295]

Related topics:

A list of IP addresses for which Kafka client connection limits are overridden and don’t apply. For example, (['127.0.0.1:90', '50.20.1.1:40']).

Requires restart: No

Optional: Yes

Visibility: user

Type: array

Default: {} (empty list)

Related topics:

Maximum number of Kafka client connections per IP address, per broker. If null, the property is disabled.

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [0, 4294967295]

Default: null

Related topics:

Flag to require authorization for Kafka connections. If null, the property is disabled, and authorization is instead enabled by enable_sasl.

Requires restart: No

Optional: No

Visibility: user

Type: boolean

Default: null

Accepted Values:

Related properties:

Whether to include Tiered Storage as a special remote:// directory in DescribeLogDirs Kafka API requests.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Enable the Kafka partition reassignment API.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Kafka group recovery timeout.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 30000 (30 sec)

Limit fetch responses to this many bytes, even if the total of partition bytes limits is higher.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 67108864

The size of the batch used to estimate memory consumption for fetch requests, in bytes. Smaller sizes allow more concurrent fetch requests per shard. Larger sizes prevent running out of memory because of too many concurrent fetch requests.

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Default: 1048576

The share of Kafka subsystem memory that can be used for fetch read buffers, as a fraction of the Kafka subsystem memory amount.

Requires restart: Yes

Optional: Yes

Visibility: user

Type: number

Default: 0.5

Principal mapping rules for mTLS authentication on the Kafka API. If null, the property is disabled.

Requires restart: No

Optional: No

Visibility: user

Type: array

Default: null

A list of topics that are protected from deletion and configuration changes by Kafka clients. Set by default to a list of Redpanda internal topics.

Requires restart: No

Optional: Yes

Visibility: user

Type: string array

Default: ['_redpanda.audit_log', '__consumer_offsets', '_schemas']

Related topics:

A list of topics that are protected from being produced to by Kafka clients. Set by default to a list of Redpanda internal topics.

Requires restart: No

Optional: Yes

Visibility: user

Type: array

Default: ['_redpanda.audit_log']

Smoothing factor for Kafka queue depth control depth tracking.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: number

Default: 0.8

Update frequency for Kafka queue depth control.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 7000

Enable Kafka queue depth control.

Requires restart: Yes

Optional: Yes

Visibility: user

Type: boolean

Default: false

Queue depth when idleness is detected in Kafka queue depth control.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 10

Smoothing parameter for Kafka queue depth control latency tracking.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: number

Default: 0.002

Maximum queue depth used in Kafka queue depth control.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 100

Maximum latency threshold for Kafka queue depth control depth tracking.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 80

Minimum queue depth used in Kafka queue depth control.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 1

Number of windows used in Kafka queue depth control latency tracking.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 12

Window size for Kafka queue depth control latency tracking.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 1500

The minimum value of the throughput quota a shard can get in the process of quota balancing, expressed in bytes per second. The value applies equally to ingress and egress traffic.

kafka_quota_balancer_min_shard_throughput_bps doesn’t override the limit settings, kafka_throughput_limit_node_in_bps and kafka_throughput_limit_node_out_bps. Consequently, the value of kafka_throughput_limit_node_in_bps or kafka_throughput_limit_node_out_bps can result in lesser throughput than kafka_quota_balancer_min_shard_throughput_bps.

Both kafka_quota_balancer_min_shard_throughput_ratio and kafka_quota_balancer_min_shard_throughput_bps can be specified at the same time. In this case, the balancer will not decrease the effective shard quota below the largest bytes-per-second (bps) value of each of these two properties.

If set to 0, no minimum is enforced.

Unit: bytes per second

Related topics:

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-9223372036854775808, 9223372036854775807]

Default: 256

Related topics:

The minimum value of the throughput quota a shard can get in the process of quota balancing, expressed as a ratio of default shard quota. While the value applies equally to ingress and egress traffic, the default shard quota can be different for ingress and egress and therefore result in different minimum throughput bytes-per-second (bps) values.

Both kafka_quota_balancer_min_shard_throughput_ratio and kafka_quota_balancer_min_shard_throughput_bps can be specified at the same time. In this case, the balancer will not decrease the effective shard quota below the largest bps value of each of these two properties.

If set to 0.0, the minimum is disabled. If set to 1.0, the balancer won’t be able to rebalance quota without violating this ratio, preventing the balancer from adjusting shards' quotas.

Unit: ratio of default shard quota

Related topics:

Requires restart: No

Optional: Yes

Visibility: user

Type: number

Default: 0.01

Related topics:

Intra-node throughput quota balancer invocation period, in milliseconds. When set to 0, the balancer is disabled and makes all the throughput quotas immutable.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 0

Time window used to average current throughput measurement for quota balancer, in milliseconds.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 5000

Maximum size of a single request processed using the Kafka API.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: 104857600

Maximum size of the user-space receive buffer. If null, this limit is not applied.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Size of the Kafka server TCP receive buffer. If null, the property is disabled.

Unit: bytes

Requires restart: No

Optional: No

Visibility: None

Type: integer

Accepted values: [-2147483648, 2147483647] aligned to 4096 bytes

Default: null

Size of the Kafka server TCP transmit buffer. If null, the property is disabled.

Unit: bytes

Requires restart: No

Optional: No

Visibility: None

Type: integer

Accepted values: [-2147483648, 2147483647] aligned to 4096 bytes

Default: null

The maximum time between Kafka client reauthentications. If a client has not reauthenticated a connection within this time frame, that connection is torn down.

Unit: milliseconds

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: null

Per-shard capacity of the cache for validating schema IDs.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 128

TCP keepalive idle timeout in seconds for Kafka connections. This describes the timeout between TCP keepalive probes that the remote site successfully acknowledged. Refers to the TCP_KEEPIDLE socket option. When changed, applies to new connections only.

Unit: seconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: 120

TCP keepalive probe interval in seconds for Kafka connections. This describes the timeout between unacknowledged TCP keepalives. Refers to the TCP_KEEPINTVL socket option. When changed, applies to new connections only.

Unit: seconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: 60

TCP keepalive unacknowledged probes until the connection is considered dead for Kafka connections. Refers to the TCP_KEEPCNT socket option. When changed, applies to new connections only.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: 3

List of throughput control groups that define exclusions from node-wide throughput limits. Clients excluded from node-wide throughput limits are still potentially subject to client-specific throughput limits.

Each throughput control group consists of:

Example values:

A connection is assigned the first matching group and is then excluded from throughput control. A name is not required, but can help you categorize the exclusions. Specifying +empty for the client_id will match on clients that opt not to send a client_id. You can also optionally omit the client_id and specify only a name, as shown. In this situation, all clients will match the rule and Redpanda will exclude them from all from node-wide throughput control.

Requires restart: No

Optional: Yes

Visibility: user

Type: string array

Accepted Values: list of control groups of the format {'name' : 'group name', 'client_id' : 'regex pattern'}

Default: [] (empty list)

Related topics:

List of Kafka API keys that are subject to cluster-wide and node-wide throughput limit control.

Requires restart: No

Optional: Yes

Visibility: user

Type: list<string>

Default: ["produce", "fetch"]

The maximum rate of all ingress Kafka API traffic for a node. Includes all Kafka API traffic (requests, responses, headers, fetched data, produced data, etc.).

If null, the property is disabled, and traffic is not limited.

Unit: bytes per second

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [-9223372036854775808, 9223372036854775807]

Default: null

Related topics:

The maximum rate of all egress Kafka traffic for a node. Includes all Kafka API traffic (requests, responses, headers, fetched data, produced data, etc.).

If null, the property is disabled, and traffic is not limited.

Unit: bytes per second

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [-9223372036854775808, 9223372036854775807]

Default: null

Related topics:

Threshold for refilling the token bucket as part of enforcing throughput limits. This only applies when kafka_throughput_throttling_v2 is true.

This threshold is evaluated with each request for data. When the number of tokens to replenish exceeds this threshold, then tokens are added to the token bucket. This ensures that the atomic is not being updated for the token count with each request. The range for this threshold is automatically clamped to the corresponding throughput limit for ingress and egress.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Accepted values: For ingress, [1, kafka_throughput_limit_node_in_bps]. For egress, [1, kafka_throughput_limit_node_out_bps]

Default: 1

Related topics:

Enables an updated algorithm for enforcing node throughput limits based on a shared token bucket, introduced with Redpanda v23.3.8. Set this property to false if you need to use the quota balancing algorithm from Redpanda v23.3.7 and older. This property defaults to true for all new or upgraded Redpanda clusters.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: true

Key-value store flush interval (in milliseconds).

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10

Key-value maximum segment size (in bytes).

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 16777216

Leadership rebalancing idle timeout.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 120000 (2min)

Mode of the leader balancer for optimizing movements of leadership between shards (logical CPU cores). Enabled by enable_leader_balancer.

Requires restart: No

Optional: Yes

Visibility: user

Type: string

Accepted Values:

Default: random_hill_climbing

Leadership rebalancing mute timeout.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 300000 (5min)

Leadership rebalancing node mute timeout.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 20000

Per shard limit for in-progress leadership transfers.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 512

Group offset retention is enabled by default starting in Redpanda version 23.1. To enable offset retention after upgrading from an older version, set this option to true.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

Flag to enable a Redpanda cluster operator to use unsafe control characters within strings, such as consumer group names or user names.

This flag applies only for Redpanda clusters that were originally on version 23.1 or earlier and have been upgraded to version 23.2 or later. Starting in version 23.2, newly-created Redpanda clusters ignore this property.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Related properties: legacy_unsafe_log_warning_interval_sec

Period at which to log a warning about using unsafe strings containing control characters.

If unsafe strings are permitted by legacy_permit_unsafe_log_operation, a warning will be logged at an interval specified by this property.

Unit: seconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: 300

Related properties: legacy_permit_unsafe_log_operation

Default cleanup policy for topic logs.

The topic property cleanup.policy overrides the value of log_cleanup_policy at the topic level.

Requires restart: No

Optional: Yes

Visibility: user

Accepted Values: compact, delete, compact,delete

Default: delete

How often to trigger background compaction.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10000

Use sliding window compaction.

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: boolean

Default: true

Default topic compression type.

The topic property compression.type overrides the value of log_compression_type at the topic level.

Requires restart: No

Optional: Yes

Visibility: user

Accepted Values: gzip, snappy, lz4, zstd, producer, none.

Default: producer

Disables the housekeeping loop for local storage.

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

Threshold in milliseconds for alerting on messages with a timestamp after the broker’s time, meaning the messages are in the future relative to the broker’s clock.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 7200000 (2h)

Threshold in milliseconds for alerting on messages with a timestamp before the broker’s time, meaning the messages are in the past relative to the broker’s clock. To disable this check, set to null.

Unit: milliseconds

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: null

Default timestamp type for topic messages (CreateTime or LogAppendTime).

The topic property message.timestamp.type overrides the value of log_message_timestamp_type at the topic level.

Requires restart: No

Optional: Yes

Visibility: user

Accepted Values: CreateTime, LogAppendTime.

Default: CreateTime

The amount of time to keep a log file before deleting it (in milliseconds). If set to -1, no time limit is applied. This is a cluster-wide default when a topic does not set or disable retention.ms.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Accepted values: [-17592186044416, 17592186044415]

Default: 604800000 (one week)

Default lifetime of log segments. If null, the property is disabled, and no default lifetime is set. Any value under 60 seconds (60000 ms) is rejected. This property can also be set in the Kafka API using the Kafka-compatible alias, log.roll.ms.

The topic property segment.ms overrides the value of log_segment_ms at the topic level.

Unit: milliseconds

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 1209600000 (2 weeks)

Related properties:

Upper bound on topic segment.ms: higher values will be clamped to this value.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 31536000000 (one year)

Lower bound on topic segment.ms: lower values will be clamped to this value.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 600000 (10min)

Default log segment size in bytes for topics which do not set segment.bytes.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: 134217728

Random variation to the segment size limit used for each partition.

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 65535]

Default: 5

Upper bound on topic segment.bytes: higher values will be clamped to this limit.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: null

Lower bound on topic segment.bytes: lower values will be clamped to this limit.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: 1048576

Disable reusable preallocated buffers for LZ4 decompression.

Requires restart: Yes

Nullable: No

Visibility: tunable

Type: boolean

Default: false

Maximum compacted segment size after consolidation.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 5368709120

Maximum number of the active producers sessions. When the threshold is passed, Redpanda terminates old sessions. When an idle producer corresponding to the terminated session wakes up and produces, its message batches are rejected, and an out of order sequence error is emitted. Consumers don’t affect this setting.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: 18446744073709551615

Maximum number of in-flight HTTP requests to HTTP Proxy permitted per shard. Any additional requests above this limit will be rejected with a 429 error.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 500

Maximum number of in-flight HTTP requests to Schema Registry permitted per shard. Any additional requests above this limit will be rejected with a 429 error.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 500

Fail-safe maximum throttle delay on Kafka requests.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 30000

Specifies the maximum number of active transaction sessions per coordinator. When the threshold is passed Redpanda terminates old sessions. When an idle producer corresponding to the terminated session wakes up and produces, it leads to its batches being rejected with invalid producer epoch or invalid_producer_id_mapping error (depends on the transaction execution phase).

For details, see Transaction usage tips.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: 18446744073709551615

Time between members backend reconciliation loop retries.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 5000 (5s)

If true, the Redpanda process will terminate immediately when an allocation cannot be satisfied due to memory exhaustion. If false, an exception is thrown.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: true

Interval for metadata dissemination batching.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 3000

Number of attempts to look up a topic’s metadata-like shard before a request fails.

This configuration controls the number of retries that request handlers perform when internal topic metadata (for topics like tx, consumer offsets, etc) is missing. These topics are usually created on demand when users try to use the cluster for the first time and it may take some time for the creation to happen and the metadata to propagate to all the brokers (particularly the broker handling the request). In the mean time Redpanda waits and retry. This configuration controls the number retries.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-32768, 32767]

Default: 30

Delay before retrying a topic lookup in a shard or other meta tables.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 500

Maximum time to wait in metadata request for cluster health to be refreshed.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 2000

Cluster metrics reporter report interval.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 86400000 (one day)

Cluster metrics reporter tick interval.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 60000 (1min)

URL of the cluster metrics reporter.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: string

Default: https://m.rp.vectorized.io/v2

Minimum allowable replication factor for topics in this cluster. The set value must be positive, odd, and equal to or less than the number of available brokers. Changing this parameter only restricts newly-created topics. Redpanda returns an INVALID_REPLICATION_FACTOR error on any attempt to create a topic with a replication factor less than this property.

If you change the minimum_topic_replications setting, the replication factor of existing topics remains unchanged. However, Redpanda will log a warning on start-up with a list of any topics that have fewer replicas than this minimum. For example, you might see a message such as Topic X has a replication factor less than specified minimum: 1 < 3.

Unit: minimum number of replicas per topic

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [1, 32767]

Default: 1

How long after the last heartbeat request a node will wait before considering itself to be isolated.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-9223372036854775808, 9223372036854775807]

Default: 3000

Timeout for executing node management operations.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 5000 (5s)

Time interval between two node status messages. Node status messages establish liveness status outside of the Raft protocol.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 100

Maximum backoff (in milliseconds) to reconnect to an unresponsive peer during node status liveness checks.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 15000

The amount of time (in seconds) to allow for when validating the expiry claim in the token.

Unit: seconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: 30

The URL pointing to the well-known discovery endpoint for the OIDC provider.

Requires restart: No

Optional: Yes

Visibility: user

Type: string

Default: https://auth.prd.cloud.redpanda.com/.well-known/openid-configuration

The frequency of refreshing the JSON Web Keys (JWKS) used to validate access tokens.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: 3600000

Rule for mapping JWT payload claim to a Redpanda user principal.

Requires restart: No

Optional: Yes

Visibility: user

Type: string

Default: $.sub

A string representing the intended recipient of the token.

Requires restart: No

Optional: Yes

Visibility: user

Type: string

Default: redpanda

Number of partitions that can be reassigned at once.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 50

When the disk usage of a node exceeds this threshold, it triggers Redpanda to move partitions off of the node.

Unit: percent of disk used

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [0, 4294967295]

Default: 80

Related topics:

Minimum size of partition that is going to be prioritized when rebalancing a cluster due to the disk size threshold being breached. This value is calculated automatically by default.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Mode of partition balancing for a cluster.

Requires restart: No

Optional: Yes

Visibility: user

Accepted values:

Default: node_add

Related topics:

When a node is unavailable for at least this timeout duration, it triggers Redpanda to move partitions off of the node.

Unit: seconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: 900 (15min)

Related topics:

Partition autobalancer tick interval.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 30000 (30s)

If the number of scheduled tick moves drops by this ratio, a new tick is scheduled immediately. Valid values are (0, 1]. For example, with a value of 0.2 and 100 scheduled moves in a tick, a new tick is scheduled when the in-progress moves are fewer than 80.

Requires restart: No

Nullable: No

Visibility: tunable

Type: number

Default: 0.2

If true, Redpanda prioritizes balancing a topic’s partition replica count evenly across all brokers while it’s balancing the cluster’s overall partition count. Because different topics in a cluster can have vastly different load profiles, this better distributes the workload of the most heavily-used topics evenly across brokers.

Requires restart: no

Optional: Yes

Type: boolean

Default: true

Related topics:

A threshold value to detect partitions which might have been stuck while shutting down. After this threshold, a watchdog in partition manager will log information about partition shutdown not making progress.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 30000

Maximum number of Cross-core(Inter-shard communication) requests pending in HTTP Proxy and Schema Registry seastar::smp group. (For more details, see the seastar::smp_service_group documentation).

See Seastar documentation

Requires restart: Yes

Optional: No

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: null

Quota manager GC frequency in milliseconds.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 30000 (30s)

Maximum delay between two subsequent flushes. After this delay, the log is automatically force flushed.

Unit: milliseconds

Requires restart: No

Nullable: No

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 100

Election timeout expressed in milliseconds.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 1500

Enables an additional step in leader election where a candidate is allowed to wait for all the replies from the broker it requested votes from. This may introduce a small delay when recovering from failure, but it prevents truncation if any of the replicas have more data than the majority.

Requires restart: No

Nullable: No

Visibility: tunable

Type: boolean

Default: true

Enables Raft optimization of heartbeats.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: true

The number of failed heartbeats after which an unresponsive TCP connection is forcibly closed. To disable forced disconnection, set to 0.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 3

Number of milliseconds for Raft leader heartbeats.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 150

Raft heartbeat RPC (remote procedure call) timeout.

Raft uses a heartbeat mechanism to maintain leadership authority and to trigger leader elections. The raft_heartbeat_interval_ms is a periodic heartbeat sent by the partition leader to all followers to assert its leadership. If a follower does not receive a heartbeat within the raft_heartbeat_timeout_ms, then it triggers an election to choose a new partition leader.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 3000

Raft I/O timeout.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10000

Raft learner recovery rate limit. Throttles the rate of data communicated to nodes (learners) that need to catch up to leaders.

This rate limit is placed on a node sending data to a recovering node. Each sending node is limited to this rate. The recovering node accepts data as fast as possible according to the combined limits of all healthy nodes in the cluster. For example, if two nodes are sending data to the recovering node, and raft_learner_recovery_rate is 100 MB/sec, then the recovering node will recover at a rate of 200 MB/sec.

Unit: bytes per second.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 104857600

Maximum number of concurrent append entry requests sent by the leader to one follower.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: 16

Maximum memory that can be used for reads in Raft recovery process by default 15% of total memory.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: null

Number of partitions that may simultaneously recover data to a particular shard. This number is limited to avoid overwhelming nodes when they come back online after an outage.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 64

Specifies the default size of a read issued during Raft follower recovery.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 524288

Disables cross shard sharing used to throttle recovery traffic. Should only be used to debug unexpected problems.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

Maximum number of bytes that are not flushed per partition. If the configured threshold is reached, the log is automatically flushed even if it has not been explicitly requested.

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Default: 262144

Maximum size of requests cached for replication.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 1048576

Maximum number of Cross-core(Inter-shard communication) requests pending in Raft seastar::smp group. For details, refer to the seastar::smp_service_group documentation).

See Seastar documentation

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: null

Timeout for Raft’s timeout_now RPC. This RPC is used to force a follower to dispatch a round of votes immediately.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 1000

Follower recovery timeout waiting period when transferring leadership.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10000

Duration after which inactive readers are evicted from cache.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 30000

Maximum desired number of readers cached per NTP. This a soft limit, meaning that a number of readers in cache may temporarily increase as cleanup is performed in the background.

Requires restart: No

Nullable: No

Visibility: tunable

Type: integer

Default: 200

Minimum amount of free memory maintained by the batch cache background reclaimer.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 67108864

Starting from the last point in time when memory was reclaimed from the batch cache, this is the duration during which the amount of memory to reclaim grows at a significant rate, based on heuristics about the amount of available memory.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 3000

Maximum batch cache reclaim size.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 4194304

Minimum batch cache reclaim size.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 131072

If the duration since the last time memory was reclaimed is longer than the amount of time specified in this property, the memory usage of the batch cache is considered stable, so only the minimum size (reclaim_min_size) is set to be reclaimed.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10000

Timeout for append entry requests issued while updating a stale follower.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 5000

Flag for specifying whether or not to release cache when a full segment is rolled.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

Timeout for append entry requests issued while replicating entries.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 300

Default maximum number of bytes per partition on disk before triggering deletion of the oldest messages. If null (the default value), no limit is applied.

The topic property retention.bytes overrides the value of retention_bytes at the topic level.

Unit: bytes per partition.

Requires restart: No

Optional: No

Visibility: user

Type: integer

Default: null

Flag to allow Tiered Storage topics to expand to consumable retention policy limits.

When this flag is enabled, non-local retention settings are used, and local retention settings are used to inform data removal policies in low-disk space scenarios.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

Trim log data when a cloud topic reaches its local retention limit. When this option is disabled Redpanda will allow partitions to grow past the local retention limit, and will be trimmed automatically as storage reaches the configured target size.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Local retention size target for partitions of topics with object storage write enabled. If null, the property is disabled.

This property can be overridden on a per-topic basis by setting retention.local.target.bytes in each topic enabled for Tiered Storage. See Configure message retention.

Related properties:

Unit: bytes

Requires restart: No

Optional: No

Visibility: user

Type: integer

Default: null

The target capacity (in bytes) that log storage will try to use before additional retention rules take over to trim data to meet the target. When no target is specified, storage usage is unbounded.

Unit: percentage of total disk size

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [0, 18446744073709551615]

Default: null

The target capacity in percent of unreserved space (disk_reservation_percent) that log storage will try to use before additional retention rules will take over to trim data in order to meet the target. When no target is specified storage usage is unbounded.

Unit: percentage of total disk size

Requires restart: No

Optional: No

Visibility: user

Type: number

Default: 80.0

Local retention time target for partitions of topics with object storage write enabled.

This property can be overridden on a per-topic basis by setting retention.local.target.ms in each topic enabled for Tiered Storage. See Configure message retention.

Related properties:

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 86400000 (one day)

The period during which disk usage is checked for disk pressure, and data is optionally trimmed to meet the target.

Unit: seconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 30

The space management control loop reclaims the overage multiplied by this this coefficient to compensate for data that is written during the idle period between control loop invocations.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: number

Default: 2.0

Resource manager’s synchronization timeout. Specifies the maximum time for this node to wait for the internal state machine to catch up with all events written by previous leaders before rejecting a request.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: user

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 10000

The maximum number of connections a broker will open to each of its peers.

Requires restart: No

Optional: Yes

Visibility: None

Type: integer

Accepted values: [-2147483648, 2147483647]

Default: 32

Enable compression for internal RPC (remote procedure call) server replies.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

Maximum TCP connection queue length for Kafka server and internal RPC server. If null (the default value), no queue length is set.

Unit: number of queue entries

Requires restart: No

Optional: No

Visibility: user

Type: integer

Accepted values: [-2147483648, 2147483647]

Default: null

Internal RPC TCP receive buffer size. If null (the default value), no buffer size is set by Redpanda.

Unit: bytes

Requires restart: No

Optional: No

Visibility: None

Type: integer

Accepted values: [-2147483648, 2147483647]

Default: null

Internal RPC TCP send buffer size. If null (the default value), then no buffer size is set by Redpanda.

Unit: bytes

Requires restart: No

Optional: No

Visibility: None

Type: integer

Accepted values: [-2147483648, 2147483647]

Default: null

Rate limit for controller ACLs and user’s operations.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 1000

Rate limit for controller configuration operations.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 1000

Rate limit for controller move operations.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 1000

Rate limit for controller node management operations.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 1000

Rate limit for controller topic operations.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 1000

When true, memory allocations are sampled and tracked. A sampled live set of allocations can then be retrieved from the Admin API. Additionally, Redpanda will periodically log the top-n allocation sites.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: true

The location of the Kerberos krb5.conf file for Redpanda.

Requires restart: No

Optional: Yes

Visibility: user

Type: string

Default: /etc/krb5.conf

The location of the Kerberos keytab file for Redpanda.

Type: string

Requires restart: No

Optional: Yes

Visibility: user

Default: /var/lib/redpanda/redpanda.keytab

The primary of the Kerberos Service Principal Name (SPN) for Redpanda.

Requires restart: No

Optional: Yes

Visibility: user

Type: string

Default: redpanda

Rules for mapping Kerberos principal names to Redpanda user principals.

Requires restart: No

Optional: Yes

Visibility: user

Type: string array

Default: [default]

A list of supported SASL mechanisms.

Requires restart: No

Optional: Yes

Visibility: user

Type: string array

Accepted values: SCRAM, GSSAPI, OAUTHBEARER.

Default: [SCRAM]

Normalize schemas as they are read from the topic on startup.

Requires restart: Yes

Optional: No

Visibility: user

Type: boolean

Default: false

Maximum delay until buffered data is written.

Unit: milliseconds

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-17592186044416, 17592186044415]

Default: 1000 (1s)

Size for segments fallocation.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 33554432

Option to explicitly disable automatic disk space management. If this property was explicitly disabled while using v23.2, it will remain disabled following an upgrade.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: true

Maximum parallel logs inspected during space management process.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 65535]

Default: 20

Maximum parallel segments inspected during space management process.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 65535]

Default: 10

Maximum number of bytes that may be used on each shard by compaction index writers.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: 134217728

Maximum number of bytes that may be used on each shard by compaction key-offset maps. Only applies when log_compaction_use_sliding_window is set to true.

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: 134217728

Limit on storage_compaction_key_map_memory, expressed as a percentage of memory per shard, that bounds the amount of memory used by compaction key-offset maps.

Requires restart: Yes

Optional: Yes

Visibility: tunable

Type: number

Default: 12.0

When set, cstore hints are ignored and not used for data access (but are otherwise generated).

Requires restart: No

Optional: Yes

Visibility: tunable

Type: boolean

Default: false

The maximum number of seconds that a record’s timestamp can be ahead of a Redpanda broker’s clock and still be used when deciding whether to clean up the record for data retention. This property makes possible the timely cleanup of records from clients with clocks that are drastically unsynchronized relative to Redpanda.

When determining whether to clean up a record with timestamp more than storage_ignore_timestamps_in_future_sec seconds ahead of the broker, Redpanda ignores the record’s timestamp and instead uses a valid timestamp of another record in the same segment, or (if another record’s valid timestamp is unavailable) the timestamp of when the segment file was last modified (mtime).

By default, storage_ignore_timestamps_in_future_sec is disabled (null).

Unit: seconds

Requires restart: No

Optional: No

Visibility: tunable

Type: integer

Accepted values: [-17179869184, 17179869183]

Default: null

Maximum number of partitions' logs that will be replayed concurrently at startup, or flushed concurrently on shutdown.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]

Default: 1024

Threshold of minimum bytes free space before rejecting producers.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 5368709120

Size of each read buffer (one per in-flight read, per log segment).

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 131072

How many additional reads to issue ahead of current read location.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-32768, 32767]

Default: 10

The number of segments per partition that the system will attempt to reserve disk capacity for. For example, if the maximum segment size is configured to be 100 MB, and the value of this option is 2, then in a system with 10 partitions Redpanda will attempt to reserve at least 2 GB of disk space.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [-32768, 32767]

Default: 2

Threshold of minimum bytes free space before setting storage space alert.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Default: 0

Threshold of minimum percent free space before setting storage space alert.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 4294967295]

Default: 5

Requires that an empty file named .redpanda_data_dir be present in the data_ directory. If set to true, Redpanda will refuse to start if the file is not found in the data directory.

Requires restart: No

Optional: Yes

Visibility: user

Type: boolean

Default: false

Target bytes to replay from disk on startup after clean shutdown: controls frequency of snapshots and checkpoints.

Requires restart: No

Optional: Yes

Visibility: tunable

Type: integer

Accepted values: [0, 18446744073709551615]