Topic Configuration Properties

The cleanup policy to apply for log segments of a topic. When cleanup.policy is set, it overrides the cluster property log_cleanup_policy for the topic.

Values:

Specifies the strategy used to determine which records to remove during log compaction. The compaction strategy controls how Redpanda identifies and removes duplicate records while preserving the latest value for each key.

The retention time for tombstone records in a compacted topic. Redpanda removes tombstone records after the retention limit is exceeded.

If you have enabled Tiered Storage and set redpanda.remote.read or redpanda.remote.write for the topic, you cannot enable tombstone removal.

If both delete.retention.ms and the cluster property tombstone_retention_ms are set, delete.retention.ms overrides the cluster level tombstone retention for an individual topic.

The maximum amount of time (in ms) that a log segment can remain unaltered before it is eligible for compaction in a compact topic. Overrides the cluster property max_compaction_lag_ms for the topic.

The minimum ratio between the number of bytes in dirty segments and the total number of bytes in closed segments that must be reached before a partition’s log is eligible for compaction in a compact topic.

The minimum amount of time (in ms) that a log segment must remain unaltered before it can be compacted in a compact topic. Overrides the cluster property min_compaction_lag_ms for the topic.

A size-based retention limit that configures the maximum size that a topic partition can grow before becoming eligible for cleanup.

If retention.bytes is set to a positive value, it overrides the cluster property retention_bytes for the topic, and the total retained size for the topic is retention.bytes multiplied by the number of partitions for the topic.

When both size-based (retention.bytes) and time-based (retention.ms) retention limits are set, cleanup occurs when either limit is reached.

A time-based retention limit that configures the maximum duration that a log’s segment file for a topic is retained before it becomes eligible to be cleaned up. To consume all data, a consumer of the topic must read from a segment before its retention.ms elapses, otherwise the segment may be compacted and/or deleted. If a non-positive value, no per-topic limit is applied.

If retention.ms is set to a positive value, it overrides the cluster property log_retention_ms for the topic.

When both size-based (retention.bytes) and time-based (retention.ms) retention limits are set, the earliest occurring limit applies.

Redpanda ignores this property and always uses producer compression semantics. If producers send compressed data, Redpanda stores and serves it as-is. If producers send uncompressed data, Redpanda stores it uncompressed.

This property exists for Apache Kafka compatibility. Configure compression in your producers instead of using this topic property.

Compression reduces message size and improves throughput, but increases CPU utilization. Enable producer batching to increase compression efficiency.

When set, this property overrides the cluster property log_compression_type for the topic.

The maximum size of a message or batch of a topic. If a compression type is enabled, max.message.bytes sets the maximum size of the compressed message or batch.

If max.message.bytes is set to a positive value, it overrides the cluster property kafka_batch_max_bytes for the topic.

Set an upper limit for max.message.bytes using the cluster property kafka_max_message_size_upper_limit_bytes.

The source of a message’s timestamp: either the message’s creation time or its log append time.

When message.timestamp.type is set, it overrides the cluster property log_message_timestamp_type for the topic.

The maximum size of an active log segment for a topic. When the size of an active segment exceeds segment.bytes, the segment is closed and a new active segment is created. The closed, inactive segment is then eligible to be cleaned up according to retention properties.

When segment.bytes is set to a positive value, it overrides the cluster property:

The maximum duration that a log segment of a topic is active (open for writes and not deletable). A periodic event, with segment.ms as its period, forcibly closes the active segment and transitions, or rolls, to a new active segment. The closed (inactive) segment is then eligible to be cleaned up according to cleanup and retention properties.

If set to a positive duration, segment.ms overrides the cluster property log_segment_ms. Values are automatically clamped between the cluster bounds set by log_segment_ms_min (default: 10 minutes) and log_segment_ms_max (default: 1 year). If your configured value exceeds these bounds, Redpanda uses the bound value and logs a warning. Check current cluster bounds with rpk cluster config get log_segment_ms_min log_segment_ms_max.

The maximum bytes not fsynced per partition. If this configured threshold is reached, the log is automatically fsynced, even though it wasn’t explicitly requested.

The maximum delay (in ms) between two subsequent fsyncs. After this delay, the log is automatically fsynced.

The preferred location (rack) for partition leaders of a topic.

This property inherits the value from the default_leaders_preference cluster configuration property. You may override the cluster-wide setting by specifying the value for individual topics.

If the cluster configuration property enable_rack_awareness is set to false, Leader Pinning is disabled across the cluster.

The number of replicas of a topic to save in different nodes (brokers) of a cluster.

If replication.factor is set to a positive value, it overrides the cluster property default_topic_replication for the topic.

The write caching mode to apply to a topic.

When write.caching is set, it overrides the cluster property write_caching_default. Write caching acknowledges a message as soon as it is received and acknowledged on a majority of brokers, without waiting for it to be written to disk. With acks=all, this provides lower latency while still ensuring that a majority of brokers acknowledge the write. Fsyncs follow flush.ms and flush.bytes, whichever is reached first.

A size-based initial retention limit for Tiered Storage that determines how much data in local storage is transferred to a partition replica when a cluster is resized. If null (default), all locally retained data is transferred.

A time-based initial retention limit for Tiered Storage that determines how much data in local storage is transferred to a partition replica when a cluster is resized. If null (default), all locally retained data is transferred.

A flag that enables deletion of data from object storage for Tiered Storage when it’s deleted from local storage for a topic.

A flag for enabling Redpanda to fetch data for a topic from object storage to local storage. When set to true together with redpanda.remote.write, it enables the Tiered Storage feature.

A flag that enables the recovery or reproduction of a topic from object storage for Tiered Storage. The recovered data is saved in local storage, and the maximum amount of recovered data is determined by the local storage retention limits of the topic.

A flag for enabling Redpanda to upload data for a topic from local storage to object storage. When set to true together with redpanda.remote.read, it enables the Tiered Storage feature.

A size-based retention limit for Tiered Storage that configures the maximum size that a topic partition in local storage can grow before becoming eligible for cleanup. It applies per partition and is equivalent to retention.bytes without Tiered Storage.

A time-based retention limit for Tiered Storage that sets the maximum duration that a log’s segment file for a topic is retained in local storage before it’s eligible for cleanup. This property is equivalent to retention.ms without Tiered Storage.

The name of the object storage bucket for a Remote Read Replica topic.

Whether the corresponding Iceberg table is deleted upon deleting the topic.

Whether to write invalid records to a dead-letter queue (DLQ).

Enable the Iceberg integration for the topic. You can choose one of four modes.

The partitioning specification for the Iceberg table.

Controls how often the data in the Iceberg table is refreshed with new data from the topic. Redpanda attempts to commit all data produced to the topic within the lag target, subject to resource availability.