Streaming

Configure Topic Properties

Topic properties control Redpanda’s behavior for individual topics, including data retention, cleanup policies, compression settings, and Tiered Storage configurations. These properties let you customize how Redpanda stores, processes, and manages data for each topic, overriding cluster-wide defaults when needed.

Redpanda stores topic properties as metadata associated with each topic and replicates them across the cluster to ensure consistency. Many topic properties correspond to cluster properties that establish default values for all topics. When you set a topic property, it overrides the corresponding cluster default for that specific topic.

All topic properties take effect immediately after being set. Do not modify properties on internal Redpanda topics (such as __consumer_offsets, _schemas, or other system topics) as this can cause cluster instability.

For a complete reference of available topic properties and their corresponding cluster properties, see Topic Configuration Properties.

Configuration methods

You can configure topic properties through multiple interfaces:

  • rpk commands - Use rpk topic create with the -c flag to set properties during topic creation, or rpk topic alter-config to modify existing topics.

  • Kafka Admin API - Any Kafka-compatible client can use the standard Admin API to configure topic properties.

  • Kubernetes Topic resources - In Kubernetes deployments, configure topic properties using the additionalConfig field in Topic resources. See Manage Topics in Kubernetes.

  • Redpanda Console - Use the web interface to edit topic configuration through a graphical interface.

Verify topic property configuration

Use rpk topic describe <topic> to view topic properties and their configuration sources:

rpk topic describe my-topic

The output shows two sections:

SUMMARY
=======
NAME        my-topic
PARTITIONS  3
REPLICAS    3

CONFIGS
=======
KEY                     VALUE          SOURCE
cleanup.policy          delete         DEFAULT_CONFIG
compression.type        producer       DEFAULT_CONFIG
retention.ms            604800000      DEFAULT_CONFIG
write.caching           true           DYNAMIC_TOPIC_CONFIG

The SOURCE column indicates how each property is configured:

  • DEFAULT_CONFIG - Redpanda’s default value

  • DYNAMIC_TOPIC_CONFIG - User-configured value

The replication factor appears as REPLICAS in the SUMMARY section, not as replication.factor in the CONFIGS list. However, you need to use the replication.factor key when modifying the value with rpk topic alter-config.

For partition-level details, add the -p flag:

rpk topic describe my-topic -p

This shows a different output focused on partition information:

PARTITION  LEADER  EPOCH  REPLICAS  LOG-START-OFFSET  HIGH-WATERMARK
0          1       2     [1 2 3]   0                 6
1          2       4     [1 2 3]   0                 10
2          3       1     [1 2 3]   0                 8

Examples

The following examples show how to configure topic-level properties. Set a topic-level property for a topic to override the value of corresponding cluster property.

Create topic with topic properties

To set topic properties when creating a topic, use the rpk topic create command with the -c option.

For example, to create a topic with the cleanup.policy property set to compact:

  • Local

  • Kubernetes

rpk topic create -c cleanup.policy=compact <topic-name>
kubectl exec <pod-name> -- rpk topic create -c cleanup.policy=compact <topic-name>

To configure multiple properties for a topic, use the -c option for each property.

For example, to create a topic with all necessary properties for Tiered Storage:

  • Local

  • Kubernetes

rpk topic create -c redpanda.remote.recovery=true -c redpanda.remote.write=true -c redpanda.remote.read=true <topic-name>
kubectl exec <pod-name> -- rpk topic create -c redpanda.remote.recovery=true -c redpanda.remote.write=true -c redpanda.remote.read=true <topic-name>

Modify topic properties

To modify topic properties of an existing topic, use the rpk topic alter-config command.

For example, to modify a topic’s retention.ms property:

  • Local

  • Kubernetes

rpk topic alter-config <topic-name> --set retention.ms=<retention-time>
kubectl exec <pod-name> -- rpk topic alter-config <topic-name> --set retention.ms=<retention-time>

Configure topic properties with Kubernetes

In Kubernetes deployments, configure topic properties using the additionalConfig field in Topic resources:

apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
  name: example-topic
spec:
  partitions: 3
  replicationFactor: 3
  additionalConfig:
    cleanup.policy: "compact"
    retention.ms: "604800000"
    segment.ms: "86400000"

Apply the configuration:

kubectl apply -f topic-config.yaml

Common topic property categories

The most commonly configured topic properties fall into these main categories:

Disk space management

Redpanda manages disk space through two main mechanisms: compaction (removing duplicate keys) and retention (removing old data).

Choose your cleanup strategy with cleanup.policy: - delete - Remove old data based on time or size limits - compact - Keep only the latest value for each key - compact,delete - Combine both strategies

Compaction

When using cleanup.policy=compact or cleanup.policy=compact,delete, configure:

Retention

When using cleanup.policy=delete or cleanup.policy=compact,delete, configure:

  • retention.bytes - Maximum size before cleanup (size-based retention)

  • retention.ms - Maximum age before cleanup (time-based retention)

  • segment.bytes - Control how frequently cleanup can occur by setting segment size

Performance

Essential performance tuning properties:

For complete details about all available topic properties, see Topic Configuration Properties.

Suggested labs

See more
Search all labs