Cloud

Tiered Storage Versions

Tiered Storage moves topic data from brokers to object storage, and Redpanda Cloud enables it by default. Tiered Storage v2 is in beta and is rolling out progressively across Redpanda Cloud. It is available on BYOC and Dedicated clusters running Redpanda 26.2 or later. Serverless clusters use Tiered Storage v1. Choose Tiered Storage v2 for new compacted topics that must not accumulate stale values or tombstones in object storage.

Starting in Redpanda v26.2, Tiered Storage is available in two versions:

  • Tiered Storage v1 (tiered_v1) is the original implementation and remains the default.

  • Tiered Storage v2 (tiered_v2) (beta) is a new implementation that adds full compaction support, including tombstone removal, for topics stored in object storage.

Tiered Storage v2 is a beta feature. It is not supported for production deployments. Beta features are available for users to test and provide feedback.

You enable both versions the same way, by setting redpanda.storage.mode=tiered on a topic, and the storage mode of a topic displays as tiered for both. The version used by each topic is reported by the read-only redpanda.storage.mode.impl topic property.

Differences between Tiered Storage v1 and v2

The two versions differ in how they compact topics with a compact cleanup policy:

Tiered Storage v1 Tiered Storage v2

Compacted data

Compaction runs on local data only. Data that has already been uploaded to object storage is not compacted, so duplicate keys can remain in object storage indefinitely.

Compaction runs on the data in object storage, so the entire partition is compacted.

Compaction window

Compaction is limited to a small window of data, and only duplicate keys that fall within the same window are removed.

Full compaction: eventually only the latest value for each key is retained across the entire partition.

Tombstone removal

Not supported. Tombstone records are not removed by compaction.

Supported. Tombstone records are removed after the retention period configured by delete.retention.ms.

Select a Tiered Storage version

The tiered value of the redpanda.storage.mode topic property is an alias. When a topic is created, it resolves to the version selected by the default_redpanda_storage_mode_tiered_impl cluster property, which defaults to tiered_v1. To make all new tiered topics use Tiered Storage v2:

rpk cluster config set default_redpanda_storage_mode_tiered_impl=tiered_v2

Changing this property doesn’t require a cluster restart. You can set cluster properties with rpk or the Cloud API. For details, see Configure Cluster Properties.

To select the version explicitly for a single topic, regardless of the cluster-wide default, set both the redpanda.storage.mode and redpanda.storage.mode.impl properties when you create the topic:

rpk topic create <topic_name> -c redpanda.storage.mode=tiered -c redpanda.storage.mode.impl=tiered_v2

The two values must agree: both tiered_v1 and tiered_v2 correspond to the tiered storage mode. You can set redpanda.storage.mode.impl on its own only when it matches the version selected by the cluster default.

To check which version an existing topic uses, run rpk topic describe <topic_name>. The redpanda.storage.mode property displays tiered for both versions, and the redpanda.storage.mode.impl property reports tiered_v1 or tiered_v2.

Tiered Storage v2 restrictions

  • Tiered Storage v2 requires a BYOC or Dedicated cluster running Redpanda v26.2 or later.

  • Upgrading a cluster never changes the version of existing topics. Changing default_redpanda_storage_mode_tiered_impl affects only topics created after the change.

  • A topic cannot be converted between Tiered Storage v1 and v2. The redpanda.storage.mode.impl property is read-only and can be set only when the topic is created.

Next steps