What’s New

This topic includes new content added in version 24.3. For a complete list of all product updates, see the Redpanda release notes. See also:

Leader pinning

For a Redpanda cluster deployed across multiple availability zones (AZs), leader pinning ensures that a topic’s partition leaders are geographically closer to clients. Leader pinning can lower networking costs and help guarantee lower latency by routing produce and consume requests to brokers located in certain AZs.

Iceberg topics

The Iceberg integration for Redpanda allows you to store topic data in the cloud in the Iceberg open table format. This makes your streaming data immediately available in downstream analytical systems without setting up and maintaining additional ETL pipelines. You can also integrate your data directly into commonly-used big data processing frameworks, standardizing and simplifying the consumption of streams as tables in a wide variety of data analytics pipelines.

Mountable topics

For topics with Tiered Storage enabled, you can unmount a topic to safely detach it from a cluster and keep the topic data in the cluster’s object storage bucket or container. You can mount the detached topic to either the same origin cluster, or a different one. This allows you to hibernate a topic and free up system resources taken up by the topic, or migrate a topic to a different cluster. See Mountable topics for details.

Intra-broker partition balancing

Intra-broker partition balancing has moved out of beta and is supported for production clusters. This dynamically balances partitions across cores within a Redpanda broker. You also can decrease the number of CPUs assigned to a broker. Existing partition replicas are reassigned to the remaining CPU cores when the broker restarts with fewer available cores. You must explicitly enable this feature.

Tombstone removal

Redpanda now supports the Kafka delete.retention.ms topic configuration. You can specify how long Redpanda keeps tombstone records for compacted topics by setting delete.retention.ms at the topic level, or tombstone_retention_ms at the cluster level.

Debug bundles in Redpanda Console

You can now generate a debug bundle in Redpanda Console for comprehensive diagnostics. A debug bundle can help debug and diagnose issues with a Redpanda cluster, a broker, or the machines on which the brokers are running. You can use this file to debug issues yourself, or you can send it to the Redpanda support team to help resolve your issue.

Backfill partitions

When running node-wise partition recovery, it’s possible that there may be more recent data (a higher offset) available in Tiered Storage. Redpanda attempts to recover partition data from object storage, recovering the latest offset available for a partition in either storage tier (local or object storage). This allows for the maximum amount of data to be recovered in all cases, even for topics with a replication factor of 1, where no replicas remain in local storage.

Configure access to object storage with a KMS key

Users on AWS or GCP with strict data compliance requirements can manage and store encryption keys separately from their cloud provider with a customer-managed Key Management Service (KMS) key.

Licensing updates

This release includes several updates to Redpanda’s licensing system to both improve transparency and make it easier to manage licenses across Redpanda clusters and Redpanda Console.

  • Upgrade restriction for enterprise features: You cannot upgrade your cluster to Redpanda 24.3 or later if any enterprise feature of Redpanda (excluding Console) is in use without a valid license. The upgrade is aborted immediately and cleanly before any broker is upgraded, minimizing disruption.

  • License reminder in rpk: Every rpk command now includes a reminder if the cluster is using licensed features without a valid license.

  • License warnings in Redpanda Console: If you are using Redpanda Console and one or more enterprise features are active in the connected Redpanda cluster without a valid license, a banner specifies which enterprise features are in use without a license.

  • Unified license management in Redpanda Console: You can now upload and apply a single license key for both Redpanda Console and the connected Redpanda cluster through the Redpanda Console UI. Any existing license key is overridden by the new one.

  • 30-day trial enterprise license: Starting with version 24.3, new Redpanda clusters automatically receive a trial license that’s valid for 30 days, allowing unrestricted use of Enterprise features. This evaluation period begins when the cluster is created for the first time. After this period expires, inactive Enterprise features are disabled, and active features enter a restricted state.

PKCS#12 support for TLS listeners

Redpanda now supports configuring TLS using PKCS#12 files, which allows you to manage both the certificate and private key in a single, password-protected file.

For more details, see Configure Kafka TLS Encryption.

This feature is not yet available in Kubernetes deployments that use the Redpanda Helm chart or the Redpanda Operator.

New features in the Redpanda Operator

Declarative user and ACL management

The Redpanda Operator now supports declarative management of users and access control lists (ACLs) using the new User custom resource, starting in version v2.2.2-24.2.4. This feature allows you to:

  • Create and manage Redpanda users and their authentication settings.

  • Define and manage ACLs to control access to Redpanda resources.

  • Automatically reconcile changes to users and ACLs using the Redpanda Operator.

To learn more, see the User custom resource documentation.

Declarative schema management

The Redpanda Operator now supports declarative schema management using the Schema custom resource, starting in version v2.3.0-24.3.1. This feature allows you to:

  • Define, create, and manage Avro, Protobuf, and JSON schemas declaratively.

  • Enforce schema compatibility and evolution policies directly in Kubernetes.

  • Automate schema lifecycle management alongside Redpanda deployments.

To learn more, see the Schema custom resource documentation.

Use Redpanda Operator without Flux (beta)

The Redpanda Operator now supports the useFlux flag, giving you control over resource management, starting in version v2.3.0-24.3.1:

  • useFlux: true (default): Delegates Redpanda resource management to Flux controllers through HelmRelease resources.

  • useFlux: false: Directly manages resources within the Redpanda Operator, bypassing Flux.

    The useFlux:false configuration is in beta. It is not supported for production deployments. Beta features are available for users to test and provide feedback.

Example:

spec:
  clusterSpec:
    useFlux: false

New commands

The following rpk commands are new in this version:

New properties

The following cluster properties are new in this version: