What’s New in Redpanda

This topic includes new content added in version 26.3. For a complete list of all product updates, see the Redpanda release notes. See also:

Custom Iceberg table names and namespaces

Set the Iceberg table name and namespace for each topic with the new redpanda.iceberg.table.name and redpanda.iceberg.table.namespace topic properties. Without these properties, Redpanda names the table after the topic and uses the cluster default namespace. Redpanda checks for configurations that would make two Iceberg-enabled topics write to the same table.

Iceberg column statistics and Bloom filters

Redpanda can write column statistics and Parquet Bloom filters to Iceberg tables. Query engines use them to skip data that cannot match a query, resulting in fewer reads.

  • Column statistics: Redpanda writes column bounds and counts for each data file to the table metadata. You do not have to configure anything.

  • Bloom filters: Redpanda writes a filter for each column that you select with an Iceberg table property in your catalog. Bloom filters speed up exact-match lookups on columns with many distinct values, such as IDs.

These features are also available in Redpanda v26.2.3. See Skip data files with column statistics and Use Bloom filters for point lookups.

OIDC: ECDSA and additional RSA signing algorithms

In addition to RS256, Redpanda now verifies OIDC access tokens signed with:

  • RS384

  • RS512

  • ES256

  • ES384

  • ES512

You can authenticate with tokens from an identity provider (IdP) or token service that signs with ECDSA keys or with a stronger RSA digest. The new algorithms apply to SASL/OAUTHBEARER on the Kafka API and to OIDC on the HTTP APIs.

Redpanda assigns each key in the IdP’s JSON Web Key Set (JWKS) exactly one algorithm, and rejects tokens whose alg header does not match the key’s algorithm. If your IdP publishes RSA keys without an alg member and signs with RS384 or RS512, set the new oidc_rsa_default_algorithm cluster property to that algorithm.

Faster client failover during leadership changes

Clients now move to a new partition leader faster during rolling restarts, maintenance-mode drains, and leadership balancing. A NOT_LEADER_OR_FOLLOWER error names the new leader, as defined in Kafka Improvement Proposal (KIP) 951, so clients that support it can retry without refreshing metadata. Brokers also briefly hold produce and fetch requests during the change, so the retry reaches a leader that can serve it. The new kafka_leader_hint_wait_ms cluster property sets the longest hold (default: 100 milliseconds). Set it to 0 to turn off these behaviors, so that requests fail immediately and name the leader known at that time.

Community Edition core limit

Redpanda Community Edition now runs clusters of up to 50 cores in total without an Enterprise Edition license. A cluster above the limit keeps running and serving clients, but it counts as using an enterprise feature. Redpanda logs license reminders and refuses upgrades to the next feature release until you install a license or reduce the core count. Because Redpanda 26.2 does not count cores against the limit, you must check your cluster’s core count before you upgrade a cluster without a license from 26.2.

Consumer group protocol (KIP-848)

Redpanda now supports the consumer group protocol defined in Kafka Improvement Proposal (KIP) 848. In this protocol, the broker computes partition assignments and hands partitions over incrementally. Consumers that keep their partitions keep consuming while the group rebalances, instead of every member pausing until the rebalance completes. Classic consumer groups are unaffected and run alongside groups that use the new protocol.

The protocol is disabled by default. Enable it with the group_consumer_protocol_enabled cluster property, then set group.protocol=consumer in your consumers. This release doesn’t support static membership or regex subscriptions, and a group switches protocols only when it has no active members.

Leaderless Cloud Topics

In a multi-AZ cluster, clients can now produce to and consume from Cloud Topics without crossing AZs. When a client identifies its AZ in its client.id, Redpanda advertises a replica in that AZ as the partition leader and forwards only the metadata for produced data to the actual leader. To use it, set the rack property on every broker, then enable the cloud_topics_leaderless_enabled and cloud_topics_az_client_id_token_enabled cluster properties.

Broker-side compression

Redpanda can now compress produced batches on the broker. This reduces storage and network use for topics whose producers send uncompressed data, without changing producer settings. Set the compression.type topic property or the log_compression_type cluster property to a codec. The broker then recompresses batches that use a different codec before storing them. Earlier versions ignored these properties. The new kafka_produce_enable_batch_compression cluster property turns the behavior on or off. It defaults to true on clusters originally created on Redpanda 26.3 or later. Clusters created on 26.2 or earlier keep the earlier behavior until you set it to true.

Follower fetching: fewer retries at the log tip

A consumer that fetches from a follower no longer receives an immediate OFFSET_NOT_AVAILABLE error when the follower hasn’t caught up to the leader’s high watermark. The fetch now waits for the follower to catch up, up to the consumer’s maximum fetch wait time. This removes tight retry loops at the end of the log.

Debug bundle permission check

You can now check that a debug bundle collects everything it needs before Redpanda support asks for one. rpk debug bundle --dry-run checks the file reads, commands, Admin API and Kafka access, and Network Time Protocol (NTP) reachability that a bundle needs. In Kubernetes, it also checks cluster DNS and Kubernetes role-based access control (RBAC) permissions. It reports any gaps without creating a bundle. The new Admin API endpoint POST /v1/debug/bundle/check_permissions runs the same check remotely.

Debug bundles started through the Admin API now also run rpk with the broker’s PATH, so steps that run commands such as df, du, and uname are no longer skipped.

Broker versions in upgrade status

The rpk cluster upgrade status command now lists the distinct logical versions that the brokers report, and prints a warning when brokers report more than two. The GetUpgradeStatus Admin API also reports the active and target releases, how many feature releases the upgrade crosses, and whether the upgrade needs an explicit finalize request.

Load Factor capacity metrics on more cloud providers

Redpanda now detects the cloud instance type on Google Cloud Platform (GCP) and Microsoft Azure, as well as on Amazon Web Services (AWS). Brokers on GCP and Azure instance types that Redpanda has capacity data for (types with local SSD storage) report instance capacity metrics, so the Load Factor dashboard shows their disk and network load factors.

Schema ID validation: header-based schema IDs and GUIDs

Server-side schema ID validation now reads the schema locator from the reserved __key_schema_id and __value_schema_id record headers before it checks the payload. A locator can be the integer schema ID or the schema globally unique identifier (GUID). As a result, producers that use Confluent’s header-based SerDes format pass validation without a payload prefix. Iceberg topics resolve schemas the same way. No new configuration is required.

Integer schema IDs now also resolve in the topic’s Schema Registry context, set by redpanda.schema.registry.context. Earlier versions always resolved integer schema IDs in the default context (.). If a topic with schema ID validation sets a context other than the default, check its schemas before you upgrade. See Context resolution.

Storage mode migration (beta)

You can now migrate an existing local, unset, or Tiered Storage v1 topic to Cloud Topics or Tiered Storage v2 in place, without recreating it. New rpk topic storage mode-migration commands migrate Tiered Storage v1 topics to Tiered Storage v2 in bulk and report migration progress. They also retry partitions whose migration stalls. See Migrate a Topic to Cloud Topics or Tiered Storage v2.

rpk shadow failover can now fail over a shadow link as a whole. Run without --all or --topic, the command permanently stops the link’s metadata sync tasks, freezes its configuration, fails over every active and paused shadow topic, and lifts Schema Registry write protection when the link reaches the FAILED_OVER state. Whole-link failover requires Redpanda 26.3 or later on every broker, and a finalized upgrade to 26.3.

The failover runs in two phases, each bounded by the new shadow_link_failover_timeout cluster property. If a phase misses its deadline, the link stops in the FAILOVER_FAULTED state, and running the command again resumes the failover from that phase. Use --wait to block until the failover completes, and track its progress with the new link states, rpk shadow status fields, and shadow link metrics. --all still fails over topics one by one and leaves the link and its sync tasks running.

You can also pause and resume individual sync tasks with the new rpk shadow task pause and rpk shadow task resume commands.

Protected mode

The new protected_mode cluster property keeps a development cluster started with default settings from being open to the network. While protected mode is on, a listener with no authentication or mTLS accepts clients only from the broker host, through a loopback address.

Protected mode is on by default only for brokers in development mode on clusters created with Redpanda 26.3 or later. Unless you set it, it stays off on production-mode brokers and in containers from the official Docker image. Clusters created with Redpanda 26.2 or earlier keep it off after an upgrade. See Protected Mode.

Schema Registry: stored schema text preserved on new clusters

The new schema_registry_canonicalize_on_consume cluster property controls which schema types Schema Registry canonicalizes when it loads the internal _schemas topic. On new clusters, the default keeps stored schema text unchanged, which keeps schema globally unique identifiers (GUIDs) stable. Clusters created on Redpanda 26.2 or earlier keep canonicalizing all types. Schema Registry still matches equivalent schemas on their canonical form, with or without normalization.

Shadowing: source cluster fencing

Source cluster fencing makes a cluster reject new writes and changes to user topics, consumer groups, Schema Registry, access control lists (ACLs), and roles. Use the new rpk cluster fencing commands to fence the source cluster of a shadow link before you fail over the link. Clients then can’t keep writing to the source cluster. A read policy lets the shadow link’s user keep fetching, so replication continues while the cluster is fenced.

Detect object storage deletions

Redpanda now stores a small canary object in the object storage bucket of each cluster that writes to object storage. If something outside Redpanda deletes the canary, for example a bucket lifecycle rule, Redpanda logs an error and sets the new redpanda_cloud_storage_canary_missing gauge to 1. Alert on this gauge to find unwanted deletions early. To disable the canary, set the new cloud_storage_canary_check_interval_sec cluster property to null.

Redpanda CLI (rpk)

New commands

New flags

Changed defaults

  • rpk container start: --console-image default changed from redpandadata/console:v3.9.0 to redpandadata/console:v3.10.0

Deprecated commands

Behavior changes

  • rpk topic delete -r no longer deletes internal topics, such as __consumer_offsets, when a regular expression matches them. To delete an internal topic, name it exactly, without -r. See Delete a topic.

  • rpk shadow failover without --all or --topic fails over the whole Shadow Link and permanently stops its metadata sync tasks. Before, the command required one of those flags. Whole-link failover requires Redpanda 26.3 or later on every broker, and a finalized upgrade to 26.3.

  • Development mode turns on protected_mode while that cluster property is unset, so listeners without authentication or mTLS accept only loopback clients. Clusters created with Redpanda 26.2 or earlier keep it off unless it is set explicitly. See Protected Mode.

  • rpk redpanda check and rpk redpanda start warn when the data directory is on XFS with rmapbt turned on and Redpanda uses 16 or more cores. To avoid the warning, format XFS with mkfs.xfs -m rmapbt=0. See System meets Redpanda requirements.

rpk plugins

connect plugin 4.118.0

New flags

  • rpk connect dry-run: Added --rpc-plugins

  • rpk connect lint: Added --secrets, --redpanda-license, --chroot, --chroot-passthrough, --disable-telemetry, --telemetry-deployment-type, --telemetry-tenant-id, --rpc-plugins

  • rpk connect list: Added --docs-format

  • rpk connect streams: Added --bind-http

  • rpk connect test: Added --secrets, --redpanda-license, --chroot, --chroot-passthrough, --disable-telemetry, --telemetry-deployment-type, --telemetry-tenant-id, --rpc-plugins

Removed flags

  • rpk connect streams: Removed --no-api

New configuration properties

Iceberg:

Security:

  • oidc_rsa_default_algorithm: Signing algorithm for RSA keys in the OIDC provider’s JWKS that omit the alg member

  • protected_mode: Restricts listeners without authentication or mTLS to loopback clients

Shadowing:

Object storage: