Upgrade Redpanda
To benefit from Redpanda’s new features and enhancements, upgrade to the latest version. Redpanda Data recommends that you perform a rolling upgrade on production clusters, which requires all brokers to be placed into maintenance mode and restarted separately, one after the other.
|
Redpanda’s Linux package repositories moved from |
Redpanda Streaming version numbers follow the convention AB.C.D, where AB is the two digit year, C is the feature release, and D is the patch release. For example, v22.3.1 indicates the first patch release on the third feature release of the year 2022. Patch releases include bug fixes and minor improvements, with no change to user-facing behavior. New and enhanced features are documented with each feature release. You can find the changes in each release in the release notes.
Limitations
The following limitations ensure a smooth transition between versions and help maintain your cluster’s stability.
-
Broker upgrades:
-
New features are enabled only after upgrading all brokers in the cluster.
-
You can upgrade only one feature release at a time, for example from 26.2 to 26.3. Skipping feature releases is not supported. A broker on the new release doesn’t start if the cluster’s active version is more than one feature release older. If you defer upgrade finalization, finalize each upgrade before you start the next one.
-
If you have enterprise features enabled without a valid license, Redpanda fails to upgrade to new feature releases, starting from version 24.3.
-
Starting in 26.3, a cluster without a valid license that runs more than 50 cores in total also fails to upgrade to new feature releases. See Community Edition Core Limit.
-
-
Rollbacks: You can roll back to the original version until the upgrade finalizes. By default, the upgrade finalizes within about 10 seconds of the last broker restarting on the new version. Without deferral, you can roll back only while at least one broker still runs the original version. If you defer upgrade finalization, you can roll back until you finalize the upgrade, even after you upgrade all brokers. Deferred finalization is an Enterprise feature available when upgrading to 26.2 or later.
-
Downgrades: Downgrades are possible only between patch releases of the same feature release. For example, you can downgrade from 26.3.2 to 26.3.1. Downgrading to previous feature releases, such as 26.2.x, is not supported after the upgrade finalizes. See Rollbacks.
-
Tiered Storage: If you have Tiered Storage enabled and you’re upgrading to 23.2, object storage uploads are paused until all brokers are upgraded. If the cluster cannot upgrade, roll it back to the original version.
In a mixed-version state, the cluster could run out of disk space. If you need to force a mixed-version cluster to upload, transfer partition leadership to brokers running the original version. -
Remote Read Replicas: Upgrade the Remote Read Replica cluster first, ensuring it’s on the same version as the origin cluster or one feature release ahead of the origin cluster. When upgrading to Redpanda 23.2, metadata from object storage is not synchronized until all brokers in the cluster are upgraded. If you need to force a mixed-version cluster to sync read replicas, transfer partition leadership to brokers running the original version.
| By default, a feature-release upgrade finalizes automatically as soon as every broker runs the new version, and rollback is then no longer possible. Starting with upgrades to v26.2, you can defer finalization to soak test the new version while keeping a rollback path open. See Defer Upgrade Finalization. |
Prerequisites
-
A running Redpanda cluster.
-
jq for listing available versions.
-
An understanding of the impact of broker restarts on clients, node CPU, and any alerting systems you use.
-
Review incompatible changes in new versions.
Review incompatible changes
-
Breaking changes in Redpanda 26.3:
-
Schema ID validation resolves integer schema IDs and subjects in the topic’s Schema Registry context, set by
redpanda.schema.registry.context, instead of always in the default context. Some topics set both schema ID validation and a context other than the default, but have their schemas registered only in the default context. Redpanda rejects records on those topics as soon as an upgraded broker leads one of their partitions. Before upgrading, remove the property from those topics, or copy their schemas into the topic’s context with the same schema IDs and subjects. See Context resolution.
-
-
Breaking changes in Redpanda 26.2:
-
Schema Registry contexts are enabled by default:
schema_registry_enable_qualified_subjectsdefaults totrue, including on existing clusters after upgrade, unless you explicitly set the property tofalsebefore upgrading. Any existing subject whose name starts with:.(for example,:.staging:user-value) is reinterpreted after the upgrade as a context-qualified subject instead of a literal subject name. Audit your subjects before upgrading withrpk registry subject list | grep '^:\.'. See Schema Registry contexts upgrade considerations.
-
-
Breaking changes in Redpanda 26.1:
-
If FIPS mode is enabled, change any SASL/SCRAM user passwords shorter than 14 characters to at least 14 characters before upgrading. FIPS 140-3 enforces stricter HMAC key size requirements than FIPS 140-2. Because Redpanda stores passwords in encrypted form, it cannot check the length of existing passwords. Clients with passwords shorter than 14 characters will fail to authenticate after the upgrade. See Configure Redpanda for FIPS.
-
-
Breaking changes in Redpanda 25.3:
-
Redpanda v25.3 introduces breaking schema changes for Iceberg topics. If you are using Iceberg topics and want to retain the data in the corresponding Iceberg tables, review Schema Changes and Migration Guide for Iceberg Topics in Redpanda v25.3 before upgrading your cluster, and follow the required migration steps to avoid sending new records to a dead-letter queue table.
-
Schema Registry no longer allows specifying a schema ID and version when registering a schema in read-write mode. You must use import mode to register a schema with a specific ID and version. See Use the Schema Registry API for more information.
-
Starting in Redpanda v25.3, several topic properties support enhanced tristate behavior. Properties like
retention.ms,retention.bytes,segment.ms, and others now distinguish between zero values (immediate eligibility for cleanup/compaction) and negative values (disable the feature entirely). Previously, zero and negative values were treated the same way. For the complete list of affected properties and detailed information, see Redpanda v26.1 behavior changes. Review your topic configurations if you currently use zero values for these properties.
-
-
Breaking change in Redpanda 25.2: Ephemeral credentials for HTTP Proxy are removed. If your HTTP Proxy API listeners use
authentication_method:none, you must configure explicit SASL credentials (scram_username,scram_password, andsasl_mechanism) for HTTP Proxy to authenticate with the Kafka API.This allows any HTTP API user to access Kafka using shared credentials. Redpanda Data recommends enabling HTTP Proxy authentication instead.
For configuration instructions, see Configure HTTP Proxy to connect to Redpanda with SASL.
For details about this breaking change, see What’s new.
-
Redpanda Console v3.0.0 introduces breaking changes. If you are using Redpanda Console v2.x, review the migration guide to address breaking changes before upgrading Redpanda Console.
-
Starting in version 24.2, when managing configuration properties using the AlterConfigs API directly, Redpanda resets all unspecified values to the default values. This aligns more closely with the behavior in Apache Kafka. There is no change if you’re managing your configuration with tools like
rpk, Redpanda Console, Kubernetes, Helm, or Terraform.This does not pertain to the redpanda.remote.topic properties, such asredpanda.remote.delete. The remote properties are not reset to their defaults by the AlterConfigs API to avoid the possibility of unintentionally disabling Tiered Storage for a topic, which could cause significant operational issues for clusters designed to use Tiered Storage exclusively or with object storage as the only durable storage tier. -
Starting in version 24.2, client throughput limits are compatible with the AlterClientQuotas and DescribeClientQuotas Kafka APIs. Redpanda determines client throughput limits on a per-broker basis. In earlier versions, client throughput quotas were applied from cluster configurations on a per-shard basis.
-
Starting in version 24.2, transaction_max_timeout_ms defaults to 15 minutes, which limits the timeout a transactional producer can set during initialization. When performing upgrades to version 24.2, if you have clients with timeouts larger than
transaction_max_timeout_ms, then new producer initialization requests will fail due to this limit enforcement. Either reduce the client set transaction timeout to less than 15 minutes or increase the defaulttransaction_max_timeout_mson the service side. -
Patch releases in 22.3.14 and 23.1.2 changed the behavior when remote read is disabled and the requested Raft term falls below the local log’s beginning. In earlier versions, Redpanda returned an offset -1. With the patch, when you request a value older than the lowest offset, Redpanda returns the lowest offset, not -1.
Impact of broker restarts
When brokers restart, clients may experience higher latency, nodes may experience CPU spikes when the broker becomes available again, and you may receive alerts about under-replicated partitions. Topics that weren’t using replication (that is, topics that had replication.factor=1) will be unavailable.
Temporary increase in latency on clients (producers and consumers)
When you restart one or more brokers in a cluster, clients (consumers and producers) may experience higher latency due to partition leadership reassignment. Clients must communicate with the leader of a partition, so a client may send a request to a broker whose leadership has moved. The broker then returns a NOT_LEADER_OR_FOLLOWER error (formerly NOT_LEADER_FOR_PARTITION).
Redpanda shortens this interruption in two ways:
-
The error response names the new leader (KIP-951). A client that supports KIP-951 retries against the new leader without first requesting metadata.
-
Brokers briefly hold requests during the leadership change instead of failing them immediately. A broker that is becoming the leader holds produce, fetch, and OffsetForLeaderEpoch requests until it can serve them. A broker that is handing off leadership holds a produce request it refuses until it knows the new leader, so the error names that leader. While the former leader doesn’t yet know its successor, a fetch of that partition returns no data instead of an error. Fetch requests from consumers that set
client.rackare not held.
Each hold lasts at most the time set by the kafka_leader_hint_wait_ms cluster property (default: 100 milliseconds). Set it to 0 to turn off these behaviors, so that requests fail immediately and name the leader known at that time.
For example, when broker A hands off leadership of a partition to broker B:
-
Client sends a produce request to broker A.
-
Broker A holds the request until it knows that broker B is the new leader, then returns
NOT_LEADER_OR_FOLLOWERnaming broker B. -
Client sends the request to broker B, which serves it.
A client still requests metadata from the cluster to find the new leader when:
-
It doesn’t support KIP-951.
-
The error doesn’t name a leader that can serve the request.
-
It receives other retryable errors that indicate that its metadata may be stale.
Clients based on librdkafka, including the confluent-kafka-* clients, receive no leader in produce error responses, so they refresh metadata instead. Fetch error responses still name the leader for these clients.
To see how often brokers hold requests, and whether the leadership change finished before the hold expired, monitor the vectorized_kafka_leader_waits_total internal metric. Its request label is produce or fetch (fetch and OffsetForLeaderEpoch requests), and its outcome label is resolved or expired.
CPU spikes upon broker restart
When a restarted broker becomes available again, you may see your nodes' CPU usage increase temporarily. This temporary increase in CPU usage is due to the cluster rebalancing the partition replicas.
Under-replicated partitions
When a broker is in maintenance mode, Redpanda continues to replicate updates to that broker. When a broker is taken offline during a restart, partitions with replicas on the broker could become out of sync until it is brought back online. Once the broker is available again, data is copied to its under-replicated replicas until all affected partitions are in sync with the partition leader.
Find a new version
Before you upgrade, find out which Redpanda version you are currently running, whether you can upgrade straight to the new version, and what’s changed since your original version. To find your current version, run:
-
Linux
-
Docker
-
macOS
rpk cluster info -b --detailed
For all available flags, see the rpk cluster info command reference.
| Running Redpanda directly on Docker is not supported for production usage. This platform should only be used for testing. |
docker exec -it <container_name><container_tag> rpk version
Remember to replace the variables <container_name> and <container_tag>. The container tag determines which version of rpk to use. The release process bundles rpk and Redpanda into the same container tag with the same version.
brew list --versions | grep redpanda
Example output:
v26.3.1 (rev 5cc343d)
|
If your current version is more than one feature release behind the latest Redpanda version, you must first upgrade to an intermediate version. To list all available versions:
|
Check the release notes to find information about what has changed between Redpanda versions.
Check license status
Check the license status before every upgrade to a new feature release, even if you haven’t turned on any enterprise features. Redpanda refuses to upgrade to a new feature release while enterprise features are in use without a valid Enterprise Edition license key. Some enterprise features are on by default:
-
Clusters created on Redpanda 25.3 or later run Continuous Data Balancing (
partition_autobalancing_modeset tocontinuous) and Continuous Intra-Broker Partition Balancing (core_balancing_continuousset totrue) by default. -
Starting in 26.3, a cluster without a valid license that runs more than 50 cores in total counts as using an enterprise feature. See Check the cluster’s core count.
After the built-in 30-day trial expires, a cluster that was created on 25.3 or later and still runs its default configuration can’t upgrade to the next feature release until you install a license or turn off these features. See Disable Enterprise Features in Redpanda.
rpk cluster license info
This command displays the license status, whether the cluster is in violation, the enterprise features in use, and the license details. For example:
LICENSE INFORMATION
===================
License status: valid
License violation: false
Cluster cores: 12 / 50 (Community Edition limit)
Enterprise features in use: [partition_auto_balancing_continuous core_balancing_continuous]
Organization: redpanda
Type: enterprise
Expires: Oct 24 2027
If License violation is true, Redpanda refuses to upgrade to a new feature release until you install a valid license or turn off the enterprise features in use. The Cluster cores row appears only when the brokers run Redpanda 26.3 or later. For details about each field, see Check License Status and Feature Usage with rpk.
|
Redpanda blocks upgrades to new feature releases if enterprise features are active without a valid license. Ensure compliance by obtaining a license to maintain access to the latest features and updates. |
Check the cluster’s core count
This step is required when you upgrade a cluster without a valid license from 26.2 to 26.3. Starting in 26.3, a cluster without a license that runs more than 50 cores in total counts as using an enterprise feature, and the first 26.3 broker refuses to start. Redpanda 26.2 does not count cores against this limit, so rpk cluster license info cannot warn you before the upgrade.
|
-
Run the following command with
rpk26.2.1 or later:rpk cluster info -b --detailed -
Add up the values in the
COREScolumn for every broker in the list. -
If the total is more than 50 and the cluster has no valid license, install a license or bring the cluster within the limit before you upgrade. See Check the core count before you upgrade from 26.2.
If a broker refuses to start after you upgrade it, see A Redpanda Enterprise Edition license is required.
Perform a rolling upgrade
A rolling upgrade involves putting a broker into maintenance mode, upgrading the broker, taking the broker out of maintenance mode, and then repeating the process on the next broker in the cluster. Placing brokers into maintenance mode ensures a smooth upgrade of your cluster while reducing the risk of interruption or degradation in service.
When a broker is placed into maintenance mode, it reassigns its partition leadership to other brokers for all topics that have a replication factor greater than one. Reassigning partition leadership involves draining leadership from the broker and transferring that leadership to another broker.
-
Check for topics that have a replication factor greater than one.
If you have topics with
replication.factor=1, and if you have sufficient disk space, Redpanda Data recommends temporarily increasing the replication factor. This can help limit outages for these topics during the rolling upgrade. Do this before the upgrade to make sure there’s time for the data to replicate to other brokers. For more information, see Change topic replication factor. -
Ensure that all brokers are active before upgrading:
rpk cluster info -b --detailedAll brokers should show
activeforMEMBERSHIPandtrueforIS-ALIVE:Example output
ID HOST PORT RACK CORES MEMBERSHIP IS-ALIVE VERSION UUID 0 redpanda-0.testcluster.local 32180 A 8 active true 25.2.13 24c08934-94f6-478c-b57d-45239f452488 1 redpanda-1.testcluster.local 32180 B 8 active true 25.2.13 7f3c1c6e-2f4d-4c8a-9c6e-0a8a6d8b2b61 2 redpanda-2.testcluster.local 32180 C 8 active true 25.2.13 b2e8d4a9-91c1-4b55-9f6a-3f7a2d3c5e44
New features in a version are enabled after all brokers in the cluster are upgraded. If problems occur, the upgrade is not committed.
Enable maintenance mode
-
Check that all brokers are healthy:
rpk cluster healthExample output:
CLUSTER HEALTH OVERVIEW ======================= Healthy: true (1) Controller ID: 0 All nodes: [0 1 2] (2) Nodes down: [] (3) Leaderless partitions: [] (3) Under-replicated partitions: [1] (3)1 The cluster is either healthy ( true) or unhealthy (false).2 The node IDs of all brokers in the cluster. 3 If the cluster is unhealthy, these fields will contain data. -
Optional: You can use the Admin API (default port: 9644) to perform additional checks for potential risks with restarting a specific broker.
curl -X GET "http://<broker-address>:<admin-api-port>/v1/broker/pre_restart_probe" | jq .Example output:// Returns tuples of partitions (in the format {namespace}/{topic_name}/{partition_id}) affected by the broker restart. { "risks": { "rf1_offline": [ "kafka/topic_a/0" ], "full_acks_produce_unavailable": [], "unavailable": [], "acks1_data_loss": [] } }In this example, the restart probe indicates that there is an under-replicated partition
kafka/topic_a/0(with a replication factor of 1) at risk of going offline if the broker is restarted.See the Admin API reference for more details on the restart probe endpoint.
-
Select a broker that has not been upgraded yet and place it into maintenance mode:
rpk cluster maintenance enable <node-id> --waitThe
--waitoption tells the command to wait until a given broker, 0 in this example, finishes draining all partitions it originally served. After the partition draining completes, the command completes.Expected output:Successfully enabled maintenance mode for node 0 Waiting for node to drain...
-
Verify that the broker is in maintenance mode:
rpk cluster maintenance statusExpected output:
NODE-ID DRAINING FINISHED ERRORS PARTITIONS ELIGIBLE TRANSFERRING FAILED 0 true true false 3 0 2 0 1 false false false 0 0 0 0 2 false false false 0 0 0 0
The
Finishedcolumn should readtruefor the broker that you put into maintenance mode. -
Validate the health of the cluster again:
rpk cluster health --watch --exit-when-healthyThe combination of the
--watchand--exit-when-healthyflags tell rpk to monitor the cluster health and exit only when the cluster is back in a healthy state.You can also evaluate metrics to determine cluster health. If the cluster has any issues, take the broker out of maintenance mode by running the following command before retrying the rolling upgrade:
rpk cluster maintenance disable <node-id>
Upgrade your version
-
Linux
-
Docker
-
macOS
For Linux distributions, the process changes according to the distribution:
-
Fedora/RedHat
-
Debian/Ubuntu
First, check which versions are available and verify your current version:
# List available versions
yum list --showduplicates redpanda
# Check current version
rpk cluster info -b --detailed
Install the specific version you want to upgrade to. Replace <version> with the desired version:
sudo yum update redpanda-<version> # example: redpanda-25.1.7-1
First, check which versions are available and verify your current version:
# List available versions
apt list --all-versions redpanda
# Check current version
rpk cluster info -b --detailed
Install the specific version you want to upgrade to. Replace <version> with the desired version (for example, 25.1.7-1):
sudo apt update
sudo apt install redpanda=<version> # example: redpanda-25.1.7-1
| Running Redpanda directly on Docker is not supported for production usage. This platform should only be used for testing. |
To perform an upgrade you must replace the current image with a new one.
First, check which image is currently running in Docker:
docker ps
Stop and remove the containers:
docker stop <container_id>
docker rm <container_id>
Remove current images:
docker rmi <image_id>
Pull the desired Redpanda version, or adjust the setting to latest in the version tag:
docker pull docker.redpanda.com/redpandadata/redpanda:<version>
After it completes, restart the cluster:
docker restart <container_name>
For more information, see the Redpanda Quickstart.
If you previously installed Redpanda with brew, run:
brew upgrade redpanda-data/tap/redpanda
For installations from binary files, download the preferred version from the release list and then overwrite the current rpk file in the installed location.
Check metrics
Before continuing with the upgrade, check these important metrics to make sure the cluster is healthy and working as expected.
| Metric Name | Description | Recommendations |
|---|---|---|
Measures the number of under-replicated Kafka replicas. Non-zero: Replication lagging. Zero: All replicas replicated. |
Pause upgrades if non-zero. |
|
Represents the number of partitions that are currently unavailable. Value of zero indicates all partitions are available. Non-zero indicates the respective count of unavailable partitions. |
Ensure metric shows zero unavailable partitions before restart. |
|
Total bytes processed for Kafka requests. |
Ensure produce and consume rate for each broker recovers to its pre-upgrade value before restart. |
|
Latency for processing Kafka requests. Indicates the delay between a Kafka request being initiated and completed. |
Ensure the p99 histogram value recovers to its pre-upgrade level before restart. |
|
Latency for processing RPC requests. Shows the delay between an RPC request initiation and completion. |
Ensure the p99 histogram value recovers to its pre-upgrade level before restart. |
|
CPU utilization for a given second. The value is a decimal between 0.0 and 1.0. A value of 1.0 means that the CPU was busy for the entire second, operating at 100% capacity. A value of 0.5 implies the CPU was busy for half the time (or 500 milliseconds) in the given second. A value of 0.0 indicates that the CPU was idle and not busy during the entire second. |
If you’re seeing high values consistently, investigate the reasons. It could be due to high traffic or other system bottlenecks. |
Restart the broker
To ensure proper ownership and permissions, restart the broker’s Redpanda service using systemctl:
sudo systemctl restart redpanda
Disable maintenance mode
After you’ve successfully upgraded the broker:
-
Take the broker out of maintenance mode:
rpk cluster maintenance disable <node-id>Expected output:
Successfully disabled maintenance mode for node 0
-
Ensure that the broker is no longer in maintenance mode:
rpk cluster maintenance statusExpected output:
NODE-ID DRAINING FINISHED ERRORS PARTITIONS ELIGIBLE TRANSFERRING FAILED 0 false false false 0 0 0 0 1 false false false 0 0 0 0 2 false false false 0 0 0 0
Post-upgrade tasks
To verify that the cluster is running properly, run:
rpk cluster health
To view additional information about your brokers, run:
rpk cluster info -b --detailed
You can also use the Admin API to check how much each broker has progressed in recovering its workloads:
curl -X GET "http://<broker-address>:<admin-api-port>/v1/broker/post_restart_probe"
// Returns the load already reclaimed by broker, as a percentage of in-sync replicas
{
"load_reclaimed_pc": 66
}
Upgrade Redpanda Console
Before upgrading to Redpanda Console v3.0.x, review the migration guide to address breaking changes.
-
Linux
-
Docker
For Linux distributions, the process changes according to the distribution:
-
Fedora/RedHat
-
Debian/Ubuntu
In the terminal, run:
sudo yum update redpanda-console -y
In the terminal, run:
sudo apt-get update && sudo apt-get install --only-upgrade redpanda-console -y
To perform an upgrade you must replace the current image with a new one.
-
Check which image is currently running in Docker:
docker ps -
Stop and remove the current container:
docker stop <container-id> docker rm <container-id> -
Remove the existing Docker image:
docker rmi <image-id> -
Pull the desired Redpanda version. Replace
<version>with the version you want to upgrade to:docker pull docker.redpanda.com/redpandadata/console:<version> -
Restart the Docker container using the new image:
docker run -d --name <container-name> docker.redpanda.com/redpandadata/console:<version>
For more information, see the Redpanda Quickstart.
Troubleshooting
A Redpanda Enterprise Edition license is required
When you upgrade to a new feature release while enterprise features are in use without a valid license, the first broker that you start on the new version refuses to start. Starting in 26.3, a cluster without a license that runs more than 50 cores in total is also refused. See Community Edition Core Limit. The other brokers keep serving clients on the previous version, and the refused broker’s data is untouched.
The refused broker logs an error like the following and exits a few seconds after it starts. This example comes from a cluster that exceeds the core limit:
ERROR ... Failure during startup: A Redpanda Enterprise Edition license is required to use the currently enabled feature(s): (cluster_cores_over_limit). To apply your license, downgrade this broker to the pre-upgrade version and provide a valid license key via rpk using 'rpk cluster license set <key>', or via Redpanda Console. To request an Enterprise license, please visit https://redpanda.com/upgrade. To try Redpanda Enterprise for 30 days, visit https://redpanda.com/try-enterprise. ...
This cluster has 51 cores across 3 brokers (0, 1, 2), more than the 50-core limit of Redpanda Community Edition. Reduce the cluster to 50 cores or fewer, or install an Enterprise license. To resolve the core limit: install a license on the running cluster with 'rpk cluster license set <key>'; or, if this broker was just added, decommission it with 'rpk cluster brokers decommission <node-id> --skip-liveness-check'; or restart it with fewer shards (--smp) so the cluster total is 50 cores or fewer. Any other feature listed in this message still needs a license. Brokers already running the new version keep running.
The features in parentheses are the ones in violation. A license clears all of them. For more ways to confirm the cause, see Confirm the cause.
To recover, choose one of these options, in order of preference:
-
Install a license. Apply a valid Redpanda Enterprise Edition license through any broker that is still running. The license takes effect on the running cluster immediately. If you don’t have a license key,
rpk generate license --applyapplies a 30-day trial license. Then start the refused broker again on the same new version, and the rolling upgrade continues. -
Roll back the refused broker. Put the previous version back on that broker and start it. The cluster returns to normal operation on the previous version with the same data.
-
Reinstall the previous version on the refused broker:
-
Debian/Ubuntu
-
Fedora/RedHat
sudo apt install redpanda=<previous-version> --allow-downgradessudo yum downgrade redpanda-<previous-version> -
-
Upgrade again after you install a license or bring the cluster within the limit.
-
-
Bring the cluster into compliance without a license. Do one of the following through any broker that is still running:
-
Disable the enterprise features that the error lists. Clusters created on Redpanda 25.3 or later run two of them by default:
partition_auto_balancing_continuous(setpartition_autobalancing_modetonode_add) andcore_balancing_continuous(setcore_balancing_continuoustofalse). -
If the error lists only
cluster_cores_over_limit, reduce the cluster to 50 cores or fewer. If the refused broker was just added, decommission it withrpk cluster brokers decommission <node-id> --skip-liveness-check. Otherwise, restart a broker with a lower--smpvalue. See Bring a cluster within the limit.Then start the refused broker again on the same new version, and the rolling upgrade continues.
-
To start the refused broker, on either version:
-
If the broker logs
Crash loop detected, delete thestartup_logfile from its data directory, or wait an hour since the last crash:sudo rm /var/lib/redpanda/data/startup_log -
Start the broker:
sudo systemctl start redpanda
If no broker is running, for example in a single-broker cluster or when you stopped every broker before the upgrade, you can’t install a license through rpk. Put the previous version back on a broker and start it, install the license, then upgrade again.
Confirm the cause
The refused broker exits with code 1. From a broker that is still running, rpk cluster upgrade status shows the refused broker as not alive (false in the ALIVE column) with no release version (- in the RELEASE-VERSION column). The cluster’s active version stays at the previous release’s logical version, for example 19 for 26.2. For details about the command’s output, see Defer Upgrade Finalization.
Earlier in its startup, the broker logs the values that the check used, which can help you confirm the cause:
Verifying enterprise license: active_version=<version>, latest_version=<version>, enterprise_features=[<features>], cluster_cores=<cores>/50, license_missing_or_expired=true
If something restarts the refused broker automatically, it can reach the crash_loop_limit, which by default allows five consecutive crashes within one hour. After that, the broker logs Crash loop detected and doesn’t start, even after you fix the license problem, until you delete the startup_log file from its data directory or an hour passes since the last crash.
On Linux, this happens when the Redpanda systemd unit uses Restart=always or Restart=on-failure. The stock unit uses Restart=on-abnormal, so a refused broker stays stopped instead.