Rolling Upgrades
One of the primary uses for maintenance mode is to perform a rolling upgrade on each node in the cluster. This process involves putting a node into maintenance mode, upgrading the node, taking the node out of maintenance mode, and then repeating the process on the next node in the cluster. Placing nodes into maintenance mode ensures a smooth upgrade of your cluster while reducing the risk of interruption or degradation in service.
If you are using the Redpanda Kubernetes Operator to trigger an upgrade, then node maintenance mode is automatically incorporated into the upgrade process.
The upgrade process cannot be applied to downgrades between major versions, since earlier versions might not support some data formats in the current version. For example, you can't downgrade from version 22.2.1 to version 22.1.7. However, downgrades are supported between minor versions; for example, you can downgrade from version 22.2.2 to version 22.2.1.
Performing a rolling upgrade​
To perform a rolling upgrade on a node in the cluster:
Repeat this process for each node to upgrade.
1. Enable maintenance mode​
Check that all nodes are healthy:
rpk cluster health
If the cluster is healthy, the output shows
Healthy: true
.Select a node that has not been upgraded yet and place it into maintenance mode:
rpk cluster maintenance enable <node-id> --wait
The
--wait
option ensures that the cluster is healthy before putting the node into maintenance mode.Verify that the node is in maintenance mode:
rpk cluster maintenance status
Confirm that
FINISHED
istrue
.Validate again the health of the cluster:
rpk cluster health
If the cluster is healthy, the output shows
Healthy: true
.noteYou can also evaluate external metrics to determine cluster health. If the cluster has any issues, take the node out of maintenance mode by running the following command before proceeding with other operations, such as decommissioning or retrying the rolling upgrade:
rpk cluster maintenance disable <node-id>
2. Disable maintenance mode​
After you've successfully upgraded the node:
Take the node out of maintenance mode:
rpk cluster maintenance disable <node-id>
Ensure that the node is no longer in maintenance mode:
rpk cluster maintenance status
3. Restart node​
After you're out of maintenance mode, restart the node's Redpanda service.
You can stop the upgrade process and roll back to the original version as long as you have not upgraded every node and restarted the cluster.