Configure Cluster Properties

When you install Redpanda software, cluster configuration properties are automatically set to the default values. Examples of cluster properties include Kafka front-end settings, authentication settings, and settings for enabling features, like Tiered Storage and rack awareness.

Redpanda includes cluster properties, broker properties, and topic properties. Some properties can be configured at either the cluster level or the topic level, such as retention_bytes (cluster-level) and retention.bytes (topic-level). Topic-level properties override cluster-level properties. Most Redpanda configuration properties are cluster properties.

Cluster properties Broker properties

Cluster properties are stored internally and automatically replicated across all nodes, which ensures that each broker is in sync.

Broker properties are stored in the redpanda.yaml file located in the /etc/redpanda directory for each broker.

See also:

Edit cluster properties

For Kubernetes deployments, see Configure Cluster Properties in Kubernetes.

To change any property settings, edit the configuration from the command line using your default text editor. As you make changes, the Redpanda Admin API verifies that the new value is valid. For example, if you change fetch_max_bytes from the default of 57671680 to 5o (using the letter “o” by mistake), the system displays the following message:

PROPERTY         PRIOR     NEW
fetch_max_bytes  57671680  5o

Validation errors:
 * fetch_max_bytes: expected type integer

No changes were made.
bash

After you save your configuration changes, the new values are automatically applied and a new version number is generated. Any subsequent edits start with the most recent version of the configuration.

Some properties require that you restart the cluster for a change to take effect. The default configuration includes this information in the descriptions for these properties.

To assign new values to cluster properties:

  1. Open a terminal window and log in to a broker on your cluster.

  2. Run rpk cluster config edit. To edit tuning properties, run rpk cluster config edit --all instead.

  3. Edit the configuration file and set values for the properties you want to change.

  4. Save the file and quit the editor.

  5. Run rpk cluster config status to see whether the cluster requires a restart. If necessary, you see a message like this:

    $ rpk cluster config status
NODE  CONFIG-VERSION  NEEDS-RESTART  INVALID  UNKNOWN
1     4               true           []       []
    bash

  6. If necessary, restart the cluster.

When you finish your edits, the system updates the configuration and displays a message that lists which property settings were changed, along with their prior and new values. The message also includes the new version number of the configuration. For example:

PROPERTY             PRIOR  NEW
tx_timeout_delay_ms  1000   2000

Successfully updated configuration. New configuration version is 2.
bash
You can also change property values using the rpk cluster config set command, but this method does not display the current setting or the description.

View current value of a property

To see the current value of a property, run rpk cluster config get <property_name>. For example:

$ rpk cluster config get log_compression_type
producer
bash

Copy configurations to other clusters

Use the export option to save the current cluster configuration to a file. You can then copy this file to other clusters, so they can use the same configuration.

  1. Export the current configuration settings to a YAML file by running rpk cluster config export --filename <filename>.yaml. To store the configuration file outside your current working directory, use the full pathname for --filename; otherwise, supply the filename to store the file in your current working directory.

  2. Copy <filename>.yaml to the other cluster.

  3. Log in to the other cluster, and import the file with the saved configuration by running rpk cluster config import --filename <filename>.yaml. This command applies the property settings in <filename>.yaml to all nodes in the cluster.

Redpanda does not support importing cluster-specific identification (such as cluster_id) with this command.

Remove cluster properties from redpanda.yaml

If you have a redpanda.yaml file that contains both cluster properties and broker properties, the cluster properties are ignored. To remove cluster properties from the redpanda.yaml file, run:

rpk cluster config lint
bash

This avoids the issue of referring to a previous version or custom configuration.

Suggested reading

Suggested labs

Search all labs