Arbitrary configuration parameters for the Redpanda operator

Before continuing please go over the Kubernetes Quick Start Guide and follow the steps for setting up the Redpanda Operator and optionally a Redpanda cluster.

Once the Redpanda Operator is installed a Redpanda cluster can be created by applying a Cluster Custom Resource (CR) containing the specification of the desired Redpanda cluster. The CR includes the desired Redpanda Cluster name, namespace, and configuration for the Admin, Kafka, Pandaproxy API, and others.

The cluster definition includes properties that are commonly used to configure at both cluster and node levels. However, Redpanda offers a broad range of custom properties for node configuration besides cluster configuration.

For this reason, Redpanda introduced the additionalConfiguration property, which allows you to include arbitrary configuration properties not included in the CR. For example, consider the single-node sample found in our sample files with additionalConfiguration added:

Sample configuration

apiVersion: redpanda.vectorized.io/v1alpha1
kind: Cluster
metadata:
  name: one-node-cluster
spec:
  image: "vectorized/redpanda"
  version: "latest"
  replicas: 1
  resources:
    requests:
      cpu: 1
      memory: 1.2Gi
    limits:
      cpu: 1
      memory: 1.2Gi
  configuration:
    rpcServer:
      port: 33145
    kafkaApi:
    - port: 9092
    pandaproxyApi:
    - port: 8082
    adminApi:
    - port: 9644
    developerMode: true
  additionalConfiguration:
    pandaproxy_client.retries: "10"
    schema_registry.schema_registry_api: "[{'name':'external','address':'0.0.0.0','port':8081}]"

You may also set additional configuration properties at the cluster level, if you set the enable_idempotency property to true:

rpk cluster config set enable_idempotence true

To define the default_topic_partitions property with a new value:

rpk cluster config set default_topic_partitions 3

The additionalConfiguration map includes parameters that are not part of the Custom Resource, but are passed to the Redpanda configuration (redpanda.yaml).

The format of each entry is <subsystem>.<property_name>: <value>. Examples of subsystems, as seen above, are redpanda, pandaproxy_client, and schema_registry. The value can be a single value or a YAML/JSON-encoded string, as is the case with the schema_registry parameter in the example.

Once the above specification is applied, the Redpanda logs should include

pandaproxy_client:
 ..
 retries: 10
redpanda:
 ..
 default_topic_partitions: 3
 enable_idempotence: true
 ..
...
schema_registry:
 schema_registry_api:
   - address: 0.0.0.0
     name: external
     port: 8081

To verify the values for the cluster configuration:

rpk cluster config get enable_idempotence
true

or

rpk cluster config get default_topic_partitions
3

Limitations

When using this capability keep in mind some limitations:

• versioning is not supported for the additionalConfiguration parameters
• key names not supported by Redpanda will lead to failure on start
• updating this map requires a manual restart of the Redpanda pods