Configure Listeners in Kubernetes

You can fine-tune client interactions with Redpanda by customizing internal and external listener configurations.

Anatomy of a listener

Listeners are gateways for client communications in Redpanda. Redpanda uses TCP sockets for client connections, where each TCP socket consists of:

  • IP address or DNS name: The unique network identifier of the Redpanda server. This can be a local address or a public address for cloud-based services.

  • Port: A specific entry point for network traffic. Redpanda listens on designated ports for client connections.

Clients must specify both the IP address or DNS name and port for a successful connection, such as 203.0.113.5:9093.

The Redpanda Helm chart defines two listeners by default:

Listener Description

default

Internal connections. These endpoints use internal addresses assigned to brokers by Kubernetes through the headless ClusterIP Service.

external

External connections. These endpoints are accessible through addresses configured in the external.domain and/or external.addresses settings in the Helm values.

Prefer external.domain for clusters that you expect to scale. Because external.addresses is a static list, extending it to add a broker forces a rolling restart of every existing broker. See Scale without restarting brokers.

For example, these are the default settings for the internal Kafka API listener:

listeners:
  kafka:
    port: 9093
    authenticationMethod:
    tls:
      enabled: true
      cert: default
      requireClientAuth: false
  • listeners.kafka.port: The container port for internal client connections.

  • listeners.kafka.authenticationMethod: The authentication method to use when the top-level auth.sasl.enabled setting is true. Default is sasl.

  • listeners.kafka.tls.enabled: Whether TLS is enabled for this listener when the top-level tls.enabled setting is true for all listeners.

  • listeners.kafka.tls.cert: The TLS certificates that this listener should use to secure client connections. TLS certificates are defined in tls.certs.

  • listeners.kafka.tls.requireClientAuth: Whether mTLS is enabled.

These are the settings for the external Kafka API listener:

listeners:
  kafka:
    external:
      default:
        enabled: true
        port: 9094
        # -- If undefined, `listeners.kafka.external.default.port` is used.
        advertisedPorts:
        - 31092
        tls:
          enabled: true
          cert: external
        # default is "sasl"
        authenticationMethod:
  • listeners.kafka.external.default.port: The container port for external client connections.

  • listeners.kafka.external.default.advertisedPorts: The node port that is routed to listeners.kafka.external.default.port. This port is opened in Kubernetes Services when external.service.enabled is true.

  • listeners.kafka.external.default.authenticationMethod: The authentication method to use when the top-level auth.sasl.enabled setting is true. Default is sasl.

  • listeners.kafka.external.default.tls.enabled: Whether TLS is enabled for this listener when the top-level tls.enabled setting is true for all listeners.

  • listeners.kafka.external.default.tls.cert: The TLS certificates that this listener should use to secure client connections. TLS certificates are defined in tls.certs.

Schema Registry and HTTP Proxy connect to Redpanda over the Kafka API. If you configure a TLS listener for the Kafka API, you must configure listeners.schemaRegistry.tls and listeners.http.tls. All APIs, except the internal RPC port, support multiple listeners.

Choose a listener configuration

Listener settings fall into four groups: how clients reach the brokers, what address and port the brokers advertise, how each listener is secured, and which listeners are enabled. Find your goal in the tables below.

All settings are Helm values. With the Redpanda Operator, set the same keys under spec.clusterSpec in the Redpanda resource.

Exposure: how clients reach the brokers

Goal Settings Where

Expose brokers through a node port on each worker node (default)

external.type: NodePort

NodePort Service

Give each broker its own cloud load balancer

external.type: LoadBalancer

LoadBalancer Service

Route all brokers through one Gateway address by TLS SNI

external.gateway, type: tlsroute on each listener

Gateway API

Bring your own Services or an external proxy

external.service.enabled: false

Custom Services

Addresses: what brokers advertise

Goal Settings Where

Advertise your own domain, with hostnames derived from Pod names

external.domain

Advertise custom domains

Advertise an explicit hostname or IP address per broker

external.addresses

Advertise custom domains, Advertise IP addresses

Add brokers without restarting the existing ones

external.prefixTemplate with external.addresses: [$PREFIX_TEMPLATE], or a reserved external.addresses list

Scale without restarting brokers

Advertise a different hostname on each external listener, such as one listener per network or proxy

listeners.<api>.external.<name>.prefixTemplate with external.addresses: [$PREFIX_TEMPLATE]

Advertise a different address for each external listener

Change the port that clients connect to

listeners.<api>.external.<name>.advertisedPorts

Customize the advertised ports

Security: TLS and authentication per listener

Goal Settings Where

Use a different TLS certificate on some listeners

listeners.<api>.tls.cert, listeners.<api>.external.<name>.tls.cert

Configure TLS certificates

Require client certificates (mTLS)

listeners.<api>.external.<name>.tls.requireClientAuth

Configure TLS certificates

Use a different authentication method on an external listener

listeners.<api>.external.<name>.authenticationMethod

Authentication

Listeners: which endpoints exist

Goal Settings Where

Offer more than one external listener, each with its own port, certificate, and authentication

Additional entries under listeners.<api>.external.<name>

Advertise a different address for each external listener for a worked example

Turn external access off for all listeners

external.enabled: false

Disable external access

Turn external access off for one listener

listeners.<api>.external.<name>.enabled: false

Disable external access

Scale without restarting brokers

Because the value of external.addresses is part of the checksum that determines whether Pods must restart, editing the list to add a broker rolls every existing broker. If you can’t use external.domain alone, choose one of the following patterns to keep the list unchanged when you scale.

If your addresses differ only by something each broker can work out for itself, such as its position in the cluster, set external.prefixTemplate to an expression and give external.addresses the single entry $PREFIX_TEMPLATE. Each broker evaluates the expression when it starts, so one entry serves every broker and the list never changes when you scale.

external.prefixTemplate is a shell expression, not a Helm template. It’s evaluated in the broker’s init container, where the following variables are available:

Variable Description

$POD_ORDINAL

The broker’s StatefulSet ordinal, such as 0.

$SERVICE_NAME

The broker’s Pod name, such as redpanda-0.

$KUBERNETES_NODE_NAME

The name of the worker node running the broker.

$HOST_IP_ADDRESS

The IP address of the worker node running the broker.

Command substitution works too, so you can derive a prefix rather than name it.

Replace <custom-hostname> with your hostname and <custom-domain> with your domain.

  • Operator

  • Helm

redpanda-cluster.yaml
apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef: {}
  clusterSpec:
    external:
      enabled: true
      type: NodePort
      domain: <custom-domain>
      prefixTemplate: <custom-hostname>-$POD_ORDINAL
      addresses:
        - $PREFIX_TEMPLATE
kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
prefix-template.yaml
external:
  enabled: true
  type: NodePort
  domain: <custom-domain>
  prefixTemplate: <custom-hostname>-$POD_ORDINAL
  addresses:
    - $PREFIX_TEMPLATE
helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
  --values prefix-template.yaml --reuse-values --wait

Each broker advertises <custom-hostname>-<pod-ordinal>.<custom-domain>. Setting external.domain also adds <custom-domain> and *.<custom-domain> to the SAN list of the chart’s TLS certificates, so this pattern works with TLS enabled.

Unlike setting fullnameOverride, this pattern doesn’t rename your Pods or Services. Use it when the hostname you advertise must differ from the name of the underlying Kubernetes resources.

The prefix is evaluated when a broker starts, not when the chart is rendered, so $PREFIX_TEMPLATE remains a literal string everywhere the chart reports addresses before startup:

  • The generated <cluster-name>-rpk ConfigMap and the NOTES output after helm upgrade show the unsubstituted string. Build client profiles from your own DNS names instead.

  • Don’t combine this pattern with external.externalDns.enabled. ExternalDNS receives the literal string as the hostname annotation and can’t create usable records. Create your DNS records yourself, such as a wildcard record for *.<custom-domain>.

external.prefixTemplate is also not part of the restart checksum. Editing it doesn’t restart your brokers, so existing brokers keep advertising their old addresses until they restart for another reason. To apply a new prefix immediately, perform a rolling restart.

Reserve addresses for brokers you haven’t added yet

If your addresses don’t follow a pattern, list an address for every broker you expect to run, not just the brokers running now. Entries beyond the current replica count are ignored, so scaling up to any broker you’ve already listed adds only the new broker and leaves the rest untouched.

statefulset:
  replicas: 3
external:
  enabled: true
  type: NodePort
  addresses:
    - <hostname-for-broker-0>
    - <hostname-for-broker-1>
    - <hostname-for-broker-2>
    # Reserved for future brokers
    - <hostname-for-broker-3>
    - <hostname-for-broker-4>
    - <hostname-for-broker-5>

Create the DNS records for the reserved addresses when you scale, not in advance.

Never set statefulset.replicas higher than the number of entries in external.addresses. The chart fails to render with an index out of range error.

Both patterns require you to know each broker’s address before the broker exists. If your addresses are assigned dynamically, such as the IP addresses of LoadBalancer Services that you don’t reserve in advance, reserve them up front if your platform supports it. Otherwise, scaling the cluster restarts every broker.

Advertise a different address for each external listener

A single external listener advertises the same hostname to every client. Some environments need more than one external Kafka API listener, each advertising its own hostnames and secured with its own certificate. For example:

  • A production listener and a management listener on separate networks, each behind its own SNI proxy or TCP ingress, each terminating TLS with a certificate from a different authority.

  • Two client populations that must not be able to reach each other’s endpoint, enforced by a firewall or proxy in front of each listener.

The Redpanda Helm chart and the Redpanda Operator support multiple external listeners under listeners.kafka.external.<listener-name>. Each listener has its own container port, advertised ports, TLS certificate, and authentication method. Advertised hostnames, however, come from the global external.addresses list, which is shared by all listeners.

To give each listener its own hostnames, combine the global $PREFIX_TEMPLATE placeholder with a prefixTemplate on each listener:

  1. Set external.addresses to the single entry $PREFIX_TEMPLATE.

  2. Set prefixTemplate on every external listener to the hostname that listener should advertise. Use $POD_ORDINAL so that each broker advertises its own hostname. The same variables as external.prefixTemplate are available.

  3. Set external.prefixTemplate as the fallback for external listeners that you don’t set individually, such as the external HTTP Proxy listener that the chart enables by default. A listener with no prefixTemplate and no fallback advertises an empty address.

  4. Leave external.domain unset. If you set it, the chart appends it to every advertised address.

Each broker evaluates the templates when it starts, so adding a broker doesn’t change the configuration of the existing brokers and doesn’t restart them.

This example advertises redpanda-broker-<pod-ordinal>.prod.example.com:443 on the default listener and redpanda-broker-<pod-ordinal>.mngmt.example.com:443 on the mngmt listener. The brokers listen on ports 9094 and 9095, and the proxies in front of them accept connections on port 443. Because an external proxy routes the traffic, the example disables the chart’s NodePort Service. The external HTTP Proxy listener uses the fallback external.prefixTemplate, so it advertises the production hostnames. To route it through a different proxy, give listeners.http.external.default its own prefixTemplate.

  • Operator

  • Helm

redpanda-cluster.yaml
apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef: {}
  clusterSpec:
    external:
      enabled: true
      service:
        enabled: false
      prefixTemplate: "redpanda-broker-$POD_ORDINAL.prod.example.com"
      addresses:
        - "$PREFIX_TEMPLATE"
    listeners:
      kafka:
        external:
          default:
            port: 9094
            advertisedPorts:
              - 443
            prefixTemplate: "redpanda-broker-$POD_ORDINAL.prod.example.com"
            tls:
              enabled: true
              cert: external-prod
              requireClientAuth: true
          mngmt:
            port: 9095
            advertisedPorts:
              - 443
            prefixTemplate: "redpanda-broker-$POD_ORDINAL.mngmt.example.com"
            tls:
              enabled: true
              cert: external-mngmt
              requireClientAuth: true
    tls:
      certs:
        external-prod:
          caEnabled: true
          secretRef:
            name: redpanda-external-prod-cert
        external-mngmt:
          caEnabled: true
          secretRef:
            name: redpanda-external-mngmt-cert
kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
per-listener-addresses.yaml
external:
  enabled: true
  service:
    enabled: false
  prefixTemplate: "redpanda-broker-$POD_ORDINAL.prod.example.com"
  addresses:
    - "$PREFIX_TEMPLATE"
listeners:
  kafka:
    external:
      default:
        port: 9094
        advertisedPorts:
          - 443
        prefixTemplate: "redpanda-broker-$POD_ORDINAL.prod.example.com"
        tls:
          enabled: true
          cert: external-prod
          requireClientAuth: true
      mngmt:
        port: 9095
        advertisedPorts:
          - 443
        prefixTemplate: "redpanda-broker-$POD_ORDINAL.mngmt.example.com"
        tls:
          enabled: true
          cert: external-mngmt
          requireClientAuth: true
tls:
  certs:
    external-prod:
      caEnabled: true
      secretRef:
        name: redpanda-external-prod-cert
    external-mngmt:
      caEnabled: true
      secretRef:
        name: redpanda-external-mngmt-cert
helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
  --values per-listener-addresses.yaml --reuse-values --wait

Broker redpanda-2 then advertises the following, which clients see in metadata responses:

advertised_kafka_api:
  - name: internal
    address: redpanda-2.redpanda.<namespace>.svc.cluster.local.
    port: 9093
  - name: default
    address: redpanda-broker-2.prod.example.com
    port: 443
  - name: mngmt
    address: redpanda-broker-2.mngmt.example.com
    port: 443
  • Because you supply these certificates through secretRef, the chart doesn’t generate them. Include each listener’s advertised hostnames in the certificate’s SAN list. The chart never adds prefixTemplate hostnames to the certificates that it issues. See Configure TLS certificates.

  • With caEnabled: true, each Secret must contain a ca.crt file, and brokers trust only client certificates signed by that CA. Without caEnabled, brokers trust the public CAs in the container’s system trust store, so requireClientAuth rejects client certificates from a private CA.

  • The limitations of $PREFIX_TEMPLATE described in Advertise a pattern instead of a list apply here too: the rpk ConfigMap and Helm notes show the literal string, and the pattern doesn’t work with external.externalDns.enabled.

Customize the advertised ports

Each external listener has two ports: port, the container port the broker listens on, and advertisedPorts, the port that brokers tell clients to connect to. What advertisedPorts has to match depends on how you expose the cluster:

  • NodePort Service: the node port that the chart opens for the listener. This is the default, so advertisedPorts are often called node ports.

  • LoadBalancer Service: the port the load balancer listens on.

  • Custom Services or an external proxy: the port that your Service or proxy accepts connections on, such as 443 for an SNI proxy. See Use a Custom Service for External Access.

  • Gateway API: not used. Brokers advertise external.gateway.advertisedPort instead. See Configure External Access through Gateway API.

advertisedPorts takes a list. With one entry, every broker advertises that port. With one entry per broker, broker N advertises entry N, which lets each broker use a different port behind a single address. The chart’s NodePort Service opens only the first entry, so a per-broker list requires custom Services or an external proxy.

To customize the advertised port for a listener, replace <port> with the port that you want to use.

Redpanda doesn’t validate the configured port numbers. Make sure to verify the following:

  • Your configured port numbers are within the range that is assigned for node ports in your Kubernetes cluster.

  • Your Kubernetes cluster is accessible through your desired node port range. You may need to edit your inbound firewall rules.

  • Your configured port numbers are not in use by any other service.

For the default ports, see the Redpanda Helm Chart Specification

For example, to customize the advertised port for the external Kafka API listener:

custom-kafka-port.yaml
listeners:
  kafka:
    external:
      default:
        advertisedPorts:
        - <port>

Configure TLS certificates

A Redpanda cluster provides granular control over the TLS certificates used by different listeners. This level of flexibility enables you to ensure the required level of security for each listener. For example, you can use a self-signed certificate for the internal RPC listener, while using a public certificate authority (CA) certificate for other listeners such as the Kafka API.

All listeners must use the same certificate if mTLS is enabled. Redpanda on Kubernetes does not support configuring different TLS certificates for mTLS-enabled listeners, or mixed configurations where some listeners use mTLS and others do not.
  • Internal listeners: By default, all internal listeners use the self-signed certificate defined globally in the tls.certs.default configuration. To customize the certificates for each internal listener, you can edit the listeners.<listener-name>.tls.cert setting.

  • External listeners: By default, all external listeners use the self-signed certificate defined globally in the tls.certs.external configuration. To customize the certificates for each external listener, you can edit the listeners.<listener-name>.external.default.tls.cert setting.

Here’s an example that configures two certificates:

The external Admin API listener is configured with the public-ca-cert certificate. The internal Kafka API listener is configured with the private-ca-cert certificate, and the other listeners are configured with the default self-signed certificate.

multiple-certs-tls.yaml
tls:
  enabled: true
  certs:
    public-ca-cert:
      issuerRef:
        name: <issuer-name>
        kind: Issuer
      caEnabled: false
    private-ca-cert:
      secretRef:
        name: <secret-name>
      caEnabled: true
    default:
      caEnabled: true
listeners:
  admin:
    external:
      default:
        tls:
          cert: public-ca-cert
  kafka:
    tls:
      cert: private-ca-cert
  http:
    tls:
      cert: default
  rpc:
    tls:
      cert: default
  schemaRegistry:
    tls:
      cert: default

Disable external access

You can disable external access for all listeners or for individual listeners.

To disable external access for all listeners:

disable-external-access.yaml
external:
  enabled: false

To disable external access for a specific listener, set listeners.<listener-name>.external.default to false. For example, to disable external access to the Kafka API listener:

disable-external-kafka-api.yaml
listeners:
  kafka:
    external:
      default:
        enabled: false