Skip to main content
Version: 22.3

Configure External Access through LoadBalancer Services

To expose your Redpanda cluster externally though load balancers, you must do the following:

  1. Create the LoadBalancer Services, one for each broker.
  2. Configure each Redpanda broker with an advertised address that points to its LoadBalancer Service.

Prerequisites

  • Install rpk on your local machine so that you can test connections to your Redpanda cluster from outside Kubernetes.

  • Understand Kubernetes networking for Redpanda.

  • Review the best practices.

  • Make sure that your Kubernetes cluster is accessible through your desired node port range. You may need to edit your inbound firewall rules.

Create the LoadBalancer Services

  1. Configure one LoadBalancer Service for each Redpanda broker in your cluster. This example manifest configures three LoadBalancer Services, one for each of the following Pods in the redpanda namespace: redpanda-0, redpanda-1, and redpanda-2. Replace the namespace and the Pod names for your deployment.

    Reveal LoadBalancer Service manifests
    redpanda-loadbalancers.yaml
    apiVersion: v1
    kind: Service
    metadata:
    name: lb-redpanda-0
    namespace: redpanda
    spec:
    type: LoadBalancer
    ports:
    - name: schemaregistry
    targetPort: 8081
    port: 8081
    - name: http
    targetPort: 8082
    port: 8082
    - name: kafka
    targetPort: 9092
    port: 9092
    - name: admin
    targetPort: 9644
    port: 9644
    selector:
    statefulset.kubernetes.io/pod-name: redpanda-0
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: lb-redpanda-1
    namespace: redpanda
    spec:
    type: LoadBalancer
    ports:
    - name: schemaregistry
    targetPort: 8081
    port: 8081
    - name: http
    targetPort: 8082
    port: 8082
    - name: kafka
    targetPort: 9092
    port: 9092
    - name: admin
    targetPort: 9644
    port: 9644
    selector:
    statefulset.kubernetes.io/pod-name: redpanda-1
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: lb-redpanda-2
    namespace: redpanda
    spec:
    type: LoadBalancer
    ports:
    - name: schemaregistry
    targetPort: 8081
    port: 8081
    - name: http
    targetPort: 8082
    port: 8082
    - name: kafka
    targetPort: 9092
    port: 9092
    - name: admin
    targetPort: 9644
    port: 9644
    selector:
    statefulset.kubernetes.io/pod-name: redpanda-2
    • metadata.namespace: Deploy the Service in the same namespace as your Redpanda brokers.
    • spec.selector: Target only one Redpanda broker to be backed by the LoadBalancer Service. Every Pod in a StatefulSet is automatically given the statefulset.kubernetes.io/pod-name label, which contains the name of the Pod. By setting the spec.selector.statefulset.kubernetes.io/pod-name field of the LoadBalancer Service to the name of a Pod, it makes the LoadBalancer Service target only the individual Redpanda broker and not the whole cluster.
  2. Apply your manifest file:

    kubectl apply -f redpanda-loadbalancers.yaml

    You now have three LoadBalancer Services. Each one is linked to a specific Pod that's running a Redpanda broker. Since each Pod should run on a dedicated worker node, each LoadBalancer Service forwards traffic from the worker node's node port to the Redpanda broker that's running on the worker node.

  3. Create a load balancer instance outside of your Kubernetes cluster to receive external traffic and forward it to the correct worker node:

    Your service provider should create the load balancer instances for you automatically.

Configure advertised addresses

You can configure brokers to advertise one of the following:

note

The LoadBalancer Services each open a random port from your node port range. To avoid conflicts with the node ports that are opened by the LoadBalancer Services, the following configurations disable the default NodePort Service by setting external.type to null.

Use the default DNS of the load balancer instances

If your load balancer instances have a default DNS name, you can configure each Redpanda broker to advertise the name of its corresonding load balancer instance.

  1. Find out on which worker nodes your Redpanda brokers are running.

    kubectl get pod -n redpanda  \
    -o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name -l \
    app.kubernetes.io/component=redpanda,job-name!=redpanda-post-install,job-name!=redpanda-post-upgrade
  2. Find the DNS names for each of your load balancers.

    If you're using managed Kubernetes,

  3. Add the DNS names to the external.addresses field in order. For example, the first DNS name in the list is assigned to redpanda-0, the second is assigned to redpanda-1, and so on.

    load-balancer-dns.yaml
    external:
    enabled: true
    type: null
    addresses:
    - <load-balancer-dns-0>
    - <load-balancer-dns-1>
    - <load-balancer-dns-2>
    helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
    --values load-balancer-dns.yaml
  4. Use the load balancers' DNS names to communicate with the Redpanda cluster from outside the Kubernetes cluster:

    rpk cluster info --brokers <load-balancer-dns-name>

Use the default Redpanda subdomains

You can use a DNS provider to create CNAME records that route traffic from the default Redpanda subdomains to the addresses of the corresponding load balancer instances. The default subdomains are redpanda-<ordinal-number>, where redpanda is the name of the chart and the <ordinal-number> placeholder is generated by the StatefulSet.

  1. Get a domain from a domain registrar.

  2. Specify your domain name in the external.domain configuration.

    Replace <custom-domain> with your domain.

    custom-domain.yaml
    external:
    domain: <custom-domain>
    enabled: true
    type: null
    helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
    --values custom-domain.yaml
  3. Find the DNS names for each of your load balancers.

  4. Update the CNAME record for your domain so that each hostname (subdomain) points to the correct load balancer's DNS name.

    HostnameLoad balancer DNS
    redpanda-0<load-balancer-dns-0>
    redpanda-1<load-balancer-dns-1>
    redpanda-2<load-balancer-dns-2>
  5. Wait for your DNS changes to be propagated.

  6. Use your domain to communicate with the Redpanda cluster from outside the Kubernetes cluster:

    rpk cluster info --brokers redpanda-0.<custom-domain>

Use custom subdomains

You can give each Redpanda broker a custom subdomain to advertise instead of the default subdomains. Then, you can use a DNS provider to create CNAME records that route traffic from the custom subdomains to the corresponding load balancer instances.

  1. Get a domain from a domain registrar.

  2. Choose an option to configure the subdomains:

    • To use fully custom subdomains, do the following.

      Replace <custom-domain> with your domain, and replace the placeholders in the external.addresses configuration with your own subdomains in the order that you want them to be applied to the Redpanda brokers.

      The subdomains are given to each Redpanda broker in order. For example, the Redpanda broker running inside the redpanda-0 Pod will advertise <subdomain-for-broker-0>.<custom-domain>.

      custom-subdomain.yaml
      external:
      enabled: true
      type: null
      domain: <custom-domain>
      addresses:
      - <subdomain-for-broker-0>
      - <subdomain-for-broker-1>
      - <subdomain-for-broker-2>
      helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
      --values custom-subdomain.yaml
    • Or, to use custom subdomains that are suffixed with the index of the StatefulSet replica, do the following.

      Replace <custom-domain> with your domain, and replace <custom-subdomain> with your subdomain.

      This configuration renames your Pods to <subdomain>-<ordinal-number>. Your Redpanda brokers will advertise the <subdomain>-<ordinal-number>.<custom-domain> address.

      custom-subdomain-ordinal.yaml
      fullnameOverride: <custom-subdomain>
      external:
      enabled: true
      type: null
      domain: <custom-domain>
      helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
      --values custom-subdomain-ordinal.yaml
  3. Find the DNS names for each of your load balancers.

  4. Update the CNAME record for your domain so that each hostname (subdomain) points to the correct load balancer's DNS name.

    HostnameLoad balancer DNS
    <subdomain-for-broker-0><load-balancer-dns-0>
    <subdomain-for-broker-1><load-balancer-dns-1>
    <subdomain-for-broker-2><load-balancer-dns-2>
  5. Wait for your DNS changes to be propagated.

  6. Use your domain to communicate with the Redpanda cluster from outside the Kubernetes cluster:

    rpk cluster info --brokers <subdomain-for-broker-0>.<custom-domain>

Next steps

Configure security for your listeners.

Configure external access to specific listeners.

What do you like about this doc?




Optional: Share your email address if we can contact you about your feedback.

Let us know what we do well: