Configure External Access through LoadBalancer Services

To expose your Redpanda cluster externally through load balancers, you must do the following:

1. Create the LoadBalancer Services, one for each broker.
2. Configure each Redpanda broker with an advertised address that points to its LoadBalancer Service.

Prerequisites

• Install rpk on your local machine so that you can test connections to your Redpanda cluster from outside Kubernetes.

• Understand Kubernetes networking for Redpanda.

• Review the best practices.

• Make sure that your Kubernetes cluster is accessible through your desired node port range. You may need to edit your inbound firewall rules.

Create the LoadBalancer Services

1. Configure one LoadBalancer Service for each Redpanda broker in your cluster. This example manifest configures three LoadBalancer Services, one for each of the following Pods in the redpanda namespace: redpanda-0, redpanda-1, and redpanda-2. Replace the namespace and the Pod names for your deployment.

   Reveal LoadBalancer Service manifests

   redpanda-loadbalancers.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: lb-redpanda-0
     namespace: redpanda
   spec:
     type: LoadBalancer
     ports:
       - name: schemaregistry
         targetPort: 8081
         port: 8081
       - name: http
         targetPort: 8082
         port: 8082
       - name: kafka
         targetPort: 9092
         port: 9092
       - name: admin
         targetPort: 9644
         port: 9644
     selector:
       statefulset.kubernetes.io/pod-name: redpanda-0
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: lb-redpanda-1
     namespace: redpanda
   spec:
     type: LoadBalancer
     ports:
       - name: schemaregistry
         targetPort: 8081
         port: 8081
       - name: http
         targetPort: 8082
         port: 8082
       - name: kafka
         targetPort: 9092
         port: 9092
       - name: admin
         targetPort: 9644
         port: 9644
     selector:
       statefulset.kubernetes.io/pod-name: redpanda-1
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: lb-redpanda-2
     namespace: redpanda
   spec:
     type: LoadBalancer
     ports:
       - name: schemaregistry
         targetPort: 8081
         port: 8081
       - name: http
         targetPort: 8082
         port: 8082
       - name: kafka
         targetPort: 9092
         port: 9092
       - name: admin
         targetPort: 9644
         port: 9644
     selector:
       statefulset.kubernetes.io/pod-name: redpanda-2
   • metadata.namespace: Deploy the Service in the same namespace as your Redpanda brokers.
   • spec.selector: Target only one Redpanda broker to be backed by the LoadBalancer Service. Every Pod in a StatefulSet is automatically given the statefulset.kubernetes.io/pod-name label, which contains the name of the Pod. By setting the spec.selector.statefulset.kubernetes.io/pod-name field of the LoadBalancer Service to the name of a Pod, it makes the LoadBalancer Service target only the individual Redpanda broker and not the whole cluster.

2. Apply your manifest file:

   kubectl apply -f redpanda-loadbalancers.yaml

   You now have three LoadBalancer Services. Each one is linked to a specific Pod that's running a Redpanda broker. Since each Pod should run on a dedicated worker node, each LoadBalancer Service forwards traffic from the worker node's node port to the Redpanda broker that's running on the worker node.

3. Create a load balancer instance outside of your Kubernetes cluster to receive external traffic and forward it to the correct worker node:

   Managed

   Your service provider should create the load balancer instances for you automatically.

   Bare-Metal

   Create a load balancer instance outside of your Kubernetes cluster for each worker node and forward the traffic to the node ports that are opened by the LoadBalancer Services. You can find these node ports using this command:

   kubectl get service -n redpanda

Configure advertised addresses

You can configure brokers to advertise one of the following:

• The default DNS of your load balancer instance
• The default Redpanda subdomains
• Custom subdomains

note

The LoadBalancer Services each open a random port from your node port range. To avoid conflicts with the node ports that are opened by the LoadBalancer Services, the following configurations disable the default NodePort Service by setting external.type to null.

Use the default DNS of the load balancer instances

If your load balancer instances have a default DNS name, you can configure each Redpanda broker to advertise the name of its corresonding load balancer instance.

1. Find out on which worker nodes your Redpanda brokers are running.

   kubectl get pod -n redpanda  \
   -o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name -l \
   app.kubernetes.io/component=redpanda,job-name!=redpanda-redpanda-configuration,job-name!=redpanda-post-upgrade

2. Find the DNS names for each of your load balancers.

3. Add the DNS names to the external.addresses field in order. For example, the first DNS name in the list is assigned to redpanda-0, the second is assigned to redpanda-1, and so on.

   --values

   load-balancer-dns.yaml

   external:
     enabled: true
     type: null
     addresses:
     - <load-balancer-dns-0>
     - <load-balancer-dns-1>
     - <load-balancer-dns-2>

   helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
     --values load-balancer-dns.yaml --reuse-values

   --set

   helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
     --set external.domain=<custom-domain> \
     --set external.type=null
     --set "external.addresses={<load-balancer-dns-0>,<load-balancer-dns-1>,<load-balancer-dns-2>}"
   note

   For default values and documentation for configuration options, see the values.yaml file.

4. Use the load balancers' DNS names to communicate with the Redpanda cluster from outside the Kubernetes cluster:

   rpk cluster info --brokers <load-balancer-dns-name>

Use the default Redpanda subdomains

You can use a DNS provider to create CNAME records that route traffic from the default Redpanda subdomains to the addresses of the corresponding load balancer instances. The default subdomains are redpanda-<ordinal-number>, where redpanda is the name of the chart and the <ordinal-number> placeholder is generated by the StatefulSet.

1. Get a domain from a domain registrar.

2. Specify your domain name in the external.domain configuration.

   Replace <custom-domain> with your domain.

   --values

   custom-domain.yaml

   external:
     domain: <custom-domain>
       enabled: true
       type: null

   helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
     --values custom-domain.yaml --reuse-values

   --set

   helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
     --set external.enabled=true \
     --set external.type=null \
     --set external.domain=<custom-domain>
   
   note

   For default values and documentation for configuration options, see the values.yaml file.

3. Find the DNS names for each of your load balancers.

4. Update the CNAME record for your domain so that each hostname (subdomain) points to the correct load balancer's DNS name.

   Hostname	Load balancer DNS
   redpanda-0	<load-balancer-dns-0>
   redpanda-1	<load-balancer-dns-1>
   redpanda-2	<load-balancer-dns-2>

5. Wait for your DNS changes to be propagated.

6. Use your domain to communicate with the Redpanda cluster from outside the Kubernetes cluster:

   rpk cluster info --brokers redpanda-0.<custom-domain>

Use custom subdomains

You can give each Redpanda broker a custom subdomain to advertise instead of the default subdomains. Then, you can use a DNS provider to create CNAME records that route traffic from the custom subdomains to the corresponding load balancer instances.

1. Get a domain from a domain registrar.

2. Choose an option to configure the subdomains:

   • To use fully custom subdomains, do the following.

     Replace <custom-domain> with your domain, and replace the placeholders in the external.addresses configuration with your own subdomains in the order that you want them to be applied to the Redpanda brokers.

     The subdomains are given to each Redpanda broker in order. For example, the Redpanda broker running inside the redpanda-0 Pod will advertise <subdomain-for-broker-0>.<custom-domain>.

     --values

     custom-subdomain.yaml

     external:
       enabled: true
       type: null
       domain: <custom-domain>
       addresses:
       - <subdomain-for-broker-0>
       - <subdomain-for-broker-1>
       - <subdomain-for-broker-2>

     helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
       --values custom-subdomain.yaml --reuse-values

     --set

     helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
       --set external.enabled=true \
       --set external.type=null \
       --set external.domain=<custom-domain> \
       --set "external.addresses={<subdomain-for-broker0>,<subdomain-for-broker1>,<subdomain-for-broker2>}"
     important

     For default values and documentation for configuration options, see the values.yaml file.

   • Or, to use custom subdomains that are suffixed with the index of the StatefulSet replica, do the following.

     Replace <custom-domain> with your domain, and replace <custom-subdomain> with your subdomain.

     This configuration renames your Pods to <subdomain>-<ordinal-number>. Your Redpanda brokers will advertise the <subdomain>-<ordinal-number>.<custom-domain> address.

     --values

     custom-subdomain-ordinal.yaml

     fullnameOverride: <custom-subdomain>
     external:
       enabled: true
       type: null
       domain: <custom-domain>

     helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
       --values custom-subdomain-ordinal.yaml --reuse-values

     --set

     helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
       --set external.enabled=true \
       --set external.type=null \
       --set external.domain=<custom-domain> \
       --set fullnameOverride=<custom-subdomain>
     important

     For default values and documentation for configuration options, see the values.yaml file.

3. Find the DNS names for each of your load balancers.

4. Update the CNAME record for your domain so that each hostname (subdomain) points to the correct load balancer's DNS name.

   Hostname	Load balancer DNS
   <subdomain-for-broker-0>	<load-balancer-dns-0>
   <subdomain-for-broker-1>	<load-balancer-dns-1>
   <subdomain-for-broker-2>	<load-balancer-dns-2>

5. Wait for your DNS changes to be propagated.

6. Use your domain to communicate with the Redpanda cluster from outside the Kubernetes cluster:

   rpk cluster info --brokers <subdomain-for-broker-0>.<custom-domain>

Next steps

• Configure security for your listeners.

• Configure listeners.