Skip to main content
Version: 23.1

Configure External Access through LoadBalancer Services

Learn how to expose your Redpanda cluster to clients outside of your Kubernetes cluster by using LoadBalancer Services.

Here's what you'll need to do:

  1. Set the external.type field to LoadBalancer to create a LoadBalancer Service for each replica in the StatefulSet.
  2. Advertise your brokers to the outside world using either a custom domain or the DNS names or IP addresses of their load balancer instances.
important

For clusters with TLS enabled (default), advertising the DNS names or IP addresses of load balancer instances is not supported. Instead, use custom domains.

How external access works with LoadBalancer Services​

When the external.type field is set to LoadBalancer, the Helm chart creates one LoadBalancer Service for each replica in the StatefulSet. Each LoadBalancer Service routes external traffic to the listener ports on a single Redpanda broker.

The Helm chart adds the spec.selector.statefulset.kubernetes.io/pod-name setting to each LoadBalancer Service so that each targets only one Redpanda broker. Each Pod in a StatefulSet is automatically given the statefulset.kubernetes.io/pod-name label, which contains the name of the Pod. By setting the spec.selector.statefulset.kubernetes.io/pod-name field of the LoadBalancer Service to the name of a Pod, it makes the LoadBalancer Service target only the individual Redpanda broker and not the whole cluster.

A client connects to a Pod through a load balancer

Prerequisites​

  • One worker node for each Redpanda broker that you want to deploy.

  • Install rpk on your local machine so that you can test connections to your Redpanda cluster from outside Kubernetes.

  • Understand Kubernetes networking for Redpanda.

  • Review the best practices.

  • Make sure that your Kubernetes cluster is accessible through your desired node port range. You may need to edit your inbound firewall rules.

To configure your Redpanda brokers to advertise a custom domain, you must give each broker a subdomain and configure your DNS records to point those subdomains at the addresses of the brokers' corresonding load balancer instances.

Your brokers can advertise one of the following:

You can use a DNS provider to create CNAME records that route traffic from the default Redpanda subdomains to the addresses of the corresponding load balancer instances. The default subdomains are redpanda-<ordinal-number>, where redpanda is the name of the chart and the <ordinal-number> placeholder is generated by the StatefulSet.

  1. Get a domain from a domain registrar.

  2. Specify your domain name in the external.domain configuration.

    Replace <custom-domain> with your domain.

    custom-domain.yaml
    external:
      enabled: true
      domain: <custom-domain>
      type: LoadBalancer
    helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
      --values custom-domain.yaml --reuse-values --wait
    note

    For default values and documentation for configuration options, see the values.yaml file.

  3. Find the addresses for each of your load balancers.

  4. Update the CNAME record for your domain so that each hostname (subdomain) points to the correct load balancer's address.

    HostnameLoad Balancer
    redpanda-0<load-balancer-address-0>
    redpanda-1<load-balancer-address-1>
    redpanda-2<load-balancer-address-2>

  5. Wait for your DNS changes to be propagated.

  6. Use your custom domain to communicate with the Redpanda cluster from outside the Kubernetes cluster:

    1. Save the root certificate authority (CA) to your local file system outside Kubernetes:

      kubectl -n redpanda get secret redpanda-default-root-certificate -o go-template='{{ index .data "ca.crt" | base64decode }}' > ca.crt

    2. Get information about the cluster:

      rpk cluster info \
        --tls-enabled \
        --tls-truststore=ca.crt \
        --brokers redpanda-0.<custom-domain>

You can give each Redpanda broker a custom subdomain to advertise instead of the default subdomains. Then, you can use a DNS provider to create CNAME records that route traffic from the custom subdomains to the corresponding load balancer instances.

  1. Get a domain from a domain registrar.

  2. Choose an option to configure the subdomains:

    • To use fully custom subdomains, do the following.

      Replace <custom-domain> with your domain, and replace the placeholders in the external.addresses configuration with your own subdomains in the order that you want them to be applied to the Redpanda brokers. The subdomains are given to each Redpanda broker in order of StatefulSet replicas. For example, the Redpanda broker running inside the redpanda-0 Pod will advertise <subdomain-for-broker-0>.<custom-domain>.

      custom-subdomain.yaml
      external:
        enabled: true
        type: LoadBalancer
        domain: <custom-domain>
        addresses:
        - <subdomain-for-broker-0>
        - <subdomain-for-broker-1>
        - <subdomain-for-broker-2>
      helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
        --values custom-subdomain.yaml --reuse-values --wait
      note

      For default values and documentation for configuration options, see the values.yaml file.

    • Or, to use custom subdomains that are suffixed with the index of the StatefulSet replica, do the following.

      Replace <custom-domain> with your domain, and replace <custom-subdomain> with your subdomain.

      This configuration renames your Pods to <subdomain>-<ordinal-number>. Your Redpanda brokers will advertise the <subdomain>-<ordinal-number>.<custom-domain> address.

      custom-subdomain-ordinal.yaml
      fullnameOverride: <custom-subdomain>
      external:
        enabled: true
        type: LoadBalancer
        domain: <custom-domain>
      helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
        --values custom-subdomain-ordinal.yaml --reuse-values --wait
      note

      For default values and documentation for configuration options, see the values.yaml file.

  3. Find the addresses for each of your load balancers.

  4. Update the CNAME record for your domain so that each hostname (subdomain) points to the correct load balancer's address.

    HostnameLoad balancer
    <subdomain-for-broker-0><load-balancer-address-0>
    <subdomain-for-broker-1><load-balancer-address-1>
    <subdomain-for-broker-2><load-balancer-address-2>

  5. Wait for your DNS changes to be propagated.

  6. Use your custom domain to communicate with the Redpanda cluster from outside the Kubernetes cluster:

    1. Save the root certificate authority (CA) to your local file system outside Kubernetes:

      kubectl -n redpanda get secret redpanda-default-root-certificate -o go-template='{{ index .data "ca.crt" | base64decode }}' > ca.crt

    2. Get information about the cluster:

      rpk cluster info \
        --tls-enabled \
        --tls-truststore=ca.crt \
        --brokers <subdomain-for-broker-0>.<custom-domain>

You can configure each Redpanda broker to advertise the DNS name or IP address of its corresonding load balancer instance.

important

If your cluster has TLS enabled (default), you must advertise custom domains because the Helm chart adds custom domains to the SAN list of TLS certificates and not IP addresses. Therefore, IP addresses assigned to LoadBalancer Services must be made resolvable by DNS names to ensure secure TLS access.

While adding entries to the /etc/hosts file may work for development purposes, it's not a suitable approach for production environments. In production, you'll need to update your organization's DNS service to make the IP addresses resolvable by DNS names. Updating your organization's DNS service ensures that users can access your services securely without encountering any SSL/TLS warnings or errors.

If you're hosting Redpanda on a managed Kubernetes platform, follow the steps in the Managed tab. Otherwise, follow the steps in the Bare-Metal tab.

  1. Deploy Redpanda with TLS disabled and enable the LoadBalancer Service type:

    loadbalancer-tls-disabled.yaml
    external:
      enabled: true
      type: LoadBalancer
    tls:
      enabled: false
    helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace \
      --values loadbalancer-tls-disabled.yaml --reuse-values --wait
    note

    For default values and documentation for configuration options, see the values.yaml file.

  2. Make sure that your managed Kubernetes platform assigned external addresses to your LoadBalancer Services:

    kubectl get service -n redpanda
    Example output
    NAME               TYPE           CLUSTER-IP       EXTERNAL-IP      
    lb-redpanda-0      LoadBalancer   10.100.113.102   loadbalancer1.com
    lb-redpanda-1      LoadBalancer   10.100.53.8      loadbalancer2.com
    lb-redpanda-2      LoadBalancer   10.100.231.13    loadbalancer3.com

  3. Configure the Redpanda brokers to advertise these external addresses:

    helm upgrade redpanda redpanda/redpanda -n redpanda --set $(kubectl get svc -n redpanda -o jsonpath='{"external.addresses={"}{ range .items[*]}{.status.loadBalancer.ingress[0].ip }{.status.loadBalancer.ingress[0].hostname}{","}{ end }{"}\n"}') --reuse-values --wait

  4. Use the load balancers' addresses to communicate with the Redpanda cluster from outside the Kubernetes cluster:

    rpk cluster info --brokers <load-balancer-address>:31092

Next steps​

What do you like about this doc?




Optional: Share your email address if we can contact you about your feedback.

Let us know what we do well:
Ask in the community
Make a contribution