Skip to main content
Version: 22.3

Deploy for Production

This section describes how to set up a production cluster of Redpanda.

See also:

Prepare infrastructure​

For best performance, provision hardware according to these requirements:

  • XFS for the data directory of Redpanda (/var/lib/redpanda/data)
  • A kernel that is at least 3.10.0-514, 4.18 or newer
  • Local NVMe, RAID-0 when using multiple disks
  • 2GB of memory per core
  • TCP ports:
    • 33145 - internal RPC
    • 9092 - Kafka API
    • 8082 - HTTP Proxy
    • 8081 - Schema Registry
    • 9644 - Prometheus and Admin API

Install Redpanda​

Install Redpanda on each system you want to be part of your cluster. There are binaries available for Fedora/RedHat or Debian systems.

You can also install Redpanda using an Ansible playbook.

curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \
sudo -E bash && sudo yum install redpanda -y

Install Redpanda Console​

Redpanda Console is a developer-friendly web UI for managing and debugging your Redpanda cluster and your applications.

For each new release, Redpanda compiles the Redpanda Console to a single binary for Linux, macOS, and Windows. You can find the binaries in the attachments of each release on GitHub.

curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \
sudo -E bash && sudo yum install redpanda-console -y

Set Redpanda to production mode​

By default, Redpanda is installed in development mode, which turns off hardware optimization.

To enable hardware optimization, set Redpanda to run in production mode:

sudo rpk redpanda mode production

To tune the hardware, on each node, run:

sudo rpk redpanda tune all
Optional: Benchmark your SSD

On taller machines, Redpanda recommends benchmarking your SSD. This can be done with rpk iotune. You only need to run this once.

For reference, a local NVMe SSD should yield around 1GB/s sustained writes. rpk iotune captures SSD wear and tear and gives accurate measurements of what your hardware is capable of delivering. Run this before benchmarking.

If you're on AWS, GCP, or Azure, creating a new instance and upgrading to an image with a recent Linux kernel version is often the easiest way to work around bad devices.

sudo rpk iotune # takes 10mins

Start Redpanda​

Configure Redpanda using the rpk redpanda config bootstrap command, then start Redpanda:

sudo rpk redpanda config bootstrap --self <private-ip> --ips <seed-node-ips> && \
sudo rpk redpanda config set redpanda.empty_seed_starts_cluster false && \
sudo systemctl start redpanda-tuner redpanda
  • The --self flag tells Redpanda to which interface address to bind. Usually this is its private IP.
  • The --ips flag lists all the seed nodes in the cluster (seed nodes correspond to the seed_servers property in redpanda.yaml).

When a Redpanda cluster starts, it instantiates a controller Raft group with all the seed nodes that are specified in the --ips flag. After all seed nodes complete their startup procedure and become accessible, the cluster is then available. After that, non-seed nodes start up and are added to the cluster.

important
  • Redpanda strongly recommends at least three seed nodes when forming a cluster. A larger number of seed nodes increases the robustness of consensus and minimizes any chance that new clusters get spuriously formed after nodes are lost or restarted without any data.
  • It's important to have one or more seed nodes in each fault domain (such as rack or cloud AZ). A higher number provides a stronger guarantee that clusters don’t fracture unintentionally.
  • It's possible to change the seed nodes for a short period of time after a cluster has been created. For example, you may want to designate one additional broker as a seed node to increase availability. To do this without cluster downtime, add the new broker to seed_servers and restart Redpanda to apply the change on a broker-by-broker basis.

Start Redpanda Console​

  1. Start Redpanda Console:

    sudo systemctl start redpanda-console
  2. Make sure that Redpanda Console is active and running:

    sudo systemctl status redpanda-console

Verify the installation​

To verify that the Redpanda cluster is up and running, check the logs:

journalctl -u redpanda

If topics were initially created in a test environment with a replication factor of 1, use rpk topic alter-config to change the topic replication factor:

rpk topic alter-config [TOPICS...] --set replication.factor=3

To create a topic:

rpk topic create panda

Next steps​

If clients will connect from a different subnet, see Configuring Listeners.

Suggested reading​

What do you like about this doc?




Optional: Share your email address if we can contact you about your feedback.

Let us know what we do well: