Skip to main content
Version: 22.3

Deploying Redpanda for Production

This section describes how to set up a production cluster of Redpanda.

See also:

Prepare infrastructure

For best performance, provision hardware according to these requirements:

  • XFS for the data directory of Redpanda (/var/lib/redpanda/data)
  • A kernel that is at least 3.10.0-514, 4.18 or newer
  • Local NVMe, RAID-0 when using multiple disks
  • 2GB of memory per core
  • TCP ports:
    • 33145 - internal RPC
    • 9092 - Kafka API
    • 8082 - HTTP Proxy
    • 8081 - Schema Registry
    • 9644 - Prometheus and Admin API

Install the Redpanda binary

Install the binary on either Fedora/RedHat or Debian systems.

You can also install Redpanda using an Ansible playbook.

curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \
sudo -E bash && sudo yum install redpanda -y

Set Redpanda production mode

By default, Redpanda is installed in development mode, which turns off hardware optimization.

To enable hardware optimization, set Redpanda to run in production mode:

sudo rpk redpanda mode production

To tune the hardware, on each node, run:

sudo rpk redpanda tune all
Optional: Benchmark your SSD

On taller machines, Redpanda recommends benchmarking your SSD. This can be done with rpk iotune. You only need to run this once.

For reference, a local NVMe SSD should yield around 1GB/s sustained writes. rpk iotune captures SSD wear and tear and gives accurate measurements of what your hardware is capable of delivering. Run this before benchmarking.

If you're on AWS, GCP, or Azure, creating a new instance and upgrading to an image with a recent Linux kernel version is often the easiest way to work around bad devices.

sudo rpk iotune # takes 10mins

Configure and start seed nodes

Configure and start up the seed servers with the rpk redpanda config bootstrap command:

sudo rpk redpanda config bootstrap --self <private ip> --ips <seed nodes ips> && \
sudo rpk redpanda config set redpanda.empty_seed_starts_cluster false && \
sudo systemctl start redpanda-tuner redpanda
  • The --self flag tells the node which interface address to bind to. Usually this is its private IP.
  • The --ips flag must be identical on all nodes when deploying a cluster for the first time.

A Redpanda cluster starts with the seed nodes that are specified in the --ips flag, instantiating a controller Raft group with the seed nodes. The cluster grows as nodes start up and is available after all seed servers are up.

info
  • Redpanda strongly recommends at least three seed nodes when forming a cluster. A larger number of seed nodes increases the robustness of consensus and minimizes any chance that new clusters get spuriously formed after nodes are lost or restarted without any data.
  • It's important to have one or more seed nodes in each fault domain (such as rack or cloud AZ). A higher number provides a stronger guarantee that clusters don’t fracture unintentionally.
  • It's possible to change the seed nodes for a short period of time after a cluster has been created. For example, you may want to designate one additional node as a seed node to increase availability. To do this without cluster downtime, add the new node to seed_servers and restart Redpanda to apply the change on a node-by-node basis.

If clients will connect from a different subnet, see Configuring Listeners.

Configure and start other nodes

Let every other node know where to reach the seed nodes:

sudo rpk redpanda config bootstrap --self <private ip> --ips <seed nodes ips> && \
sudo rpk redpanda config set redpanda.empty_seed_starts_cluster false && \
sudo systemctl start redpanda-tuner redpanda

Verify the installation

To verify that the cluster is up and running, check the logs:

journalctl -u redpanda

If topics were initially created in a test environment with a replication factor of 1, use rpk topic alter-config to change the topic replication factor:

rpk topic alter-config [TOPICS...] --set replication.factor 3

To create a topic:

rpk topic create panda

Suggested reading