23.3 Beta 23.2 23.1 22.3 22.2 22.1 21.11

Deploying Redpanda for Production

This section describes how to set up a production cluster of Redpanda.

To try out Redpanda, see the Install Redpanda guides.

Before you set up your Redpanda cluster, see Manage Disk Space for guidelines on cluster creation.

Prepare infrastructure

For best performance, provision hardware according to these requirements:

  • XFS for the data directory of Redpanda (/var/lib/redpanda/data)

  • A kernel that is at least 3.10.0-514, 4.18 or newer is preferred

  • Local NVMe, RAID-0 when using multiple disks

  • 2GB of memory per core

  • TCP ports:

    • 33145 - internal RPC port

    • 9092 - Kafka API port

    • 8082 - HTTP Proxy port

    • 8081 - Schema Registry port

    • 9644 - Prometheus and HTTP admin port

You can use Terraform to deploy Redpanda.

Install Redpanda

After the hardware is provisioned, install Redpanda and configure it for production use.

You can also install Redpanda using an Ansible playbook.

Install the binary

Install the binary on either Fedora/RedHat or Debian systems.

  • Fedora/RedHat

  • Debian

curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \
sudo -E bash && sudo yum install redpanda -y
curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.deb.sh' | \
sudo -E bash && sudo apt install redpanda -y

Set Redpanda production mode

By default, Redpanda is installed in development mode, which turns off hardware optimization.

To enable hardware optimization, set Redpanda to run in production mode:

sudo rpk redpanda mode production

To tune the hardware, on each node:

sudo rpk redpanda tune all
Optional: Benchmark your SSD

On taller machines, Redpanda recommends benchmarking your SSD. This can be done with rpk iotune. You only need to run this once. For reference, a local NVMe SSD should yield around 1GB/s sustained writes. rpk iotune captures SSD wear and tear and gives accurate measurements of what your hardware is capable of delivering. Run this before benchmarking.

If you’re on AWS, GCP, or Azure, creating a new instance and upgrading to an image with a recent Linux kernel version is often the easiest way to work around bad devices.

sudo rpk iotune # takes 10mins

Configure and start the root node

After the software is installed, you must configure it. The first step is to set up the root node. The root node starts as a standalone node, and every other one joins it, forming a cluster along the way.

For the root node, choose 0 as its ID. The --self option tells the node which interface address to bind to. Usually you want that to be its private IP.

sudo rpk config bootstrap --id 0 --self <ip> && \
sudo systemctl start redpanda-tuner redpanda

If clients will connect from a different subnet, see Configuring Listeners.

Configure and start other nodes

For every other node, you have to choose a unique integer ID for it, and let it know where to reach the root node.

sudo rpk config bootstrap --id <unique id> \
--self <private ip>                        \
--ips <root node ip> &&                    \
sudo systemctl start redpanda-tuner redpanda

Verify the installation

To verify that the cluster is up and running, check the logs:

journalctl -u redpanda

To create a topic:

rpk topic create panda

Suggested reading