23.2 23.1 22.3 22.2 22.1 21.11

Deploy for Development

You can deploy Redpanda using well-known configuration properties optimized for a development or test environment. This configuration uses less system resources and simplifies setup, but it’s not suitable for a production deployment. For example, in development mode, the default group_topic_partitions is 3, and the default storage_min_free_bytes is 1 GiB. In production mode, the default group_topic_partitions is 16, and the default storage_min_free_bytes is 5 GiB.

Development mode bypasses fsync, acknowledging messages before they’re stored to disk. This reduces the durability of messages, could cause potential data loss, and could give unrealistic performance data for a production environment.

See Deploy for Production to deploy for a production environment.

See Redpanda Quickstart to try out Redpanda in Docker.

Prerequisites

Hardware and software

The same prerequisites for a production environment are recommended for a development environment.

  • Operating system

    • Minimum recommended version of RHEL of RHEL/CentOS: 8

    • Minimum version of Ubuntu: 21.1

  • CPU and memory

    • A minimum of three physical nodes or virtual machines are required.

    • Two physical (not virtual) cores are required. Four physical cores are strongly recommended.

    • x86_64 (Westmere or newer) and AWS Graviton family processors are supported.

  • Storage

    • An XFS or ext4 file system for the data directory of Redpanda (/var/lib/redpanda/data) or the Tiered Storage cache. XFS is highly recommended. NFS is not supported.

    • Locally-attached NVMe devices. RAID-0 is required if you use multiple disks.

    • Ephemeral cloud instance storage is only recommended in combination with Tiered Storage or for Tiered Storage cache. Without Tiered Storage, attached persistent volumes (for example, EBS) are recommended.

  • Object storage providers for Tiered Storage

    • Amazon Simple Storage Service (S3)

    • Google Cloud Storage (GCS), using the Google Cloud Platform S3 API

    • Azure Blob Storage (ABS)

  • Networking

    • Minimum 10 GigE

  • See Manage Disk Space for guidelines on cluster creation.

TCP/IP ports

Redpanda uses the following default ports:

Port Purpose

9092

Kafka API

8082

HTTP Proxy

8081

Schema Registry

9644

Admin API and Prometheus

33145

internal RPC

Install Redpanda

Install Redpanda on each system you want to be part of your cluster. There are binaries available for Fedora/RedHat or Debian systems.

You can also install Redpanda using an Ansible playbook.

  • Fedora/RedHat

  • Debian/Ubuntu

curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \
sudo -E bash && sudo yum install redpanda -y
curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.deb.sh' | \
sudo -E bash && sudo apt install redpanda -y

Install Redpanda Console

Redpanda Console is a developer-friendly web UI for managing and debugging your Redpanda cluster and your applications. Install Redpanda Console alongside a Redpanda cluster.

For each new release, Redpanda compiles the Redpanda Console to a single binary for Linux, macOS, and Windows. You can find the binaries in the attachments of each release on GitHub.

  • Fedora/RedHat

  • Debian/Ubuntu

curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \
sudo -E bash && sudo yum install redpanda-console -y
curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.deb.sh' | \
sudo -E bash && sudo apt-get install redpanda-console -y

Start Redpanda

Configure Redpanda using the rpk redpanda config bootstrap command, then start Redpanda:

sudo rpk redpanda config bootstrap --self <private-ip> --ips <seed-node-ips> && \
sudo rpk redpanda config set redpanda.empty_seed_starts_cluster false && \
sudo systemctl start redpanda

  • The --self flag tells Redpanda the interface address to bind to. Usually this is its private IP.

  • The --ips flag lists all the seed nodes in the cluster, including the one being started. Seed nodes correspond to the seed_servers property in redpanda.yaml.

    • The --ips flag must be set identically (with nodes listed in identical order) on each node.

    • Do not run sudo rpk redpanda tune all on development deployments.

When a Redpanda cluster starts, it instantiates a controller Raft group with all the seed nodes that are specified in the --ips flag. After all seed nodes complete their startup procedure and become accessible, the cluster is then available. After that, non-seed nodes start up and are added to the cluster.

  • It’s important to have one or more seed nodes in each fault domain (such as rack or cloud AZ). A higher number provides a stronger guarantee that clusters don’t fracture unintentionally.

  • It’s possible to change the seed nodes for a short period of time after a cluster has been created. For example, you may want to designate one additional broker as a seed node to increase availability. To do this without cluster downtime, add the new broker to seed_servers and restart Redpanda to apply the change on a broker-by-broker basis.

  • If firewall ports block the internal RPC port 33145, then the non-seed nodes could take time to fail. If that happens, run sudo systemctl stop redpanda (or yum remove redpanda or apt-get remove redpanda), fix the firewall issue, and then start Redpanda again.

Start Redpanda Console

  1. Start Redpanda Console:

    sudo systemctl start redpanda-console

  2. Make sure that Redpanda Console is active and running:

    sudo systemctl status redpanda-console

Verify the installation

To verify that the Redpanda cluster is up and running, use rpk to get information about the cluster:

rpk cluster info

To create a topic:

rpk topic create panda

Next steps

If clients connect from a different subnet, see Configure Listeners.

Suggested reading