Deploy for Development
You can deploy Redpanda using well-known configuration properties optimized for a development or test environment. This configuration uses less system resources and simplifies setup, but it’s not suitable for a production deployment. For example, in development mode, the default group_topic_partitions
is 3, and the default storage_min_free_bytes
is 1 GiB. In production mode, the default group_topic_partitions
is 16, and the default storage_min_free_bytes
is 5 GiB.
|
To deploy for a production environment, see Deploy for Production. Or to try out Redpanda in Docker, see Redpanda Quickstart.
Prerequisites
Make sure you meet the hardware and software requirements.
Install Redpanda
Install Redpanda on each system you want to be part of your cluster. There are binaries available for Fedora/RedHat or Debian systems.
-
Fedora/RedHat
-
Debian/Ubuntu
curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \
sudo -E bash && sudo yum install redpanda -y
curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.deb.sh' | \
sudo -E bash && sudo apt install redpanda -y
Install Redpanda Console
Redpanda Console is a developer-friendly web UI for managing and debugging your Redpanda cluster and your applications.
For each new release, Redpanda compiles the Redpanda Console to a single binary for Linux, macOS, and Windows. You can find the binaries in the attachments of each release on GitHub.
-
Fedora/RedHat
-
Debian/Ubuntu
curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \
sudo -E bash && sudo yum install redpanda-console -y
curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.deb.sh' | \
sudo -E bash && sudo apt-get install redpanda-console -y
Bootstrap broker configurations
Each broker requires a set of broker configurations that determine how all brokers communicate with each other and with clients. Bootstrapping a cluster configures the listener, seed servers, and advertised listeners, which ensure proper network connectivity and accessibility.
Starting in version 23.3.8, rpk
enhances the bootstrapping process with additional flags for configuring advertised listener addresses directly.
Use the rpk redpanda config bootstrap
command to bootstrap Redpanda:
sudo rpk redpanda config bootstrap --self <listener-address> --advertised-kafka <advertised-kafka-address> --ips <seed-server1-ip>,<seed-server2-ip>,<seed-server3-ip> && \
sudo rpk redpanda config set redpanda.empty_seed_starts_cluster false
Replace the following placeholders:
-
<listener-address>
: The--self
flag tells Redpanda the interfaces to bind to for the Kafka API, the RPC API, and the Admin API. These addresses determine on which network interface and port Redpanda listens for incoming connections.-
Set the listener address to
0.0.0.0
to listen on all network interfaces available on the machine. -
Set the listener address to a specific IP address to bind the listener to that address, restricting connections to that interface.
-
-
<advertised-kafka-address>
: The--advertised-kafka
flag sets a different advertised Kafka address, which is useful for scenarios where the accessible address differs from the bind address.Redpanda does not allow advertised addresses set to 0.0.0.0
. If you set any advertised addresses to0.0.0.0
, Redpanda will output startup validation errors. -
<seed-server-ips>
: The--ips
flag lists all the seed servers in the cluster, including the one being started.The --ips
flag must be set identically (with nodes listed in identical order) on each node.
Bootstrapping Redpanda updates your /etc/redpanda/redpanda.yaml
configuration file:
/etc/redpanda/redpanda.yaml
redpanda:
data_directory: /var/lib/redpanda/data
empty_seed_starts_cluster: false
seed_servers:
- host:
address: <seed-server1-ip>
port: 33145
- host:
address: <seed-server2-ip>
port: 33145
- host:
address: <seed-server3-ip>
port: 33145
rpc_server:
address: <listener-address>
port: 33145
kafka_api:
- address: <listener-address>
port: 9092
admin:
- address: <listener-address>
port: 9644
advertised_rpc_api:
address: <listener-address>
port: 33145
advertised_kafka_api:
- address: <advertised-kafka-address>
port: 9092
Recommendations
-
Redpanda Data strongly recommends at least three seed servers when forming a cluster. A larger number of seed servers increases the robustness of consensus and minimizes any chance that new clusters get spuriously formed after brokers are lost or restarted without any data.
-
It’s important to have one or more seed servers in each fault domain (for example, in each rack or cloud AZ). A higher number provides a stronger guarantee that clusters don’t fracture unintentionally.
-
It’s possible to change the seed servers for a short period of time after a cluster has been created. For example, you may want to designate one additional broker as a seed server to increase availability. To do this without cluster downtime, add the new broker to the
seed_servers
property and restart Redpanda to apply the change on a broker-by-broker basis.
Listeners for mixed environments
For clusters serving both internal and external clients, configure multiple listeners for the Kafka API to separate internal from external traffic.
For more details, see Configure Listeners.
Start Redpanda
To start Redpanda:
sudo systemctl start redpanda-tuner redpanda
When a Redpanda cluster starts, it instantiates a controller Raft group with all the seed servers specified in the --ips
flag. After all seed servers complete their startup procedure and become accessible, the cluster is then available. After that, non-seed servers start up and are added to the cluster.
Start Redpanda Console
-
Start Redpanda Console:
sudo systemctl start redpanda-console
-
Make sure that Redpanda Console is active and running:
sudo systemctl status redpanda-console
Verify the installation
To verify that the Redpanda cluster is up and running, use rpk
to get information about the cluster:
rpk cluster info
You should see a list of advertised addresses.
To create a topic:
rpk topic create <topic-name>
If topics were initially created in a test environment with a replication factor of 1
, use rpk topic alter-config
to change the topic replication factor:
rpk topic alter-config <topic-names> --set replication.factor=3
Perform a self test
To understand the performance capabilities of your Redpanda cluster, Redpanda offers built-in self-test features that evaluate the performance of both disk and network operations.
For more information, see Disk and network self-test benchmarks.
Next steps
-
If clients connect from a different subnet, see Configure Listeners.
-
Observability is essential. See Monitor Redpanda.