Docs Self-Managed Get Started Quickstarts Redpanda Self-Managed You are viewing the Self-Managed v24.3 beta documentation. We welcome your feedback at the Redpanda Community Slack #beta-feedback channel. To view the latest available version of the docs, see v24.2. Redpanda Self-Managed Quickstart Learn how to quickly start working with a local Redpanda cluster. This quickstart introduces you to Redpanda Self-Managed in a self-hosted environment with a minimal setup. Docker Compose provides a straightforward and reproducible setup process, whether you’re exploring Redpanda for the first time or testing new configurations. The option between a single broker and a three-broker setup helps you test Redpanda for simplicity and resilience. Redpanda in Docker is supported only for development and testing. To deploy Redpanda Self-Managed in production, use one of the following environments: Linux Kubernetes To download the Redpanda binary, see GitHub. Prerequisites You need Docker Compose. For installation instructions, see the Docker Compose documentation. To check if you have Docker Compose installed, run: docker compose version You should see the version of Docker Compose that’s installed on your local machine. Deploy Redpanda For lightweight testing, you can start a single Redpanda broker, or you can use a more robust example with three Redpanda brokers. A single broker setup is the simplest way to get Redpanda up and running. It’s ideal for initial exploration, learning, or development environments where high availability and fault tolerance are not critical concerns. For production environments where you need more resilience, a three-broker setup is recommended. This configuration allows you to leverage Redpanda’s replication capabilities, enhancing the durability and availability of your data. Single Broker Three Brokers Download the following docker-compose.yml file on your local file system. Reveal the YAML content docker-compose.yml name: redpanda-quickstart networks: redpanda_network: driver: bridge volumes: redpanda-0: null redpanda-1: null redpanda-2: null services: ################## # Redpanda Brokers # ################## redpanda-0: command: - redpanda - start - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092 # Address the broker advertises to clients that connect to the Kafka API. # Use the internal addresses to connect to the Redpanda brokers' # from inside the same Docker network. # Use the external addresses to connect to the Redpanda brokers' # from outside the Docker network. - --advertise-kafka-addr internal://redpanda-0:9092,external://localhost:19092 - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:18082 # Address the broker advertises to clients that connect to the HTTP Proxy. - --advertise-pandaproxy-addr internal://redpanda-0:8082,external://localhost:18082 - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081 # Redpanda brokers use the RPC API to communicate with each other internally. - --rpc-addr redpanda-0:33145 - --advertise-rpc-addr redpanda-0:33145 # Mode dev-container uses well-known configuration properties for development in containers. - --mode dev-container # Tells Seastar (the framework Redpanda uses under the hood) to use 1 core on the system. - --smp 1 - --default-log-level=info image: docker.redpanda.com/redpandadata/redpanda-unstable:v24.3.1-rc6 container_name: redpanda-0 volumes: - redpanda-0:/var/lib/redpanda/data networks: - redpanda_network ports: - 18081:18081 - 18082:18082 - 19092:19092 - 19644:9644 #################### # Redpanda Console # #################### console: container_name: redpanda-console image: docker.redpanda.com/redpandadata/console:v2.8.0-beta.1 networks: - redpanda_network entrypoint: /bin/sh command: -c 'echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml; /app/console -config.filepath=${CONFIG_FILEPATH:-/tmp/config.yml}' volumes: - ./config:/tmp/config/ environment: CONFIG_FILEPATH: ${CONFIG_FILEPATH:-/tmp/config.yml} CONSOLE_CONFIG_FILE: | kafka: brokers: ["redpanda-0:9092"] schemaRegistry: enabled: true urls: ["http://redpanda-0:8081"] redpanda: adminApi: enabled: true urls: ["http://redpanda-0:9644"] ports: - 8080:8080 depends_on: - redpanda-0 Run the following in the directory where you saved the docker-compose.yml file: docker compose up -d [+] Running 4/4 ⠿ Network redpanda_network Created 0.0s ⠿ Volume "redpanda-quickstart_redpanda-0" Created 0.0s ⠿ Container redpanda-0 Started 0.3s ⠿ Container redpanda-console Started 0.6s Download the following docker-compose.yml file on your local file system. Reveal the YAML content docker-compose.yml name: redpanda-quickstart networks: redpanda_network: driver: bridge volumes: redpanda-0: null redpanda-1: null redpanda-2: null services: ################## # Redpanda Brokers # ################## redpanda-0: command: - redpanda - start - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092 # Address the broker advertises to clients that connect to the Kafka API. # Use the internal addresses to connect to the Redpanda brokers' # from inside the same Docker network. # Use the external addresses to connect to the Redpanda brokers' # from outside the Docker network. - --advertise-kafka-addr internal://redpanda-0:9092,external://localhost:19092 - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:18082 # Address the broker advertises to clients that connect to the HTTP Proxy. - --advertise-pandaproxy-addr internal://redpanda-0:8082,external://localhost:18082 - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081 # Redpanda brokers use the RPC API to communicate with each other internally. - --rpc-addr redpanda-0:33145 - --advertise-rpc-addr redpanda-0:33145 # Mode dev-container uses well-known configuration properties for development in containers. - --mode dev-container # Tells Seastar (the framework Redpanda uses under the hood) to use 1 core on the system. - --smp 1 - --default-log-level=info image: docker.redpanda.com/redpandadata/redpanda-unstable:v24.3.1-rc6 container_name: redpanda-0 volumes: - redpanda-0:/var/lib/redpanda/data networks: - redpanda_network ports: - 18081:18081 - 18082:18082 - 19092:19092 - 19644:9644 redpanda-1: command: - redpanda - start - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:29092 - --advertise-kafka-addr internal://redpanda-1:9092,external://localhost:29092 - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:28082 - --advertise-pandaproxy-addr internal://redpanda-1:8082,external://localhost:28082 - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:28081 - --rpc-addr redpanda-1:33145 - --advertise-rpc-addr redpanda-1:33145 - --mode dev-container - --smp 1 - --default-log-level=info - --seeds redpanda-0:33145 image: docker.redpanda.com/redpandadata/redpanda-unstable:v24.3.1-rc6 container_name: redpanda-1 volumes: - redpanda-1:/var/lib/redpanda/data networks: - redpanda_network ports: - 28081:28081 - 28082:28082 - 29092:29092 - 29644:9644 depends_on: - redpanda-0 redpanda-2: command: - redpanda - start - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:39092 - --advertise-kafka-addr internal://redpanda-2:9092,external://localhost:39092 - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:38082 - --advertise-pandaproxy-addr internal://redpanda-2:8082,external://localhost:38082 - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:38081 - --rpc-addr redpanda-2:33145 - --advertise-rpc-addr redpanda-2:33145 - --mode dev-container - --smp 1 - --default-log-level=info - --seeds redpanda-0:33145 image: docker.redpanda.com/redpandadata/redpanda-unstable:v24.3.1-rc6 container_name: redpanda-2 volumes: - redpanda-2:/var/lib/redpanda/data networks: - redpanda_network ports: - 38081:38081 - 38082:38082 - 39092:39092 - 39644:9644 depends_on: - redpanda-0 #################### # Redpanda Console # #################### console: container_name: redpanda-console image: docker.redpanda.com/redpandadata/console:v2.8.0-beta.1 networks: - redpanda_network entrypoint: /bin/sh command: -c 'echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml; /app/console -config.filepath=${CONFIG_FILEPATH:-/tmp/config.yml}' volumes: - ./config:/tmp/config/ environment: CONFIG_FILEPATH: ${CONFIG_FILEPATH:-/tmp/config.yml} CONSOLE_CONFIG_FILE: | kafka: brokers: ["redpanda-0:9092", "redpanda-1:9092", "redpanda-2:9092"] schemaRegistry: enabled: true urls: ["http://redpanda-0:8081", "http://redpanda-1:8081", "http://redpanda-2:8081"] redpanda: adminApi: enabled: true urls: ["http://redpanda-0:9644", "http://redpanda-1:9644", "http://redpanda-2:9644"] ports: - 8080:8080 depends_on: - redpanda-0 - redpanda-1 - redpanda-2 Run the following in the directory where you saved the docker-compose.yml file: docker compose up -d [+] Running 8/8 ⠿ Network redpanda_network Created 0.0s ⠿ Volume "redpanda-quickstart_redpanda-0" Created 0.0s ⠿ Volume "redpanda-quickstart_redpanda-1" Created 0.0s ⠿ Volume "redpanda-quickstart_redpanda-2" Created 0.0s ⠿ Container redpanda-0 Started 0.3s ⠿ Container redpanda-console Started 0.6s ⠿ Container redpanda-1 Started 0.7s ⠿ Container redpanda-2 Started 0.7s Start streaming To start building a basic streaming application, you can use the rpk command-line tool to create a topic, produce messages to it, and consume messages from it. Each Redpanda broker comes preinstalled with rpk, so you can use it inside one of the Redpanda broker’s Docker containers, or you can install it on your local machine to test external connectivity. The docker-compose.yml file configured the containers to expose ports on your localhost, so you can communicate with the cluster outside the Docker network. To use rpk inside the Redpanda broker’s Docker container: Get information about the cluster: docker exec -it redpanda-0 rpk cluster info Create a topic called chat-room: docker exec -it redpanda-0 rpk topic create chat-room Output: TOPIC STATUS chat-room OK If you deployed three brokers, you can configure your topics with a replication factor of three. This replication factor provides high availability and data durability. For more details, see Choose the replication factor. docker exec -it redpanda-0 rpk topic create chat-room --replicas 3 Produce a message to the topic: docker exec -it redpanda-0 rpk topic produce chat-room Enter a message, then press Enter: Pandas are fabulous! Example output: Produced to partition 0 at offset 0 with timestamp 1663282629789. Press Ctrl+C to finish producing messages to the topic. Consume one message from the topic: docker exec -it redpanda-0 rpk topic consume chat-room --num 1 Your message is displayed along with its metadata: { "topic": "chat-room", "value": "Pandas are fabulous!", "timestamp": 1663282629789, "partition": 0, "offset": 0 } To test external connectivity using your local machine: Install rpk on your local machine: The rpk binary is not supported on Windows. Linux macOS amd64 arm64 curl -LO https://github.com/redpanda-data/redpanda/releases/latest/download/rpk-linux-amd64.zip && mkdir -p ~/.local/bin && export PATH="~/.local/bin:$PATH" && unzip rpk-linux-amd64.zip -d ~/.local/bin/ curl -LO https://github.com/redpanda-data/redpanda/releases/latest/download/rpk-linux-arm64.zip && mkdir -p ~/.local/bin && export PATH="~/.local/bin:$PATH" && unzip rpk-linux-amd64.zip -d ~/.local/bin/ If you don’t have Homebrew installed, install it. To install or update rpk, run: brew install redpanda-data/tap/redpanda This method installs the latest version of rpk, which is supported only with the latest version of Redpanda. Get information about the cluster to test the connection: rpk cluster info -X brokers=127.0.0.1:19092 The Redpanda broker returns a list of all broker addresses, so rpk can communicate with all brokers directly. Each broker returns its configured advertise-* address that matches the port to which the initial connection has been made. Explore your topic To explore your topic, you can use Redpanda Console, which you deployed in the docker-compose.yml file and is running locally on port 8080. Redpanda Console is a developer-friendly web UI for managing and debugging your Redpanda cluster and your applications. Open Redpanda Console in a web browser. An overview of the cluster status, cluster health, and broker details is displayed. Go to Topics > chat-room. The message that you produced to the topic is displayed along with some other details about the topic. Customize the quickstart If you want to configure Redpanda or Redpanda Console, you need to customize the example Docker Compose file. Configure Redpanda in Docker To configure the Redpanda services with node configuration properties, you can do the following: Pass configuration properties to the --set option in the redpanda start command. For example: redpanda: command: - redpanda - start - --set pandaproxy_client.retries=6 Create a redpanda.yaml file and mount it to the /etc/redpanda/ directory on the redpanda containers. For example, if you create the redpanda.yaml file in a directory called redpanda-mounts, configure the following volume mounts: redpanda: volumes: - ./redpanda-mounts:/etc/redpanda/ Configure Redpanda Console in Docker To configure the Redpanda Console service with configuration properties, you can do the following: Use environment variables, for example: console: environment: KAFKA_RACKID: rack1 Create a redpanda-console-config.yaml file and mount it to the /etc/redpanda/ directory on the redpanda-console container. For example, if you create the redpanda-console-config.yaml file in a directory called console-mounts, configure the following volume mounts: console: volumes: - ./console-mounts:/etc/redpanda/ Clean up If you don’t want to continue experimenting with your cluster, you can shut it down and delete the containers: docker compose down To delete the volumes along with all your cluster data: docker compose down -v Next steps Try more examples in Redpanda Labs Learn more about Redpanda Console Learn more about rpk Deploy for development or production Find the Docker images for Redpanda on Docker Hub Find the Docker images for Redpanda Console on Docker Hub Suggested reading Explore the rpk commands that you used in this quickstart: rpk cluster rpk topic create rpk topic produce rpk topic consume Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Quickstarts Redpanda Console