Redpanda Quickstart

Spin up a Redpanda cluster in Docker to evaluate Redpanda on your Linux, macOS, or Windows machine. Then, create a basic streaming application and explore your cluster in Redpanda Console.

Redpanda in Docker is supported only for development and testing. To deploy self-hosted Redpanda in production, use one of the following environments:

Prerequisites

You need Docker Compose. For installation instructions, see the Docker Compose documentation.

To check if you have Docker Compose installed, run the following:

docker compose version

You should see the version of Docker Compose that’s installed on your local machine.

Deploy Redpanda

For lightweight testing, you can start a single Redpanda broker, or you can use a more robust example with three Redpanda brokers. With three brokers, you can configure your topics with a replication factor of three so that the cluster can recover from a single-broker failure.

Single Broker
Three Brokers

Copy and paste the following YAML content into a file named docker-compose.yml on your local filesystem.

Reveal the YAML content

docker-compose.yml

version: "3.7"
name: redpanda-quickstart
networks:
  redpanda_network:
    driver: bridge
volumes:
  redpanda-0: null
services:
  redpanda-0:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
      # Address the broker advertises to clients that connect to the Kafka API.
      # Use the internal addresses to connect to the Redpanda brokers'
      # from inside the same Docker network.
      # Use the external addresses to connect to the Redpanda brokers'
      # from outside the Docker network.
      - --advertise-kafka-addr internal://redpanda-0:9092,external://localhost:19092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:18082
      # Address the broker advertises to clients that connect to the HTTP Proxy.
      - --advertise-pandaproxy-addr internal://redpanda-0:8082,external://localhost:18082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
      # Redpanda brokers use the RPC API to communicate with eachother internally.
      - --rpc-addr redpanda-0:33145
      - --advertise-rpc-addr redpanda-0:33145
      # Tells Seastar (the framework Redpanda uses under the hood) to use 1 core on the system.
      - --smp 1
      # The amount of memory to make available to Redpanda.
      - --memory 1G
      # Mode dev-container uses well-known configuration properties for development in containers.
      - --mode dev-container
      # enable logs for debugging.
      - --default-log-level=debug
    image: docker.redpanda.com/redpandadata/redpanda:v23.1.13
    container_name: redpanda-0
    volumes:
      - redpanda-0:/var/lib/redpanda/data
    networks:
      - redpanda_network
    ports:
      - 18081:18081
      - 18082:18082
      - 19092:19092
      - 19644:9644
  console:
    container_name: redpanda-console
    image: docker.redpanda.com/redpandadata/console:v2.5.1
    networks:
      - redpanda_network
    entrypoint: /bin/sh
    command: -c 'echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml; /app/console'
    environment:
      CONFIG_FILEPATH: /tmp/config.yml
      CONSOLE_CONFIG_FILE: |
        kafka:
          brokers: ["redpanda-0:9092"]
          schemaRegistry:
            enabled: true
            urls: ["http://redpanda-0:8081"]
        redpanda:
          adminApi:
            enabled: true
            urls: ["http://redpanda-0:9644"]
    ports:
      - 8080:8080
    depends_on:
      - redpanda-0

Run the following in the directory where you saved the docker-compose.yml file:

docker compose up -d

[+] Running 4/4
⠿ Network redpanda_network                 Created  0.0s
⠿ Volume "redpanda-quickstart_redpanda-0"  Created  0.0s
⠿ Container redpanda-0                     Started  0.3s
⠿ Container redpanda-console               Started  0.6s

Copy and paste the following YAML content into a file named docker-compose.yml on your local filesystem.

Reveal the YAML content

docker-compose.yml

version: "3.7"
name: redpanda-quickstart
networks:
  redpanda_network:
    driver: bridge
volumes:
  redpanda-0: null
  redpanda-1: null
  redpanda-2: null
services:
  redpanda-0:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
      # Address the broker advertises to clients that connect to the Kafka API.
      # Use the internal addresses to connect to the Redpanda brokers'
      # from inside the same Docker network.
      # Use the external addresses to connect to the Redpanda brokers'
      # from outside the Docker network.
      - --advertise-kafka-addr internal://redpanda-0:9092,external://localhost:19092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:18082
      # Address the broker advertises to clients that connect to the HTTP Proxy.
      - --advertise-pandaproxy-addr internal://redpanda-0:8082,external://localhost:18082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
      # Redpanda brokers use the RPC API to communicate with eachother internally.
      - --rpc-addr redpanda-0:33145
      - --advertise-rpc-addr redpanda-0:33145
      # Tells Seastar (the framework Redpanda uses under the hood) to use 1 core on the system.
      - --smp 1
      # The amount of memory to make available to Redpanda.
      - --memory 1G
      # Mode dev-container uses well-known configuration properties for development in containers.
      - --mode dev-container
      # enable logs for debugging.
      - --default-log-level=debug
    image: docker.redpanda.com/redpandadata/redpanda:v23.1.13
    container_name: redpanda-0
    volumes:
      - redpanda-0:/var/lib/redpanda/data
    networks:
      - redpanda_network
    ports:
      - 18081:18081
      - 18082:18082
      - 19092:19092
      - 19644:9644
  redpanda-1:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:29092
      - --advertise-kafka-addr internal://redpanda-1:9092,external://localhost:29092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:28082
      - --advertise-pandaproxy-addr internal://redpanda-1:8082,external://localhost:28082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:28081
      - --rpc-addr redpanda-1:33145
      - --advertise-rpc-addr redpanda-1:33145
      - --smp 1
      - --memory 1G
      - --mode dev-container
      - --default-log-level=debug
      - --seeds redpanda-0:33145
    image: docker.redpanda.com/redpandadata/redpanda:v23.1.13
    container_name: redpanda-1
    volumes:
      - redpanda-1:/var/lib/redpanda/data
    networks:
      - redpanda_network
    ports:
      - 28081:28081
      - 28082:28082
      - 29092:29092
      - 29644:9644
    depends_on:
      - redpanda-0
  redpanda-2:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:39092
      - --advertise-kafka-addr internal://redpanda-2:9092,external://localhost:39092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:38082
      - --advertise-pandaproxy-addr internal://redpanda-2:8082,external://localhost:38082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:38081
      - --rpc-addr redpanda-2:33145
      - --advertise-rpc-addr redpanda-2:33145
      - --smp 1
      - --memory 1G
      - --mode dev-container
      - --default-log-level=debug
      - --seeds redpanda-0:33145
    image: docker.redpanda.com/redpandadata/redpanda:v23.1.13
    container_name: redpanda-2
    volumes:
      - redpanda-2:/var/lib/redpanda/data
    networks:
      - redpanda_network
    ports:
      - 38081:38081
      - 38082:38082
      - 39092:39092
      - 39644:9644
    depends_on:
      - redpanda-0
  console:
    container_name: redpanda-console
    image: docker.redpanda.com/redpandadata/console:v2.5.1
    networks:
      - redpanda_network
    entrypoint: /bin/sh
    command: -c 'echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml; /app/console'
    environment:
      CONFIG_FILEPATH: /tmp/config.yml
      CONSOLE_CONFIG_FILE: |
        kafka:
          brokers: ["redpanda-0:9092"]
          schemaRegistry:
            enabled: true
            urls: ["http://redpanda-0:8081"]
        redpanda:
          adminApi:
            enabled: true
            urls: ["http://redpanda-0:9644"]
    ports:
      - 8080:8080
    depends_on:
      - redpanda-0

Run the following in the directory where you saved the docker-compose.yml file:

docker compose up -d

[+] Running 8/8
⠿ Network redpanda_network                  Created  0.0s
⠿ Volume "redpanda-quickstart_redpanda-0"   Created  0.0s
⠿ Volume "redpanda-quickstart_redpanda-1"   Created  0.0s
⠿ Volume "redpanda-quickstart_redpanda-2"   Created  0.0s
⠿ Container redpanda-0                      Started  0.3s
⠿ Container redpanda-console                Started  0.6s
⠿ Container redpanda-1                      Started  0.7s
⠿ Container redpanda-2                      Started  0.7s

Start streaming

To start building a basic streaming application, you can use rpk to create a topic, produce messages to it, and consume messages from it. rpk is a command-line tool for connecting to and interacting with Redpanda brokers.

Each Redpanda broker comes preinstalled with rpk, so you can use it inside one of the Redpanda broker’s Docker containers or you can use install it on your local machine to test external connectivity. The docker-compose.yml file configured the containers to expose ports on your localhost so that you can communicate with the cluster outside the Docker network.

To use rpk inside the Redpanda broker’s Docker container:

Get information about the cluster.

docker exec -it redpanda-0 rpk cluster info

Create a topic called chat-room:

docker exec -it redpanda-0 rpk topic create chat-room

Output:

TOPIC       STATUS
chat-room  OK

Produce a message to the topic:

docker exec -it redpanda-0 rpk topic produce chat-room

Enter a message, then press Enter:

Pandas are fabulous!

Example output:

Produced to partition 0 at offset 0 with timestamp 1663282629789.

Press Ctrl+C to finish producing messages to the topic.

Consume one message from the topic:

docker exec -it redpanda-0 rpk topic consume chat-room --num 1

Your message is displayed along with its metadata,:

{
  "topic": "chat-room",
  "value": "Pandas are fabulous!",
  "timestamp": 1663282629789,
  "partition": 0,
  "offset": 0
}

To test external connectivity using your local machine:

Install rpk on your local machine:

The rpk binary is not supported on Windows.
- Linux
- macOS
Download the rpk archive for Linux, and make sure the version matches your Redpanda version.

To download the latest version of rpk:

curl -LO \ https://github.com/redpanda-data/redpanda/releases/latest/download/rpk-linux-amd64.zip

To download a version other than the latest:

curl -LO \ https://github.com/redpanda-data/redpanda/releases/download/v<version>/rpk-linux-amd64.zip

Ensure that you have the folder ~/.local/bin:

mkdir -p ~/.local/bin

Add it to your $PATH:

export PATH="~/.local/bin:$PATH"

Unzip the rpk files to your ~/.local/bin/ directory:

unzip rpk-linux-amd64.zip -d ~/.local/bin/

Run rpk version to display the version of the rpk binary:

rpk version

23.1.13 (rev 9eefb907c)
If you don’t have Homebrew installed, install it.

Install rpk:

brew install redpanda-data/tap/redpanda

Run rpk version to display the version of the rpk binary:

rpk version

23.1.13 (rev 9eefb907c)

This method installs the latest version of rpk, which is supported only with the latest version of Redpanda.

Get information about the cluster to test the connection.

rpk cluster info --brokers 127.0.0.1:19092

The Redpanda broker returns a list of all broker addresses so that rpk can communicate with all brokers directly. Each broker returns its configured advertise-* address that matches the port to which the initial connection has been made.

Explore your topic

To explore your topic, you can use Redpanda Console, which you deployed in the docker-compose.yml file and is running locally on port 8080. Redpanda Console is a developer-friendly web UI for managing and debugging your Redpanda cluster and your applications.

Open Redpanda Console in a web browser.

All your Redpanda brokers are listed along with their IP addresses and IDs.
Go to Topics > chat-room.

The message that you produced to the topic is displayed along with some other details about the topic.

Clean up

If you don’t want to continue experimenting with your cluster, you can shut it down and delete the containers:

docker compose down

To delete the volumes along with all your cluster data, run the following:

docker compose down -v

Next steps

Continue learning about client application.
Learn more about Redpanda Console.
Learn more about rpk.
Deploy for development or production.
Try more Docker Compose examples.
Find the Docker images for Redpanda on Docker Hub.
Find the Docker images for Redpanda Console on Docker Hub.

What do you like about this doc?

Let us know what we do well:

Let us contact you about your feedback:

What did you not like about this doc?

Let us know what we can improve: