23.2 23.1 22.3 22.2 22.1 21.11

Docker Compose Samples for Redpanda

Use these Docker Compose samples to test Redpanda features and applications during development.

Redpanda in Docker is supported only for development and testing.

To deploy Redpanda in production, use one of the following environments:

Prerequisites

To run docker compose you must install Docker on your machine. See the Docker documentation.

Owl shop sample application

This docker-compose.yml file starts a single Redpanda broker, Redpanda Console, and a sample application called owl shop. Owl shop simulates a simple ecommerce shop that uses Redpanda as an asynchronous communication exchange. Owl shop creates topics, produces sample data to those topics, and consumes from those topics.

Paste the following YAML into a docker-compose.yml file on your computer, and run docker compose up -d inside the same directory.

If you are running on an ARM-based device such as the Apple M1 chip, uncomment the platform: 'linux/amd64' lines.
docker-compose.yml
version: '3.7'
name: redpanda-owl-shop
networks:
  redpanda_network:
    driver: bridge
volumes:
  redpanda: null
services:
  redpanda:
    image: docker.redpanda.com/redpandadata/redpanda:v22.3.11
    command:
      - redpanda start
      - --smp 1
      - --overprovisioned
      - --kafka-addr PLAINTEXT://0.0.0.0:29092,OUTSIDE://0.0.0.0:9092
      - --advertise-kafka-addr PLAINTEXT://redpanda:29092,OUTSIDE://localhost:9092
      - --pandaproxy-addr 0.0.0.0:8082
      - --advertise-pandaproxy-addr localhost:8082
    ports:
      - 8081:8081
      - 8082:8082
      - 9092:9092
      - 9644:9644
      - 29092:29092
    volumes:
      - redpanda:/var/lib/redpanda/data
    networks:
      - redpanda_network

console:
    image: docker.redpanda.com/redpandadata/console:v2.3.1
    entrypoint: /bin/sh
    command: -c "echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml; /app/console"
    environment:
      CONFIG_FILEPATH: /tmp/config.yml
      CONSOLE_CONFIG_FILE: |
        kafka:
          brokers: ["redpanda:29092"]
          schemaRegistry:
            enabled: true
            urls: ["http://redpanda:8081"]
        redpanda:
          adminApi:
            enabled: true
            urls: ["http://redpanda:9644"]
        connect:
          enabled: true
          clusters:
            - name: local-connect-cluster
              url: http://connect:8083
    ports:
      - 8080:8080
    networks:
      - redpanda_network
    depends_on:
      - redpanda

owl-shop:
    image: quay.io/cloudhut/owl-shop:latest
    networks:
      - redpanda_network
    #platform: 'linux/amd64'
    environment:
      - SHOP_KAFKA_BROKERS=redpanda:29092
      - SHOP_KAFKA_TOPICREPLICATIONFACTOR=1
      - SHOP_TRAFFIC_INTERVAL_RATE=1
      - SHOP_TRAFFIC_INTERVAL_DURATION=0.1s
    depends_on:
      - redpanda

connect:
    image: docker.redpanda.com/redpandadata/connectors:latest
    hostname: connect
    container_name: connect
    networks:
      - redpanda_network
    #platform: 'linux/amd64'
    depends_on:
      - redpanda
    ports:
      - "8083:8083"
    environment:
      CONNECT_CONFIGURATION: |
          key.converter=org.apache.kafka.connect.converters.ByteArrayConverter
          value.converter=org.apache.kafka.connect.converters.ByteArrayConverter
          group.id=connectors-cluster
          offset.storage.topic=_internal_connectors_offsets
          config.storage.topic=_internal_connectors_configs
          status.storage.topic=_internal_connectors_status
          config.storage.replication.factor=-1
          offset.storage.replication.factor=-1
          status.storage.replication.factor=-1
          offset.flush.interval.ms=1000
          producer.linger.ms=50
          producer.batch.size=131072
      CONNECT_BOOTSTRAP_SERVERS: redpanda:29092
      CONNECT_GC_LOG_ENABLED: "false"
      CONNECT_HEAP_OPTS: -Xms512M -Xmx512M
      CONNECT_LOG_LEVEL: info

Redpanda Enterprise Edition

This docker-compose.yml file configures a Redpanda broker and Redpanda Console with a license key for Redpanda Enterprise Edition. The license key is uploaded to the Redpanda broker through a separate container called redpandarpk. This container executes the rpk cluster license set command to load the license key from the given filepath.

To request a trial license, to extend your trial period, or to purchase an Enterprise Edition license, contact Redpanda Sales.

  1. Save your license key to a file called redpanda.license in a directory called license, or update the redpandarpk.volumes host path in the docker-compose.yml file to another directory that contains your license key.

  2. Paste the following YAML into a docker-compose.yml file on your computer, and run docker compose up -d inside the same directory.

    docker-compose.yml
    ---
version: "3.7"
name: redpanda-enterprise
networks:
  redpanda_network:
    driver: bridge
volumes:
  redpanda: null
services:
  redpanda:
    image: docker.redpanda.com/redpandadata/redpanda:v22.3.11
    command:
      - redpanda start
      - --smp 1
      - --overprovisioned
      - --kafka-addr PLAINTEXT://0.0.0.0:29092,OUTSIDE://0.0.0.0:9092
      - --advertise-kafka-addr PLAINTEXT://redpanda:29092,OUTSIDE://localhost:9092
      - --pandaproxy-addr 0.0.0.0:8082
      - --advertise-pandaproxy-addr localhost:8082
    ports:
      - 8081:8081
      - 8082:8082
      - 9092:9092
      - 9644:9644
      - 29092:29092
    volumes:
      - redpanda:/var/lib/redpanda/data
    networks:
      - redpanda_network

  redpandarpk:
    command:
      - cluster
      - license
      - set
      - --path
      # this is the default location in which rpk searches for the redpanda.license file.
      # if you mount the license key file to a different location, update this path.
      - /etc/redpanda/redpanda.license
      # rpk connects to the admin API of one broker to set the license key for the whole cluster.
      - --api-urls redpanda:9644
    image: docker.redpanda.com/redpandadata/redpanda:v22.3.11
    # mount the local directory that contains your license key to the container.
    # give Redpanda read and write access to the license.
    volumes:
      - ./license:/etc/redpanda:rw
    networks:
      - redpanda_network
    depends_on:
      - redpanda

  console:
    container_name: redpanda-console
    image: docker.redpanda.com/redpandadata/console:v2.3.1
    # mount the local directory that contains your license key to the container.
    # give Redpanda Console read access to the license.
    volumes:
      - ./license:/etc/redpanda:ro
    networks:
      - redpanda_network
    entrypoint: /bin/sh
    command: -c 'echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml; /app/console'
    environment:
      REDPANDA_LICENSE_FILEPATH: /etc/redpanda/redpanda.license
      CONFIG_FILEPATH: /tmp/config.yml
      CONSOLE_CONFIG_FILE: |
        kafka:
          brokers: ["redpanda:9092"]
          schemaRegistry:
            enabled: true
            urls: ["http://redpanda:8081"]
        redpanda:
          adminApi:
            enabled: true
            urls: ["http://redpanda:9644"]
    ports:
      - 8080:8080
    depends_on:
      - redpanda

  3. Verify that the redpandarpk container successfully uploaded your license key to the Redpanda cluster.

    docker logs redpanda-enterprise-redpandarpk-1

Successfully uploaded license.

Customize the samples

If you want to configure Redpanda or Redpanda Console, you need to customize the sample YAML files.

Configure Redpanda in Docker

To configure the Redpanda services with node configuration properties, you can do the following:

  • Pass configuration properties to the --set option in the redpanda start command. For example:

    redpanda-0:
  command:
    - redpanda
    - start
    - --set pandaproxy_client.retries=6

  • Create a redpanda.yaml file and mount it to the /etc/redpanda/ directory on the redpanda containers. For example, if you create the redpanda.yaml file in a directory called redpanda-mounts, configure the following volume mounts:

    redpanda-0:
  volumes:
    - ./redpanda-mounts:/etc/redpanda/

Configure Redpanda Console in Docker

To configure the Redpanda Console service with configuration properties, you can do the following:

  • Use environment variables, for example:

    console:
  environment:
    KAFKA_RACKID: rack1

  • Create a redpanda-console-config.yaml file and mount it to the /etc/redpanda/ directory on the redpanda-console container. For example, if you create the redpanda-console-config.yaml file in a directory called console-mounts, configure the following volume mounts:

    console:
  volumes:
    - ./console-mounts:/etc/redpanda/