Docker Compose Samples for Redpanda

Use these Docker Compose samples to test Redpanda features and applications during development.

Redpanda in Docker is supported only for development and testing.

To deploy Redpanda in production, use one of the following environments:

Prerequisites

To run docker compose you must install Docker on your machine. See the Docker documentation.

Owl shop sample application

This docker-compose.yml file starts a single Redpanda broker, Redpanda Console, and a sample application called owl shop. Owl shop simulates a simple ecommerce shop that uses Redpanda as an asynchronous communication exchange. Owl shop creates topics, produces sample data to those topics, and consumes from those topics.

Paste the following YAML into a docker-compose.yml file on your computer, and run docker compose up -d inside the same directory.

If you are running on an ARM-based device such as the Apple M1 chip, uncomment the platform: 'linux/amd64' lines.

Reveal the docker-compose.yml

docker-compose.yml

version: '3.7'
name: redpanda-owl-shop
networks:
  redpanda_network:
    driver: bridge
volumes:
  redpanda: null
services:
  redpanda:
    image: docker.redpanda.com/redpandadata/redpanda:v23.2.21
    command:
      - redpanda start
      - --smp 1
      - --overprovisioned
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
      # Address the broker advertises to clients that connect to the Kafka API.
      # Use the internal addresses to connect to the Redpanda brokers
      # from inside the same Docker network.
      # Use the external addresses to connect to the Redpanda brokers
      # from outside the Docker network.
      - --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:18082
      # Address the broker advertises to clients that connect to the HTTP Proxy.
      - --advertise-pandaproxy-addr internal://redpanda:8082,external://localhost:18082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
      # Redpanda brokers use the RPC API to communicate with each other internally.
      - --rpc-addr redpanda:33145
      - --advertise-rpc-addr redpanda:33145
      - --mode dev-container
    ports:
      - 18081:18081
      - 18082:18082
      - 19092:19092
      - 19644:9644
    volumes:
      - redpanda:/var/lib/redpanda/data
    networks:
      - redpanda_network
    healthcheck:
      test: ["CMD-SHELL", "rpk cluster health | grep -E 'Healthy:.+true' || exit 1"]
      interval: 15s
      timeout: 3s
      retries: 5
      start_period: 5s
  console:
    image: docker.redpanda.com/redpandadata/console:v2.7.2
    entrypoint: /bin/sh
    command: -c "echo \"$$CONSOLE_CONFIG_FILE\" > /tmp/config.yml; /app/console"
    environment:
      CONFIG_FILEPATH: /tmp/config.yml
      CONSOLE_CONFIG_FILE: |
        kafka:
          brokers: ["redpanda:9092"]
          schemaRegistry:
            enabled: true
            urls: ["http://redpanda:8081"]
        redpanda:
          adminApi:
            enabled: true
            urls: ["http://redpanda:9644"]
        connect:
          enabled: true
          clusters:
            - name: local-connect-cluster
              url: http://connect:8083
    ports:
      - 8080:8080
    networks:
      - redpanda_network
    depends_on:
      - redpanda
  owl-shop:
    image: quay.io/cloudhut/owl-shop:sha-042112b
    networks:
      - redpanda_network
    # platform: 'linux/amd64'
    entrypoint: /bin/sh
    command: -c "echo \"$$OWLSHOP_CONFIG_FILE\" > /tmp/config.yml; /app/owlshop"
    environment:
      CONFIG_FILEPATH: /tmp/config.yml
      OWLSHOP_CONFIG_FILE: |
        shop:
          requestRate: 1
          interval: 0.1s
          topicReplicationFactor: 1
          topicPartitionCount: 1
        kafka:
          brokers: "redpanda:9092"
    depends_on:
      - redpanda
  connect:
    image: docker.redpanda.com/redpandadata/connectors:latest
    hostname: connect
    container_name: connect
    networks:
      - redpanda_network
    # platform: 'linux/amd64'
    depends_on:
      - redpanda
    ports:
      - "8083:8083"
    environment:
      CONNECT_CONFIGURATION: |
          key.converter=org.apache.kafka.connect.converters.ByteArrayConverter
          value.converter=org.apache.kafka.connect.converters.ByteArrayConverter
          group.id=connectors-cluster
          offset.storage.topic=_internal_connectors_offsets
          config.storage.topic=_internal_connectors_configs
          status.storage.topic=_internal_connectors_status
          config.storage.replication.factor=-1
          offset.storage.replication.factor=-1
          status.storage.replication.factor=-1
          offset.flush.interval.ms=1000
          producer.linger.ms=50
          producer.batch.size=131072
      CONNECT_BOOTSTRAP_SERVERS: redpanda:9092
      CONNECT_GC_LOG_ENABLED: "false"
      CONNECT_HEAP_OPTS: -Xms512M -Xmx512M
      CONNECT_LOG_LEVEL: info

Open Redpanda Console at localhost:8080 and go to Topics to see the Owl Shop topics.

Samples for Redpanda Enterprise Edition

This section contains docker-compose.yml files with features that require a license key for Redpanda Enterprise Edition. The license key is uploaded to the Redpanda broker through a separate container called redpandarpk. This container executes the rpk cluster license set command to load the license key from the given filepath.

To request a trial license, to extend your trial period, or to purchase an Enterprise Edition license,
contact Redpanda Sales.

In the same location as your docker-compose.yml file, save your license key to a file called redpanda.license in a directory called license. Or, to use another location, update the license paths in the docker-compose.yml files to another directory that contains your license key.

This docker-compose.yml file configures Redpanda Console with a plain login provider so that users of Redpanda Console must log in with a username and password.

Paste the following YAML into a docker-compose.yml file on your computer, and run docker compose up -d inside the same directory.

Reveal the docker-compose.yml

docker-compose.yml

---
version: "3.7"
name: redpanda-enterprise
networks:
  redpanda_network:
    driver: bridge
volumes:
  redpanda: null
services:
  redpanda:
    image: docker.redpanda.com/redpandadata/redpanda:v23.2.21
    command:
      - redpanda start
      - --smp 1
      - --overprovisioned
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
      # Address the broker advertises to clients that connect to the Kafka API.
      # Use the internal addresses to connect to the Redpanda brokers
      # from inside the same Docker network.
      # Use the external addresses to connect to the Redpanda brokers
      # from outside the Docker network.
      - --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:18082
      # Address the broker advertises to clients that connect to the HTTP Proxy.
      - --advertise-pandaproxy-addr internal://redpanda:8082,external://localhost:18082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
      # Redpanda brokers use the RPC API to communicate with each other internally.
      - --rpc-addr redpanda:33145
      - --advertise-rpc-addr redpanda:33145
      - --mode dev-container
    ports:
      - 18081:18081
      - 18082:18082
      - 19092:19092
      - 19644:9644
    volumes:
      - redpanda:/var/lib/redpanda/data
    networks:
      - redpanda_network
  redpandarpk:
    command:
      - cluster
      - license
      - set
      - --path
      # this is the default location in which rpk searches for the redpanda.license file.
      # if you mount the license key file to a different location, update this path.
      - /etc/redpanda/redpanda.license
      # rpk connects to the admin API of one broker to set the license key for the whole cluster.
      - --api-urls redpanda:9644
    image: docker.redpanda.com/redpandadata/redpanda:v23.2.21
    # mount the local directory that contains your license key to the container.
    # give Redpanda read and write access to the license.
    volumes:
      - ./license:/etc/redpanda:rw
    networks:
      - redpanda_network
    depends_on:
      - redpanda
  console:
    container_name: redpanda-console
    image: docker.redpanda.com/redpandadata/console:v2.7.2
    # mount the local directory that contains your license key to the container.
    # give Redpanda Console read access to the license.
    volumes:
      - ./license:/etc/redpanda:ro
    networks:
      - redpanda_network
    entrypoint: /bin/sh
    command: -c 'echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml && echo "$$CONSOLE_ROLEBINDINGS_CONFIG_FILE" > /tmp/role-bindings.yml && /app/console'
    environment:
      REDPANDA_LICENSE_FILEPATH: /etc/redpanda/redpanda.license
      CONFIG_FILEPATH: /tmp/config.yml
      CONSOLE_CONFIG_FILE: |
        kafka:
          brokers: ["redpanda:9092"]
          schemaRegistry:
            enabled: true
            urls: ["http://redpanda:8081"]
        redpanda:
          adminApi:
            enabled: true
            urls: ["http://redpanda:9644"]
        login:
          enabled: true
          jwtSecret: change-this-to-something-secret
          useSecureCookies: false
          plain:
            enabled: true
            credentials:
            - username: "jane"
              password: "some-other-secret-password"
            - username: "john"
              password: "some-secret-password"
        enterprise:
          rbac:
            enabled: true
            roleBindingsFilepath: /tmp/role-bindings.yml
      CONSOLE_ROLEBINDINGS_CONFIG_FILE: |
        roleBindings:
        - metadata:
            name: Platform Ops
          subjects:
            - kind: user
              provider: Plain
              name: jane
          roleName: admin
        - metadata:
            name: Software Engineers
          subjects:
            - kind: user
              provider: Plain
              name: john
          roleName: editor
    ports:
      - 8080:8080
    depends_on:
      - redpanda

Open Redpanda Console at localhost:8080 and log in with the credentials for john or jane.

Customize the samples

If you want to configure Redpanda or Redpanda Console, you need to customize the sample YAML files.

Configure Redpanda in Docker

To configure the Redpanda services with node configuration properties, you can do the following:

Pass configuration properties to the --set option in the redpanda start command. For example:
```
redpanda:
  command:
    - redpanda
    - start
    - --set pandaproxy_client.retries=6
```
Create a redpanda.yaml file and mount it to the /etc/redpanda/ directory on the redpanda containers. For example, if you create the redpanda.yaml file in a directory called redpanda-mounts, configure the following volume mounts:
```
redpanda:
  volumes:
    - ./redpanda-mounts:/etc/redpanda/
```

Configure Redpanda Console in Docker

To configure the Redpanda Console service with configuration properties, you can do the following:

Use environment variables, for example:

console:
  environment:
    KAFKA_RACKID: rack1

Create a redpanda-console-config.yaml file and mount it to the /etc/redpanda/ directory on the redpanda-console container. For example, if you create the redpanda-console-config.yaml file in a directory called console-mounts, configure the following volume mounts:
```
console:
  volumes:
    - ./console-mounts:/etc/redpanda/
```

Was this helpful?

group Ask in the community

mail Share your feedback

group_add Make a contribution

What do you like about this doc?

Let us know what we do well:

Let us contact you about your feedback:

What did you not like about this doc?

Let us know what we can improve: