Redpanda Console Quickstart

This quickstart introduces Redpanda Console, a powerful web UI for managing and monitoring your Redpanda clusters. Explore practical examples and scenarios to help you understand how to leverage Redpanda Console for different use cases, including data observability, Redpanda management, access control, and connectivity.

This quickstart uses Docker Compose to deploy Redpanda, Redpanda Console, and Redpanda Connect. For instructions on deploying Redpanda Console in Linux or Kubernetes, see:

Prerequisites

You need Docker Compose to complete this guide. For installation instructions, see the Docker Compose documentation.

To check if you have Docker Compose installed, run:

docker compose version

You should see the version of Docker Compose that’s installed on your local machine.

Deploy Redpanda Console

Download the following Docker Compose file on your local file system.

This Docker Compose file deploys a Redpanda Self-Managed cluster, Redpanda Console, and Redpanda Connect. Redpanda Connect is used to generate demo data and produce it to a topic on the Redpanda cluster.

Reveal the YAML content

docker-compose.yml

name: redpanda-console-quickstart
networks:
  redpanda_network:
    driver: bridge
volumes:
  redpanda-0: null
  redpanda-1: null
  redpanda-2: null
services:
  ##################
  # Redpanda Brokers #
  ##################
  redpanda-0:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
      # Address the broker advertises to clients that connect to the Kafka API.
      # Use the internal addresses to connect to the Redpanda brokers'
      # from inside the same Docker network.
      # Use the external addresses to connect to the Redpanda brokers'
      # from outside the Docker network.
      - --advertise-kafka-addr internal://redpanda-0:9092,external://localhost:19092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:18082
      # Address the broker advertises to clients that connect to the HTTP Proxy.
      - --advertise-pandaproxy-addr internal://redpanda-0:8082,external://localhost:18082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
      # Redpanda brokers use the RPC API to communicate with each other internally.
      - --rpc-addr redpanda-0:33145
      - --advertise-rpc-addr redpanda-0:33145
      # Mode dev-container uses well-known configuration properties for development in containers.
      - --mode dev-container
      # Tells Seastar (the framework Redpanda uses under the hood) to use 1 core on the system.
      - --smp 1
      - --default-log-level=info
    image: docker.redpanda.com/redpandadata/redpanda-unstable:v24.3.1-rc6
    container_name: redpanda-0
    # Sets the username and password of the SCRAM superuser
    environment:
      RP_BOOTSTRAP_USER: "superuser:secretpassword"
    volumes:
      - redpanda-0:/var/lib/redpanda/data
      - ./bootstrap.yml:/etc/redpanda/.bootstrap.yaml
    networks:
      - redpanda_network
    ports:
      - 18081:18081
      - 18082:18082
      - 19092:19092
      - 19644:9644
    healthcheck:
      test: ["CMD", "rpk", "cluster", "info", "-X", "user=superuser", "-X", "pass=secretpassword"]
      interval: 10s
      timeout: 5s
      retries: 5
  redpanda-1:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:29092
      - --advertise-kafka-addr internal://redpanda-1:9092,external://localhost:29092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:28082
      - --advertise-pandaproxy-addr internal://redpanda-1:8082,external://localhost:28082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:28081
      - --rpc-addr redpanda-1:33145
      - --advertise-rpc-addr redpanda-1:33145
      - --mode dev-container
      - --smp 1
      - --default-log-level=info
      - --seeds redpanda-0:33145
    image: docker.redpanda.com/redpandadata/redpanda-unstable:v24.3.1-rc6
    container_name: redpanda-1
    environment:
      RP_BOOTSTRAP_USER: "superuser:secretpassword"
    volumes:
      - redpanda-1:/var/lib/redpanda/data
      - ./bootstrap.yml:/etc/redpanda/.bootstrap.yaml
    networks:
      - redpanda_network
    ports:
      - 28081:28081
      - 28082:28082
      - 29092:29092
      - 29644:9644
    depends_on:
      - redpanda-0
  redpanda-2:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:39092
      - --advertise-kafka-addr internal://redpanda-2:9092,external://localhost:39092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:38082
      - --advertise-pandaproxy-addr internal://redpanda-2:8082,external://localhost:38082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:38081
      - --rpc-addr redpanda-2:33145
      - --advertise-rpc-addr redpanda-2:33145
      - --mode dev-container
      - --smp 1
      - --default-log-level=info
      - --seeds redpanda-0:33145
    image: docker.redpanda.com/redpandadata/redpanda-unstable:v24.3.1-rc6
    container_name: redpanda-2
    environment:
      RP_BOOTSTRAP_USER: "superuser:secretpassword"
    volumes:
      - redpanda-2:/var/lib/redpanda/data
      - ./bootstrap.yml:/etc/redpanda/.bootstrap.yaml
    networks:
      - redpanda_network
    ports:
      - 38081:38081
      - 38082:38082
      - 39092:39092
      - 39644:9644
    depends_on:
      - redpanda-0
  ####################
  # Redpanda Console #
  ####################
  console:
    container_name: redpanda-console
    image: docker.redpanda.com/redpandadata/console:v2.8.0-beta.1
    networks:
      - redpanda_network
    entrypoint: /bin/sh
    command: -c 'echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml; /app/console -config.filepath=${CONFIG_FILEPATH:-/tmp/config.yml}'
    volumes:
      - ./config:/tmp/config/
    environment:
      CONFIG_FILEPATH: ${CONFIG_FILEPATH:-/tmp/config.yml}
      CONSOLE_CONFIG_FILE: |
        kafka:
          brokers: ["redpanda-0:9092"]
          schemaRegistry:
            enabled: true
            urls: ["http://redpanda-0:8081"]
          sasl:
            enabled: true
            username: superuser
            password: secretpassword
            mechanism: SCRAM-SHA-256
        redpanda:
          adminApi:
            enabled: true
            urls: ["http://redpanda-0:9644"]
        console:
          topicDocumentation:
            enabled: true
            git:
              enabled: true
              repository:
                url: https://github.com/redpanda-data/docs
                branch: main
                baseDirectory: modules/console/test-resources
    ports:
      - 8080:8080
    depends_on:
      - redpanda-0
  ####################
  # Redpanda Connect #
  ####################
  connect:
    container_name: redpanda-connect
    image: docker.redpanda.com/redpandadata/connect
    networks:
      - redpanda_network
    entrypoint: /bin/sh
    depends_on:
      - redpanda-0
    command: -c 'echo "$$CONNECT_CFG_FILE" > /tmp/connect.yml; /redpanda-connect -c /tmp/connect.yml'
    environment:
      # This is a Redpanda Connect configuration that creates fake data,
      # process it, and it writes the output to a set of topics randomly.
      #
      # Input:
      #  - Uses Redpanda Connect's generate input to generate fake data.
      # Pipeline:
      #  - Bloblang mapping to batch each input and map 1 message to 'logins'
      #    topic, and a random number (1-3) of messages to 'transaction' topic
      #  - Unarchive processor to parse the JSON array and extract each
      #    element into its own message.
      # Output:
      #  - kafka_franz output to write the messages to the Redpanda brokers.
      CONNECT_CFG_FILE: |
        input:
          generate:
            interval: 1s
            mapping: |
              let first_name = fake("first_name")
              let last_name  = fake("last_name")

              root.user_id    = counter()
              root.name       = $$first_name + " " + $$last_name
              root.email      = ($$first_name.slice(0,1) + $$last_name + "@" + fake("domain_name")).lowercase()
              root.login_time = now()
        pipeline:
          processors:
            - mapping: |
                root = range(0, random_int(min:2, max:4)).map_each(cust -> this)
            - unarchive:
                format: "json_array"
            - mapping: |
                if batch_index() == 0 {
                  meta topic = "logins"
                  root = this
                } else {
                  meta topic       = "transactions"
                  root.user_id     = this.user_id
                  root.email       = this.email
                  root.index       = batch_index() - 1
                  root.product_url = fake("url")
                  root.price       = fake("amount_with_currency")
                  root.timestamp   = now()
                }
        output:
          kafka_franz:
            seed_brokers: [ "redpanda-0:9092" ]
            topic: $${! metadata("topic") }
            sasl:
              - mechanism: SCRAM-SHA-256
                password: secretpassword
                username: superuser
  ####################
  # rpk #
  ####################
  redpandarpk:
    command:
      - registry
      - schema
      - create
      - transactions
      - --schema
      - /etc/redpanda/transactions-schema.json
      - -X user=superuser
      - -X pass=secretpassword
      - -X registry.hosts=redpanda-0:8081
    image: docker.redpanda.com/redpandadata/redpanda-unstable:v24.3.1-rc6
    # mount the local directory that contains your schema to the container.
    volumes:
      - ./transactions-schema.json:/etc/redpanda/transactions-schema.json
    networks:
      - redpanda_network
    depends_on:
      redpanda-0:
        condition: service_healthy

Download the following Redpanda bootstrap file on your local file system and save it in the same directory as the Docker Compose file.

This file contains configuration properties for the Redpanda brokers to apply on startup.
Reveal the YAML content
bootstrap.yml
# Enable SASL authentication for the Kafka API # Allow topics to be created on first access superusers: - superuser enable_sasl: true auto_create_topics_enabled: true data_transforms_enabled: true

Download the following JSON schema on your local file system and save it in the same directory as the Docker Compose file.

This file contains an example schema that the redpandarpk Docker Compose service uploads to your cluster so that you can explore the Schema Registry page in Redpanda Console.

Reveal the YAML content

transactions-schema.json

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Transactions",
  "type": "object",
  "properties": {
    "email": {
      "type": "string",
      "format": "email",
      "description": "The email address of the user involved in the transaction."
    },
    "index": {
      "type": "integer",
      "description": "A numeric index associated with the transaction."
    },
    "price": {
      "type": "string",
      "pattern": "^XXX \\d+\\.\\d{6}$",
      "description": "A string representing the price of the product, including a currency code followed by the amount."
    },
    "product_url": {
      "type": "string",
      "format": "uri",
      "description": "A URL that points to the product involved in the transaction."
    },
    "timestamp": {
      "type": "string",
      "format": "date-time",
      "description": "The timestamp of when the transaction occurred, formatted in ISO 8601."
    },
    "user_id": {
      "type": "integer",
      "description": "A numeric identifier for the user."
    }
  },
  "required": ["email", "index", "price", "product_url", "timestamp", "user_id"],
  "additionalProperties": false
}

Run the following in the directory where you saved the docker-compose.yml file:

docker compose up -d --wait

[+] Running 9/10
 ✔ Network redpanda-console-quickstart_redpanda_network  Created           0.0s
 ✔ Volume "redpanda-console-quickstart_redpanda-1"       Created           0.0s
 ✔ Volume "redpanda-console-quickstart_redpanda-2"       Created           0.0s
 ✔ Volume "redpanda-console-quickstart_redpanda-0"       Created           0.0s
 ✔ Container redpanda-0                                  Healthy          11.5s
 ⠇ Container redpanda-console-quickstart-redpandarpk-1   Waiting          11.5s
 ✔ Container redpanda-1                                  Healthy          11.4s
 ✔ Container redpanda-console                            Healthy          11.4s
 ✔ Container redpanda-connect                            Healthy          11.4s
 ✔ Container redpanda-2                                  Healthy          11.4s
container redpanda-console-quickstart-redpandarpk-1 exited (0)

This command deploys Redpanda and Redpanda Console, which is exposed on port 8080 and connected to your Redpanda cluster at localhost:9092.

Access Redpanda Console

When Redpanda Console is running, open your web browser and go to http://localhost:8080 to see an overview of your cluster’s status, health, and broker details.

To view details about a specific broker, click View at the end of the row in the Broker Details table.

View topics and filter messages

If you’re working with a particular topic, you can view its configuration, inspect its messages, and see which consumers are connected.

You can filter messages using several methods:

Text search: Search for specific text within the message.
Partition offset filters: Set the start and end offsets to narrow down the message range.
JavaScript filters: Use JavaScript function bodies to create complex filters that run server-side.

Suppose you’re asked to find all transactions related to the .edu domain. You can use a JavaScript filter to display only messages that include email address in that domain.

In the left menu, click Topics.
Click on the transactions topic. When the topic’s page opens, the messages tab is selected by default.
Click Add filter > JavaScript Filter.
Give your filter a name such as 'Find .edu domains'.
Replace the default JavaScript code with the following:
```
return value.email.includes(".edu");
```
Click Save to apply the filter.

You should see only messages that include the specific domain.

By default, the filter runs on the newest 50 messages. You can run the filter on older messages by changing the start offset. For example, to start from the oldest messages, select Oldest.

Schema Registry

In the Schema Registry menu, you can see an overview of your schemas as well as create, manage, and inspect your schemas without leaving Redpanda Console.

For more details, see Use Schema Registry in Redpanda Console.

Manage consumer groups

In the Consumer Groups menu, you can see detailed insights into consumer groups, including their configuration and status. You can:

See which members are assigned to partitions.
Monitor consumer lag to ensure your consumers are keeping up with the data stream.

If a consumer group falls behind, you can check the lag directly in Redpanda Console and, if necessary, reset the offsets to reprocess messages from a specific point.

Configure access control for Redpanda

Managing access control lists (ACLs) in Redpanda can be complex, but Redpanda Console simplifies this with an intuitive interface.

In the Security menu, you can:

Create, view, and edit ACLs for topics, consumer groups, transactions and the entire cluster.
Quickly understand what each principal (user) is authorized to do.

Suppose you’re onboarding a new team member who needs access to specific topics and consumer groups. You can use Redpanda Console to ensure users have the right permissions to perform their tasks.

In the left menu, click Security.
In the Users tab, click Create user.
Enter "Sasha" in the username field.
Enter "secretpassword" in the password field.
Click Create.

You now have a new user called Sasha. But this user has no permissions yet.

In the ACLs tab, click Create ACL.
Enter "Sasha" in the user/principal field.
In the topic selector dropdown, select Literal and enter "transactions" in the input field.
Under Operations, click the All dropdown and select Allow.
Scroll down and click OK.

Now Sasha has full access only to the topic called transactions.

To test these permissions, you can use a Kafka client library or the [Redpanda CLI (rpk)] to connect to Redpanda as the user Sasha.

Install rpk on your local machine.

Try to access the logins topic as Sasha:

rpk topic describe logins \
  -X user=Sasha \
  -X pass=secretpassword \
  -X sasl.mechanism=SCRAM-SHA-256 \
  -X brokers=localhost:19092

You are not authorized to view the topic.

SUMMARY
=======
NAME        logins
PARTITIONS  0
ERROR       TOPIC_AUTHORIZATION_FAILED: Not authorized to access topics: [Topic authorization failed.]

CONFIGS
=======
config response contained error: TOPIC_AUTHORIZATION_FAILED: Not authorized to access topics: [Topic authorization failed.]

Access the transactions topic.

rpk topic describe transactions \
  -X user=Sasha \
  -X pass=secretpassword \
  -X sasl.mechanism=SCRAM-SHA-256 \
  -X brokers=localhost:19092

You have access to this topic.

Monitor client quotas

Redpanda provides configurable throughput quotas that apply to an individual client or a group of clients. You can view quotas for an individual client based on an exact match with its client_id, or a group of clients based on IDs that start with a given prefix.

Connectors

In the Connectors menu, you can manage your Kafka Connect clusters or view instructions for deploying Redpanda Connect.

Data transforms

Data transforms let you run common data streaming tasks, like filtering, scrubbing, and transcoding, on the Redpanda broker. For example, you may have consumers that require you to redact credit card numbers or convert JSON to Avro. Data transforms can also interact with the Redpanda Schema Registry to work with encoded data types.

You can use Redpanda Console to manage and monitor your transform functions.

Clean up

If you don’t want to continue experimenting with Redpanda, you can shut it down and delete the containers:

docker compose down

To delete the volumes along with all your cluster data:

docker compose down -v

Next steps

Learn more about configuration options in Redpanda Console.

Suggested labs

Enable Plain Login Authentication for Redpanda Console

Search all labs

Was this helpful?

group Ask in the community

mail Share your feedback

group_add Make a contribution

What do you like about this doc?

Let us know what we do well:

Let us contact you about your feedback:

What did you not like about this doc?

Let us know what we can improve: