Redact Information in JSON Messages

This example shows you how to use Redpanda data transforms to redact information in JSON messages.

The demo runs using Docker Compose, with the following containers:

  • redpanda: Includes a single Redpanda broker.

  • console: Includes Redpanda Console.

  • owl-shop: Includes the Owlshop demo application that produces e-commerce data to the Redpanda broker.

  • transform: Includes all the requirements for deploying the transforms function to the Redpanda broker, including rpk, Go, and the redaction transform code. The source code for the redaction transform is available in the redaction directory.

Prerequisites

You must have Docker and Docker Compose installed on your host machine.

Run the lab

  1. Clone the repository:

    git clone https://github.com/redpanda-data/redpanda-labs.git

  2. Change into the data-transforms/redaction/demo/ directory:

    cd redpanda-labs/data-transforms/go/redaction/demo

  3. Set the REDPANDA_VERSION environment variable to at least version 23.3.1. Data transforms was introduced in this version. For all available versions, see the GitHub releases.

    For example:

    export REDPANDA_VERSION=24.2.10

  4. Set the REDPANDA_CONSOLE_VERSION environment variable to the version of Redpanda Console that you want to run. For all available versions, see the GitHub releases.

    For example:

    export REDPANDA_CONSOLE_VERSION=2.7.2

  5. Build the data transforms container:

    docker compose build

  6. If you are running on an ARM-based device such as the Apple M1 chip, open the docker-compose.yml file and uncomment the platform: 'linux/amd64' line.

  7. Start the containers:

    docker compose up --detach --wait

  8. Navigate to http://localhost:8080 to see the Redpanda Console.

  9. Go to Topics and select owlshop-orders-redacted and see the redacted orders.

Clean up

To stop the containers:

docker compose down