Redpanda Connect MCP Server Quickstart

This quickstart shows you how to build a Redpanda Connect configuration that searches Bluesky posts using a customizable query. You’ll expose this configuration to Claude Code as a tool through an MCP server, then ask Claude to perform the search using natural language.

Find recent posts on Bluesky mentioning Redpanda.

The MCP server makes your local tools AI-consumable. It exposes them with metadata that Claude can understand and use to:

  • Discover available tools

  • Understand required inputs and output format

  • Make intelligent, structured HTTP requests

This enables natural-language automation for internal workflows, using your own business logic.

Prerequisites

You’ll need the following to complete this quickstart:

Initialize an MCP server project

To get started, create a new MCP server project using the Redpanda CLI.

  1. Create a directory for your MCP server project:

    mkdir redpanda-connect-mcp
cd redpanda-connect-mcp

  2. Initialize the MCP server project:

    rpk connect mcp-server init

    This command scaffolds your project with the necessary directories and template YAML files for defining tools and observability resources.

    ├── o11y
│   ├── metrics.yaml
│   └── tracer.yaml
└── resources
    ├── caches
    │   └── example-cache.yaml
    ├── inputs
    │   └── example-input.yaml
    ├── outputs
    │   └── example-output.yaml
    └── processors
        └── example-processor.yaml

  3. Make sure your Redpanda Connect version is at least 4.66.1:

    rpk connect --version

    If you need to upgrade, see Upgrade Redpanda Connect.

Create a tool definition

In this step, you’ll create a Redpanda Connect configuration that searches public Bluesky posts using a customizable query. You’ll define this configuration as a tool that Claude can discover and use.

Save the following YAML to a file named bluesky.yaml and save it in the resources/processors/ directory:

label: search-bluesky-posts (1)
try: (2)
  - mutation: |
      root.limit = root.limit.number(10)
      root.limit = [ root.limit, 100 ].min()
      root.limit = [ root.limit, 1 ].max()
      meta query_string = "q=" + root.query.escape_url_query() + "&limit=%v".format(root.limit)
      root = ""
  - http:
      url: "https://api.bsky.app/xrpc/app.bsky.feed.searchposts?${! @query_string }"
      verb: GET
  - mapping: 'root = this.posts'
  - unarchive:
      format: json_array

meta:
  tags: [ bluesky ] (3)
  mcp:
    enabled: true
    description: Search public Bluesky posts based on a query string. (4)
    properties: (5)
      - name: query
        type: string
        required: true
        description: A Lucene-style query string to search posts.
      - name: limit
        type: number
        description: Number of posts to return (1-100). Defaults to 10.

This example defines a tool that searches public Bluesky posts based on a query string. Important parts of the configuration include:

1 A unique label for identification. Claude uses this label to discover and call the tool.
2 A series of processors to build the HTTP request, handle the response, and format the output.
3 A unique tag (bluesky) to control exposure. You can use tags to group related tools or isolate experiments.
4 A clear description for LLMs helps Claude understand what the tool does.
5 Structured properties for inputs like query and limit.

Start the MCP server

  1. Navigate to your MCP server project directory if you’re not already there:

    cd redpanda-connect-mcp

  2. Start the MCP server to expose the tool over HTTP:

    rpk connect mcp-server --address localhost:4195 --tag bluesky

    You should see output like this:

    time=2025-06-27T15:20:27.976+01:00 level=INFO msg="Registering processor tool" label=search-bluesky-posts
time=2025-06-27T15:20:27.978+01:00 level=INFO msg="Successfully loaded Redpanda license" expires_at=2035-06-25T15:20:27+01:00 license_org="" license_type="open source"

    This command creates an MCP server listening on localhost:4195, and makes your tool discoverable to AI agents. Leave this terminal open while you interact with Claude Code.

Only tools with the specified --tag are exposed. This helps you:

  • Keep experiments isolated

  • Avoid exposing sensitive functionality accidentally

  • Create sets of tools that are relevant to specific agents or workflows

Connect Claude Code to your MCP server

To connect Claude Code to your MCP server, you need to expose a live event stream that Claude can consume. This is done using the mcp-remote utility, which bridges your local service to Claude’s MCP interface. mcp-remote is a lightweight bridge that turns any streaming HTTP endpoint into a source of MCP-compatible messages.

  1. Open a new terminal window.

  2. To install mcp-remote, run:

    claude mcp add local -- npx mcp-remote http://localhost:4195/sse

    You should see output like this:

    Added stdio MCP server local with command: npx mcp-remote http://localhost:4195/sse to local config

  3. Verify that the local input channel is set up correctly by running:

    claude /mcp

    You should see an entry for local.

  4. Press Enter until you see the tools list.

    Tools for local (1 tools)
│ ❯ 1. search-bluesky-posts

  5. Press Esc until you return to the main prompt.

Write a prompt that uses the tool

To use the search-bluesky-posts tool in Claude, write a prompt that includes a natural language request.

  1. Enter the following prompt to start a conversation with Claude Code:

    Search Bluesky for the latest news about Redpanda Data

    If you exited the prompt, you can re-enter it by running:

    claude Search Bluesky for the latest news about Redpanda Data

  2. When Claude prompts you for permissions, select Yes:

    > Search Bluesky for the latest news about Redpanda Data

⏺ local:search-bluesky-posts (MCP)(value: "Redpanda")

╭─────────────────────────────────────────────
│ Tool use
│   local:search-bluesky-posts(value: "Redpanda") (MCP)
│   Search public Bluesky posts based on a query string.
│ Do you want to proceed?
│ ❯ 1. Yes
│   2. Yes, and don't ask again for local:search-bluesky-posts commands in /Users/jakecahill/Documents/my-agent  │
│   3. No, and tell Claude what to do differently (esc)

Claude will:

  1. Fill in the query property

  2. Send an HTTP request to your local MCP server

  3. Return the result in conversation

If you change the YAML configuration of your tools, make sure to restart the MCP server to pick up the changes.

Here’s an example of what the result might look like:

⏺ Based on the search results from Bluesky, here are the latest posts about Redpanda Data:

  Recent Company Activity:
  - Product Development: Expanding Redpanda Connect beyond data infrastructure to support AI agent tools like
   Slack, GitHub, and Google Drive
  - New Snowflake Connector: Launched with 2x faster throughput than Kafka Connect for data streaming

  Events & Marketing:
  - Participated in Data+AI Summit (Booth E500)
  - Hosting disaster recovery streamcast on June 12th
  - Fireside chat with Entain's Chief Data Officer about replacing Confluent with Redpanda in 2 weeks

  Success Stories:
  - Gaming company Entain replaced Confluent with Redpanda in just two weeks
  - Focus on real-time financial data streaming to Snowflake for analytics

  The posts show Redpanda positioning itself as a faster, simpler alternative to Kafka/Confluent with strong
  momentum in hiring and customer wins.

Stop or disconnect the MCP server

To disconnect or stop the MCP server, press Ctrl+C in the terminal where the server is running. This will gracefully shut down the MCP server process and disconnect any connected clients.

You can also close the terminal window or kill the process using standard OS commands (such as kill <pid> on Linux/macOS).

Troubleshoot

This section covers issues you might encounter when setting up and using the MCP server.

JSON schema errors

This error indicates you’re using an outdated version of Redpanda Connect with an incompatible JSON schema format:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "tools.17.custom.input_schema: JSON schema is invalid. It must match JSON Schema draft 2020-12 (https://json-schema.org/draft/2020-12). Learn more about tool use at https://docs.claude.com/en/docs/tool-use."
  },
  "request_id": "req_011CTWPsqnC26DJVUhxnQnn6"
}

Solution: Upgrade to at least version 4.66.1 of Redpanda Connect.

rpk connect --version

If you need to upgrade, see Upgrade Redpanda Connect.

Tool not found or not exposed

If Claude Code doesn’t show your tool in the tools list, or shows "No tools available":

  1. Ensure your YAML files are in the correct directory structure. The expected structure is:

    • Processors: resources/processors/

    • Inputs: resources/inputs/

    • Outputs: resources/outputs/

    • Caches: resources/caches/

  2. Verify MCP metadata:

    • Confirm your tool has the meta.mcp.enabled: true property

    • Check that the meta.tags array includes the tag you specified when starting the server

      Example correct structure
      label: my-tool
# ... processor configuration ...
meta:
  tags: [ bluesky ]  # Must match --tag argument
  mcp:
    enabled: true    # Required for exposure
    description: Tool description

Connection issues

If Claude Code can’t connect to your MCP server:

  • Server not running:

    • Verify the MCP server is still running in your terminal.

    • Check that you see the "Registering processor tool" log messages.

    • Ensure the server is listening on the expected address (localhost:4195 by default).

  • If port 4195 is already in use, specify a different port:

    rpk connect mcp-server --address localhost:4196 --tag bluesky

    Then update your mcp-remote connection:

    claude mcp add local -- npx mcp-remote http://localhost:4196/sse

Next steps

Try adding more tools under the same example tag to expand Claude Code’s capabilities. See Build an MCP Server in Redpanda Connect.

View the full catalog of connectors you can use to build more tools.

You can connect any MCP client to your MCP server. For a list of example clients, see the MCP documentation.