Docs Connect MCP Servers Developer Guide Build an MCP Server in Redpanda Connect This guide teaches you how to build MCP servers with self-managed Redpanda Connect. MCP servers run as part of your Redpanda Connect deployment and expose tools that AI clients can call using the Model Context Protocol. Prerequisites The Redpanda CLI (rpk) At least version 4.56.0 of Redpanda Connect If you need to upgrade, see Install or Upgrade with rpk. Basic understanding of YAML, HTTP APIs, and event-stream processing concepts (Optional) Claude Desktop or other MCP-compatible AI client for testing For a quickstart, see Redpanda Connect MCP Server Quickstart. Concepts and architecture MCP server A service that exposes tools through the Model Context Protocol. In Redpanda Connect, this is enabled using the rpk connect mcp-server command to start a dedicated MCP server process. Tool A single request/response operation exposed to MCP clients. Each tool is implemented as a Redpanda Connect configuration and described to MCP with meta.mcp metadata. Contract The formal interface a tool exposes to AI clients, including its description, input parameters and types. The contract ensures that clients know how to call the tool and what results to expect. In Redpanda Connect MCP, the contract is declared in the tool’s meta.mcp metadata. Redpanda Connect configuration A YAML file that defines how data flows through inputs, processors, and outputs. When used as an MCP tool, Redpanda Connect configurations typically receive JSON input and return structured results. Secrets Credentials and tokens should be stored as environment variables and referenced as ${ENV_VAR}. Never hardcode secrets in YAML files. Development workflow Initialize the project: Use the Redpanda CLI to scaffold your project. Design the tool contract: Annotate MCP metadata with meta.mcp.enabled: true, provide a concise description, and declare parameters. Implement the configuration: Build the logic for your MCP tool using Redpanda Connect configurations. Lint and start the MCP server: Launch your server using rpk connect mcp-server to expose your tools. Connect your AI assistants: Set up Claude Code or other MCP clients to use your tools. Publish your MCP server (optional): Make your MCP server discoverable by AI clients and agents by publishing it to an MCP registry. This enables authentication and access control for your tools. Initialize a new MCP server project Initialize a new MCP server project using the Redpanda CLI. This sets up the recommended folder structure and example configuration files. rpk connect mcp-server init This command scaffolds your project with the necessary directories and template YAML files for defining tools and observability resources. ├── o11y │ ├── metrics.yaml │ └── tracer.yaml └── resources ├── caches │ └── example-cache.yaml ├── inputs │ └── example-input.yaml ├── outputs │ └── example-output.yaml └── processors └── example-processor.yaml The resources directory is where you define your tools such as inputs, outputs, and processors. Each tool is defined in its own YAML file. By default, the example- files are provided as templates. The o11y directory contains configuration for observability, including metrics and tracing. Remove the example- files and add your own tools: rm resources/inputs/example-input.yaml rm resources/outputs/example-output.yaml rm resources/processors/example-processor.yaml rm resources/caches/example-cache.yaml Then, you can create new YAML files for each tool you want to expose. For example: touch resources/processors/weather-lookup.yaml touch resources/processors/database-query.yaml See the next sections for details on customizing these files for your use case. Design the tool contract and MCP metadata Each MCP tool must declare its interface using meta.mcp metadata. This metadata allows AI clients to discover and invoke the tool correctly. Define a clear, stable interface for each tool. Keep the description task-oriented and keep parameters to a minimum. meta: mcp: enabled: true (1) description: "Fetches a compact summary from an external API using two optional parameters." (2) properties: (3) - name: parameter1 type: string description: "Primary filter; defaults to provider standard when omitted." required: false - name: parameter2 type: number description: "Limit of results (1-100)." required: false 1 Set meta.mcp.enabled: true to expose the tool using MCP. 2 Add a concise description that explains what the tool does. The description is passed as a tool option, making it available to clients and documentation. This should be understandable by an AI model. 3 List the input parameters (properties) for the tool. Property guidance: Use string, number, or boolean types. Validate ranges and enums using Bloblang. Mark only mandatory fields as required. Document defaults in the description and enforce them in the configuration. After defining your tool contract, implement the configuration to handle input validation, defaults, and the main processing steps. Implement the configuration Use Redpanda Connect components to implement the logic of your MCP tool. Here are some best practices: Single responsibility: Each tool should do one thing well. Descriptive naming: Use clear, specific labels like fetch-user-profile instead of generic names like get-data. The top-level label becomes the tool’s name in the MCP server. Input validation: Always validate and sanitize user inputs using Bloblang. Error handling: Provide meaningful error messages. Documentation: Write clear descriptions that explain what the tool does and what it returns. For common patterns and examples, see Redpanda Connect Patterns for MCP Servers. Here’s a complete example that demonstrates best practices: label: weather-service processors: - label: validate_inputs mutation: | # Validate and sanitize city input meta city = this.city.string(). re_replace_all("[^a-zA-Z\\s\\-]", ""). trim() # Check for empty input root = if @city == "" { throw("City name cannot be empty") } else { "" } - label: fetch_weather_data try: - http: url: "https://wttr.in/${! @city }?format=j1" verb: GET headers: User-Agent: "redpanda-connect-mcp/1.0" timeout: "10s" - mutation: | root = { "city": @city, "temperature_c": this.current_condition.0.temp_C.number(), "temperature_f": this.current_condition.0.temp_F.number(), "feels_like_c": this.current_condition.0.FeelsLikeC.number(), "humidity": this.current_condition.0.humidity.number(), "description": this.current_condition.0.weatherDesc.0.value, "wind_speed_kmh": this.current_condition.0.windspeedKmph.number(), "timestamp": now().format_timestamp("2006-01-02T15:04:05Z07:00") } - log: message: "Weather data fetched for city: ${! @city }" level: "INFO" - label: handle_weather_errors catch: - mutation: | root = { "error": "Failed to fetch weather data", "city": @city, "details": error(), "timestamp": now().format_timestamp("2006-01-02T15:04:05Z07:00") } - log: message: "Weather API error for city ${! @city }: ${! error() }" level: "ERROR" meta: tags: [ weather, example ] mcp: enabled: true description: "Get current weather conditions for any city worldwide" properties: - name: city type: string description: "Name of the city (such as 'San Francisco', 'London')" required: true YAML configuration rules Each YAML file should contain exactly one component type. The component type is inferred from the directory structure: Directory Component Type resources/inputs/ Input component resources/outputs/ Output component resources/processors/ Processor component resources/caches/ Cache component Correct example when inside resources/inputs/ label: event-reader redpanda: seed_brokers: [ "${REDPANDA_BROKERS}" ] topics: [ "events" ] consumer_group: "mcp-reader" meta: mcp: enabled: true description: "Consume events from Redpanda" Correct example when inside resources/processors/ label: fetch-example-data processors: - label: safe_operation try: - http: url: "https://api.example.com/data" timeout: "10s" - mutation: | root = this.merge({"processed": true}) - label: handle_errors catch: - mutation: | root = { "error": "Operation failed", "details": error() } Incorrect (do not include the input wrapper) label: incorrect-example input: redpanda: seed_brokers: [ "${REDPANDA_BROKERS}" ] topics: [ "events" ] Incorrect (multiple component types in one file) label: incorrect-example input: redpanda: { ... } processors: - mutation: { ... } output: redpanda: { ... } Incorrect (try/catch as single processor) label: incorrect-example processors: - label: operation try: - http: { ... } catch: - mutation: { ... } Start the MCP server Navigate to the directory where you initialized the MCP server project. Lint your configuration files before starting the server: rpk connect mcp-server lint This command checks all YAML files in your resources directory for errors or misconfigurations. Fix any issues reported before proceeding. Start the MCP server to expose all your tools over HTTP: rpk connect mcp-server --address localhost:4195 This command creates an MCP server listening on localhost:4195. You should see output like this: time=2025-06-27T15:20:27.976+01:00 level=INFO msg="Registering processor tool" label=... time=2025-06-27T15:20:27.978+01:00 level=INFO msg="Successfully loaded Redpanda license" expires_at=2035-06-25T15:20:27+01:00 license_org="" license_type="open source" Expose a subset of tools To expose only a subset of tools, use the --tag flag. This helps you: Keep experiments isolated. Avoid exposing sensitive functionality accidentally. Create sets of tools that are relevant to specific agents or workflows. For example, to expose only tools tagged with example, start the server with: rpk connect mcp-server --address localhost:4195 --tag example Connect an MCP client You can connect any MCP-compatible client to your MCP server using the server’s HTTP endpoint. Most clients require the server address and may support authentication or custom headers. To connect a generic MCP client: Ensure your MCP server is running and accessible at the desired address (for example, http://localhost:4195). Configure your client with the MCP server’s endpoint. For example: http://localhost:4195/sse. Refer to your client’s documentation for details on supported options and advanced configuration. For a list of example clients, see the MCP documentation. Connect Claude Code to your MCP server To connect Claude Code to your MCP server, you need to expose a live event stream that Claude can consume. This is done using the mcp-remote utility, which bridges your local service to Claude’s MCP interface. mcp-remote is a lightweight bridge that turns any streaming HTTP endpoint into a source of MCP-compatible messages. To install mcp-remote, run: claude mcp add local -- npx mcp-remote http://localhost:4195/sse You should see output like this: Added stdio MCP server local with command: npx mcp-remote http://localhost:4195/sse to local config claude mcp add local -- This tells Claude to set up a new local input channel, which is a subprocess or pipe that streams messages. The -- delimiter ensures that everything after it is treated as a shell command to execute. npx mcp-remote http://localhost:4195/sse This runs the mcp-remote utility, which: Connects to the provided SSE endpoint Converts events into MCP message format Writes them to stdout Claude reads these messages from the subprocess, treating them as if they were emitted by a native MCP agent. Verify that the local input channel is set up correctly by running: claude /mcp You should see an entry for local with the command you just added. Press Enter until you see the tools list. Tools for local (1 tools) │ ❯ 1. search-bluesky-posts In summary, the flow is: Your MCP server exposes a Server-Sent Events (SSE) stream at http://localhost:4195/sse. mcp-remote connects to that stream and reads the events. mcp-remote converts each event into a structured MCP message. Those messages are forwarded to Claude Code through the local input channel created by claude mcp add local. Publish your MCP server To make your MCP server discoverable by AI clients and agents, you can publish it to an MCP registry. This also enables you to provide authentication and access control for your tools. See the official guide for publishing and securing your server: MCP Registry Publishing Guide. Troubleshooting This section covers common issues and debugging techniques when developing MCP tools with Redpanda Connect. Tool not appearing in MCP client Check that meta.mcp.enabled: true is set. Verify the MCP server is running on the expected port. Ensure the tool has the correct tag specified in the server startup command. Unable to infer component type If you see errors like: resources/inputs/redpanda-consume.yaml(1,1) unable to infer component type: [input processors cache_resources meta] resources/outputs/redpanda-publish.yaml(1,1) unable to infer component type: [processors output meta] This means your YAML file contains more than one component type, or uses a wrapper (such as input: or output:) that is not allowed. Each YAML file must contain only a single component type, and should not be wrapped in an input: or output: block. To fix this: Split out each component type into its own file (for example, one file for the input, one for the processors, one for the output). See YAML configuration best practices for correct examples. Example of incorrect YAML input: redpanda: { ... } processors: - mutation: { ... } output: redpanda: { ... } Example of correct YAML (for an input) label: my_input redpanda: seed_brokers: [ "${REDPANDA_BROKERS}" ] topics: [ "events" ] consumer_group: "mcp-reader" meta: mcp: enabled: true description: "Consume events from Redpanda" Suggested reading Redpanda Connect components reference Redpanda Connect Patterns for MCP Servers Bloblang language guide Model Context Protocol documentation MCP Server quickstart Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution 🎉 Thanks for your feedback! Quickstart MCP Server Patterns