Remote MCP Server Quickstart

Model Context Protocol (MCP) enables AI assistants like Claude to interact with external systems through custom tools. This quickstart teaches you how to create an MCP server in Redpanda Cloud with tools that generate and publish event data, then connect Claude Code to use those tools through natural language commands.

What you’ll learn

By completing this quickstart, you’ll understand:

What MCP is and how it extends AI capabilities
How MCP servers, tools, and clients work together
How to create and deploy custom tools in Redpanda Cloud
How to connect AI assistants to your Redpanda data infrastructure

How MCP works

The Model Context Protocol defines a standard way for AI assistants to interact with external tools and data sources:

MCP Server: Hosts a collection of tools that perform specific actions. MCP servers run as managed services in your Redpanda Cloud cluster.
Tools: Individual capabilities your AI can use, built with Redpanda Connect components. Tools can read from sources (inputs), transform data (processors), write to destinations (outputs), or cache results.
Client: An AI assistant (like Claude Code) that discovers available tools and calls them based on user requests.

In this quickstart, you’ll create a server with two tools:

generate_input: Creates fake user event data with randomized fields
redpanda_output: Publishes generated events to a Redpanda topic

Once connected, you can ask Claude in natural language to "generate 10 events and publish them to Redpanda," and Claude will automatically call your tools to complete the task.

Prerequisites

A Redpanda Cloud cluster with Remote MCP enabled.
Access to the Secrets Store for storing credentials.
At least version 25.2.5 of the Redpanda CLI (rpk) installed on your computer.
Claude Code installed.

Prepare your cluster

Before creating the MCP server, you need to set up a topic for event publishing.

rpk
Data Plane API

Log in to your Redpanda Cloud account:
```
rpk cloud login
```
This opens a browser window to authenticate. The token is saved locally inside your rpk configuration file. It is valid for 4 hours. You can refresh it by running rpk cloud login again.
Create a topic called events for storing user event data:
```
rpk topic create events --partitions 3 --replicas 3
```
Create a user called mcp with a strong password:
```
rpk acl user create mcp --password <your-secure-password>
```
Save the password securely. You need it later when configuring the MCP server.

Grant the mcp user permissions to produce and consume from the events topic:

rpk acl create --allow-principal User:mcp --operation all --topic events

Authenticate to the Control Plane API to get an access token.

Access tokens expire after 1 hour. To refresh, make the same authentication request again with your service account credentials. The same token works for both Control Plane and Data Plane API requests.

Get the Data Plane API URL for your cluster:

BYOC or Dedicated
Serverless

Make a request to the GET /v1/clusters/{id} endpoint:

curl "https://api.redpanda.com/v1/clusters/<cluster-id>" \
  -H "Authorization: Bearer <control-plane-token>"

Make a request to the GET /v1/serverless/clusters/{id} endpoint:

curl "https://api.redpanda.com/v1/serverless/clusters/<cluster-id>" \
  -H "Authorization: Bearer <control-plane-token>"

The response includes a dataplane_api.url value:

  "id": "....",
  "name": "my-cluster",
....
  "dataplane_api": {
    "url": "https://api-xyz.abc.fmc.ppd.cloud.redpanda.com"
  },
...

The dataplane_api.url field might not be immediately available when a cluster reaches STATE_READY. If the field is missing or null, wait a few minutes and make the request again. The Data Plane API URL is typically available within 5-10 minutes after cluster creation completes.

Make a request to POST /v1/topics to create the topic:

curl -X POST "https://<dataplane-api-url>/v1/topics" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "events",
    "partition_count": 3,
    "replication_factor": 3
  }'

Make a request to POST /v1/users to create a user called mcp:

curl -X POST "https://<dataplane-api-url>/v1/users" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "mcp",
    "password": "<your-secure-password>",
    "mechanism": "SASL_MECHANISM_SCRAM_SHA_256"
  }'

Save the password securely. You need it later when configuring the MCP server.

Make a request to POST /v1/acls to grant the mcp user permissions to produce and consume from the events topic:

curl -X POST "https://<dataplane-api-url>/v1/acls" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "resource_type": "RESOURCE_TYPE_TOPIC",
    "resource_name": "events",
    "resource_pattern_type": "RESOURCE_PATTERN_TYPE_LITERAL",
    "principal": "User:mcp",
    "host": "*",
    "operation": "OPERATION_ALL",
    "permission_type": "PERMISSION_TYPE_ALLOW"
  }'

Create an MCP Server in Redpanda Cloud

Cloud UI
Data Plane API

Log in to the Redpanda Cloud Console.
Navigate to Remote MCP.

This page shows a list of existing servers.

Click Create new MCP Server. In Server Metadata, configure the basic information and resources:

Display Name: A human-friendly name such as event-data-generator. This name is shown in the Redpanda Cloud Console. It is not the name of the MCP server itself.
Description: Explain what the server does. For example, Generates fake user event data and publishes it to Redpanda topics.
Tags: Add key/value tags such as owner=platform or env=demo. The tag names service_account_id and secret_id are reserved and cannot be used.
Resources: Choose a size (XSmall / Small / Medium / Large / XLarge). Larger sizes allow more concurrent requests and faster processing, but cost more. You can change this later.

Service Account: A service account is automatically created for authenticating the MCP server to your cluster. The name is pre-filled. You can customize this name or keep the default.

Service accounts authenticate MCP server requests to your Redpanda cluster. The service account has editor permissions to the cluster, allowing it to perform operations like reading and writing data, managing topics, and accessing cluster resources. Service account credentials are stored in the Secrets Store with the ID SERVICE_ACCOUNT_<mcp-server-id> and scope set to MCP server.

Click Next to define tools.

Tools define the actions your MCP server can perform. In this example, you create two tools: one for generating user event data and another for publishing that data to Redpanda.
From the Template dropdown, select Generate Input.

The template populates the configuration with YAML for the tool definition.
Click Add Tool to create a second tool.
From the Template dropdown, select Redpanda Output.

The template populates the configuration for publishing to Redpanda and a section for adding the required secrets is displayed.
Enter the values for the mcp user’s credentials in the Add Required Secrets section.
Click Lint to check the configuration. You should see no errors.
Click Create MCP Server to deploy the server.

It may take a few seconds to start. The status changes from Starting to Running when it’s ready.
Open the MCP Inspector tab to test the tools:
- For the generate_input tool, click Run Tool to generate sample event data.
- For the redpanda_output tool, enter some sample event data such as user_id=user123, event_type=login, and timestamp=2025-10-21T10:30:00Z then click Run Tool to publish it to the events topic.

Create a secret for the username:

curl -X POST "https://<dataplane-api-url>/v1/secrets" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "MCP_USERNAME",
    "scopes": ["SCOPE_MCP_SERVER"],
    "secret_data": "bWNw"
  }'

The secret_data value bWNw is the base64-encoded string mcp.

Create a secret for the password:

curl -X POST "https://<dataplane-api-url>/v1/secrets" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "MCP_PASSWORD",
    "scopes": ["SCOPE_MCP_SERVER"],
    "secret_data": "<base64-encoded-password>"
  }'

Replace <base64-encoded-password> with your password encoded in base64. You can encode it with: echo -n '<your-secure-password>' | base64.

Using the Data Plane API URL from the previous section, make a request to POST /v1/redpanda-connect/mcp-servers to create the MCP server:

curl -X POST "https://<dataplane-api-url>/v1/redpanda-connect/mcp-servers" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "event-data-generator",
    "description": "Generates fake user event data and publishes it to Redpanda topics",
    "tags": {
      "owner": "platform",
      "env": "demo"
    },
    "resources": {
      "memory_shares": "400M",
      "cpu_shares": "100m"
    },
    "tools": {
      "generate_input": {
        "component_type": "COMPONENT_TYPE_INPUT",
        "config_yaml": "generate:\n  interval: 1s\n  mapping: |\n    root.user_id = \"user\" + random_int(min: 1, max: 1000).string()\n    root.event_type = [\"login\", \"logout\", \"purchase\", \"view\"].index(random_int(max: 3))\n    root.timestamp = now().ts_format(\"2006-01-02T15:04:05Z07:00\")"
      },
      "redpanda_output": {
        "component_type": "COMPONENT_TYPE_OUTPUT",
        "config_yaml": "redpanda:\n  seed_brokers: [ \"${REDPANDA_BROKERS}\" ]\n  topic: events\n  tls:\n    enabled: true\n  sasl:\n    - mechanism: SCRAM-SHA-256\n      username: \"${secrets.MCP_USERNAME}\"\n      password: \"${secrets.MCP_PASSWORD}\"\n"
      }
    }
  }'

The response includes the MCP server ID. Wait for the status to show Running before testing the tools.

To test the tools, use the GET /v1/redpanda-connect/mcp-servers/{mcp_server_id} endpoint to verify the server is running.

Connect an AI client

Now that your MCP server is running with two tools available, you’ll connect Claude Code so it can discover and use them. You can connect any MCP-compatible AI client to your MCP server.

When you connect Claude Code:

Claude automatically discovers your generate_input and redpanda_output tools
You can ask Claude in natural language to perform tasks using these tools
Claude decides which tools to call and in what order based on your request
The Redpanda CLI acts as a secure proxy, forwarding Claude’s tool requests to your MCP server in the cloud

This example uses Claude Code, but the same pattern works with any MCP-compatible client.

Log in to your Redpanda Cloud account:
```
rpk cloud login
```
This opens a browser window to authenticate. The token is saved locally inside your rpk configuration file. It is valid for 4 hours. You can refresh it by running rpk cloud login again.

Open the Connection tab in Redpanda Cloud to get connection details and run the rpk command for Claude Code.

For BYOC and Dedicated clusters, use:

rpk cloud mcp proxy \
--cluster-id <cluster-id> \
--mcp-server-id <server-id> \
--install --client claude-code

For Serverless clusters, use:

rpk cloud mcp proxy \
--serverless-cluster-id <cluster-id> \
--mcp-server-id <server-id> \
--install --client claude-code

Restart Claude Code and invoke your tool.
```
claude
```
Ask Claude Code to use your tools. Try these example requests:
- "Generate 10 user events and then publish them to the events topic."
- "Create sample login events for users user001, user002, and user003, then publish them to Redpanda."
- "Generate purchase events with metadata and publish them to the events topic."
  
  Watch what happens:
  Claude analyzes your natural language request
  
  Claude identifies which tools to use (generate_input to create data, redpanda_output to publish)
  
  Claude calls your tools via the MCP server running in Redpanda Cloud
  
  You see the tool execution results in your Claude Code session
  You may need to respond to prompts to grant Claude permission to call the tools.
Verify the events were published by consuming from the events topic:
```
rpk topic consume events --num 10
```
You should see the generated event data in JSON format, confirming that Claude successfully used your custom tools to generate data and publish it to Redpanda.

Summary

You extended Claude’s capabilities with custom tools that interact with your Redpanda cluster. Claude can now generate and publish event data through natural language commands without manual scripting. This same pattern works for any workflow: reading from topics, transforming data, querying databases, or calling external APIs.

Troubleshoot

MCP server not starting

If your server is not starting or shows an error state:

Check the Logs tab for specific error messages.
Verify your YAML syntax by clicking Lint.

Connection issues

If Claude Code can’t connect to your server:

Verify you’re logged in with rpk cloud login.
Check that your rpk version is 25.2.5 or later: rpk version.
Ensure your server status shows Running in Redpanda Cloud.
Try restarting Claude Code after running the proxy command.

Publishing failures

If the publish tool returns errors:

Verify the events topic exists:
```
rpk topic list
```
Use the MCP Inspector in Redpanda Cloud to test with different event data.
Check the MCP server logs for authentication or authorization errors.

Next steps

Now that you understand how MCP works, you can create custom tools for your specific use cases:

Data ingestion: Build tools that read from external APIs, databases, or file systems and publish to Redpanda topics.
Data processing: Create tools that consume from topics, transform data, and write results back to Redpanda.
Monitoring and observability: Add tools that query metrics, check system health, or analyze streaming data.
Integration workflows: Connect Redpanda to other systems through custom tools that Claude can orchestrate.

See Build Remote MCP Servers in Redpanda Cloud to learn how to build custom tools using Redpanda Connect components.

Was this helpful?

group Ask in the community

mail Share your feedback

group_add Make a contribution