Agentic Data Plane
beta

Kafka Managed MCP Server

The Kafka managed MCP server lets agents produce messages to topics on either an Apache Kafka cluster or a Redpanda cluster. Despite the name, it works against any Kafka-compatible broker.

After reading this page, you will be able to:

  • Configure the Kafka managed MCP server against a Kafka or Redpanda cluster

  • Produce a test message through the Inspector

  • Pick the right SASL mechanism for your broker (PLAIN or SCRAM-SHA-256/512)

What this MCP server does

The Kafka managed type proxies a managed Kafka producer. It exposes the following tool:

  • produce: Send a message to a Kafka topic, with an optional key, an optional partition, and optional headers.

Consuming messages, listing topics, and inspecting metadata are not currently exposed by this managed type. To read from topics, use a self-managed MCP server or Redpanda Connect.

Prerequisites

  • A Kafka or Redpanda cluster reachable from the Agentic Data Plane.

  • The cluster’s bootstrap servers and SASL/TLS settings.

  • For SCRAM or PLAIN: Secrets in the Redpanda ADP secret store for the username and password (UPPER_SNAKE_CASE, for example KAFKA_SASL_USER and KAFKA_SASL_PASSWORD).

Configure

  1. Open MCP Servers > Create Server.

  2. Pick Kafka from the marketplace picker.

  3. Fill in the identity fields (name, description).

  4. In the Kafka configuration form, provide:

    • Seed brokers: Bootstrap broker addresses for the cluster.

    • TLS: Enable for production. You can optionally skip certificate verification for local development only.

    • SASL mechanism: PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512. Leave empty to disable SASL.

    • Username / password: UPPER_SNAKE_CASE secret references for the SASL credentials.

    • Compression (optional): Defaults to lz4.

    • Max in-flight (optional): Maximum number of in-flight produce requests. Defaults to 10.

    • Timeout (optional): Produce request timeout. Defaults to 10s.

  5. Click Create.

Test

  1. Open the Inspector tab on the server’s detail page.

  2. In Tools, select the produce tool.

  3. Produce a test message to a sandbox topic and confirm a successful response.

  4. Verify the message landed by consuming the topic with rpk topic consume or another Kafka client.

See Test an MCP Server’s Tools with the Inspector for general Inspector usage.

Authentication

The Kafka managed type’s authentication is part of its config, not the generic MCP authentication modes: it uses Kafka protocol authentication (SASL over TLS), not MCP authentication.

Mechanism Use when

PLAIN

Username and password over TLS. Common for managed Kafka services.

SCRAM-SHA-256 / SCRAM-SHA-512

Salted challenge-response. Default for Redpanda.

TLS controls transport encryption: enable it, and optionally skip certificate verification for development. Client-certificate (mTLS) and OAUTHBEARER authentication are not currently supported by this managed type.

Use with agents

Once the Kafka server is created, point an agent at the API URL on the server’s detail page. The agent can then produce messages to topics through the exposed produce tool.

Troubleshooting

Symptom What to check

connection refused or dial timeout

Brokers aren’t reachable from ADP egress. Confirm bootstrap addresses and any private-network requirements.

SASL authentication failed

Check username/password reference content and the SASL mechanism.

TLS handshake error

Certificate chain isn’t trusted, or you’ve enabled TLS against a plaintext broker. Confirm broker config.

unknown topic or partition

The target topic doesn’t exist on the broker. Create it first, or confirm the topic name.

Limitations

  • Reading and administration: This server only produces messages. Consuming, listing topics, and managing topics or ACLs aren’t exposed. Use rpk, a Kafka client, or Redpanda Connect.

  • Schema registry: Not exposed by this MCP server.

  • Streaming joins or processing: For stream processing, use Redpanda Connect.