# Self-Managed Agent Telemetry Reference

> For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [agentic-data-plane-full.txt](https://docs.redpanda.com/agentic-data-plane-full.txt)

---
title: Self-Managed Agent Telemetry Reference
latest-operator-version: v26.2.3
latest-console-tag: v3.11.0
latest-connect-version: 4.108.0
latest-redpanda-tag: v26.2.2
docname: self-managed-telemetry
page-component-name: agentic-data-plane
page-version: master
page-component-version: master
page-component-title: Agentic Data Plane
page-relative-src-path: self-managed-telemetry.adoc
page-edit-url: https://github.com/redpanda-data/adp-docs/edit/main/modules/connect/pages/self-managed-telemetry.adoc
description: The OTLP endpoint, span contract, ingestion limits, and validation checks for streaming transcripts from a self-managed agent.
page-topic-type: reference
personas: agent_builder, platform_engineer
learning-objective-1: Look up the span attributes a transcript requires
learning-objective-2: Find the endpoint, authentication, and exporter settings for span export
learning-objective-3: Identify why exported spans were rejected or never arrived
page-git-created-date: "2026-08-20"
page-git-modified-date: "2026-08-27"
---

<!-- Source: https://docs.redpanda.com/agentic-data-plane/connect/self-managed-telemetry.md -->

This reference documents the OpenTelemetry ingestion contract for [self-managed agents](https://docs.redpanda.com/agentic-data-plane/connect/self-managed-agents/): the OTLP endpoint, the span attributes transcripts require, the request limits, and the validation checks.

Use this reference to:

-   Look up the span attributes a transcript requires

-   Find the endpoint, authentication, and exporter settings for span export

-   Identify why exported spans were rejected or never arrived


Agentic Data Plane assembles a self-managed agent’s **Transcripts** tab entirely from OpenTelemetry spans the agent exports. Routing LLM and MCP calls through the [AI Gateway](https://docs.redpanda.com/agentic-data-plane/reference/glossary/#ai-gateway) attributes spend, tokens, and latency on its own, but transcript detail (prompts, responses, tool calls, and timing) exists only in exported spans. A self-managed agent that exports nothing has a permanently empty **Transcripts** tab. Managed agents don’t need any of this: the managed runtime exports their spans automatically.

The agent’s **Setup** tab generates a wired, copy-paste sample for your framework that satisfies this contract. Use this page to understand what the generated code does, to adapt it to a framework the picker doesn’t cover, and to diagnose ingestion failures.

## [](#otlp-endpoint)OTLP endpoint

Your cluster runs an authenticated OpenTelemetry trace endpoint alongside the AI Gateway. Its host is the gateway host with `aigw.` swapped for `otlp.`:

```text
https://otlp.<cluster-id>.clusters.rdpa.co
```

The same base URL serves both transports, so any standard OpenTelemetry SDK exporter works without Redpanda-specific packages:

-   OTLP over HTTP: The SDK appends `/v1/traces` to the base URL.

-   OTLP over gRPC: The SDK dials the same host.


The generated environment block configures the exporter with three variables. Copy the exact values from the **Setup** tab:

| Variable | Value |
| --- | --- |
| OTEL_EXPORTER_OTLP_ENDPOINT | The cluster’s OTLP endpoint. Most SDKs default to localhost:4318 when this is unset, which fails silently. |
| OTEL_EXPORTER_OTLP_PROTOCOL | http/protobuf for the HTTPS URL. A transport that doesn’t match the endpoint also fails silently in many SDKs. |
| OTEL_SERVICE_NAME | The agent’s name. |

## [](#authentication-and-identity)Authentication and identity

Span exports authenticate with the same client credential the agent already uses for its gateway calls. One credential authorizes the agent’s LLM calls, its MCP tool calls, and its span exports.

-   Each export carries an `Authorization: Bearer` token minted with the `client_credentials` grant. A token lasts about an hour, so mint tokens as you export rather than pinning one in an environment variable. The samples the **Setup** tab generates do this for you.

-   The endpoint derives the agent, tenant, and organization from the access token and stamps them onto the spans as resource attributes. Identity attributes claimed in the payload are stripped and replaced. You don’t need to configure any attribution headers.


## [](#span-contract)Span contract

Transcripts read the OpenTelemetry semantic conventions for generative AI: the [GenAI span conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md) for model and tool spans, and the [GenAI agent span conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-agent-spans.md) for the `invoke_agent` turn. Every span, not just the root, must carry `gen_ai.conversation.id` set to a stable identifier for the conversation; spans without it don’t associate with any transcript. The generated samples enforce this with a span processor that stamps the attribute on every span at creation.

Beyond the conversation ID, each kind of span needs its own attributes:

| Span kind | gen_ai.operation.name | Other required attributes |
| --- | --- | --- |
| Turn (one per user request) | invoke_agent | None |
| Model step | chat, text_completion, or generate_content | gen_ai.request.model, gen_ai.provider.name, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and, when content capture is on, gen_ai.input.messages and gen_ai.output.messages |
| Tool call | execute_tool | gen_ai.tool.name, gen_ai.tool.call.arguments, gen_ai.tool.call.result |

How spans become a transcript:

-   Each distinct `gen_ai.conversation.id` becomes one conversation on the **Transcripts** tab.

-   Each `invoke_agent` span becomes a turn in that conversation.

-   Model and tool spans are attributed to a turn by their parent-span chain. When no `invoke_agent` ancestor is present, they fall back to the turn whose time window covers them. Spans with neither land in a provisional turn of their own.


## [](#message-content-capture)Message content capture

OpenTelemetry GenAI instrumentation records prompt and response text only when asked, because message content can hold sensitive data. This is a setting in the agent’s own instrumentation, not in Agentic Data Plane. For the Python-based frameworks, the generated environment block enables it with:

```bash
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="SPAN_ONLY"
export OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_latest_experimental"
```

This is separate from the per-provider **Record inputs** and **Record outputs** toggles, which control the message bodies the AI Gateway records for the calls it proxies. See [Configure transcript logging](https://docs.redpanda.com/agentic-data-plane/gateway/configure-provider/#transcript-logging). For a self-managed agent’s transcripts, the text shown comes from the spans the agent exports, so the instrumentation setting is the one that matters.

## [](#ingestion-limits)Ingestion limits

Each export request must stay within these limits:

| Limit | Value |
| --- | --- |
| Spans per request | 10,000 |
| Request payload | 4 MiB |
| Attributes per span | 256 |
| Single attribute value | 1 MiB |
| Span timestamps | Within the last 30 days, up to 10 minutes in the future |

The limits are enforced in two ways:

-   A request that exceeds the span count or the payload size fails as a whole: the export returns an error and none of its spans are stored. Reduce the exporter’s batch size and resend.

-   The attribute and timestamp limits are enforced per span: the request succeeds with an OTLP partial success naming the rejected span count and the reason, and the remaining spans are stored.


Either way, each rejection appears in the [validation log](#validate-ingestion).

## [](#framework-caveats)Framework caveats

The **Setup** tab generates samples for Redpanda AI SDK, ADK Go, Vercel AI SDK, Mastra, LangChain, CrewAI, ADK Python, and ADK Java. Some frameworks' instrumentation limits what a transcript can show:

| Framework | Caveat |
| --- | --- |
| ADK Go | Its instrumentation carries message text as span events rather than as the attributes a transcript reads. Turns, steps, token counts, and tool calls appear; prompt and response text does not. |
| ADK Java | Its model span is named call_llm, which is not a GenAI operation name, and it carries message text in framework-specific attributes. Turns and tool calls appear; model steps, token counts, and message text do not. |
| ADK Python | It records tool arguments and results under its own vendor attributes, so a tool call appears in the transcript without its arguments and result. |
| CrewAI | Its tool execution is not instrumented, so tool calls do not appear as their own transcript rows; the model steps that requested them do. The generated environment block also sets CREWAI_TRACING_ENABLED="false" so CrewAI’s own tracing does not compete with the exporter. |

## [](#validate-ingestion)Validate ingestion

Spans pass through three stages: received at the OTLP endpoint (authentication, identity stamping, and limit checks), validated for attribution, and assembled into transcripts.

Failures are durable, not just logged. The validation log at the bottom of the agent’s **Setup** tab shows the rejected telemetry from the last 24 hours, grouped by cause, one entry per problem, with the evidence and the fix.

![The Validation log on the Setup tab, showing the no-rejected-telemetry quiet state and its exporter-side checks: Endpoint, Protocol, Auth, and Flush](https://docs.redpanda.com/agentic-data-plane/connect/_images/self-managed-agent-validation-log.png)

The log records only permanent drops, so an empty log cannot distinguish a clean export from an agent that never sent anything. If the log is empty and the **Transcripts** tab is too, nothing was rejected server-side, so check the exporter side:

| Check | What to confirm |
| --- | --- |
| Endpoint | OTEL_EXPORTER_OTLP_ENDPOINT points at this cluster’s OTLP host. SDKs default to localhost:4318 when it’s unset. |
| Protocol | The transport matches the endpoint (http/protobuf for the HTTPS URL). A mismatch fails silently in many SDKs. |
| Authentication | Every export carries a freshly minted bearer token for this agent. SDKs do not always log authentication failures. |
| Flush | Short-lived processes exit before the batch exporter flushes. Call shutdown() or force_flush() on exit. |

One trap looks identical to an agent that never sent anything: an export authorized with a valid token that is not this agent’s credential (for example, a user token or a plain service-account token) is rejected, but the rejection carries no agent identity, so it never appears in this agent’s validation log. If everything else checks out, confirm the exporter authenticates with the agent’s own client credential from its **Credentials** tab.

## [](#next-steps)Next steps

-   [Set up a self-managed agent](https://docs.redpanda.com/agentic-data-plane/connect/self-managed-agents/)

-   [See what your agent did](https://docs.redpanda.com/agentic-data-plane/monitor/transcripts/)

-   [How observability works](https://docs.redpanda.com/agentic-data-plane/monitor/concepts/)