Agentic Data Plane

Monitor Agent Activity

Use monitoring to track agent performance, analyze conversation patterns, debug execution issues, and optimize token costs.

After reading this page, you will be able to:

  • Verify agent behavior and review its MCP capabilities using the Playground tab

  • Track token usage and performance metrics

  • Debug agent execution using transcripts

For conceptual background on traces and observability, see How Observability Works.

Prerequisites

You must have a running agent. If you do not have one, see Agentic Data Plane Quickstart for Agent Builders.

Debug agent execution with transcripts

An agent’s Transcripts tab shows each conversation with timing, errors, and token usage. Use it to debug issues, verify agent behavior, and monitor performance.

  1. Open Agents in the sidebar and select your agent.

  2. Open the Transcripts tab.

The tab lists the agent’s recent conversations, one row per conversation:

  • Conversation: The conversation ID, with the conversation title when one exists.

  • Started: When the conversation began.

  • Duration: End-to-end wall-clock time.

  • Turns: Number of turns in the conversation.

  • Status: Completed, Error, or Running.

  • Tokens: Total tokens across the conversation.

Use the search box to match a conversation ID or title, and the status dropdown to narrow the list to Completed, Error, or Running conversations. The list loads in pages; click Load more to fetch older conversations.

Conversation detail

Click a row to open the conversation. The header shows the conversation ID with a status badge, the start time, duration, and turn count, plus a total-tokens chip. Toggle between two views:

  • Chat: The user-visible exchange, as a conversation.

  • Detailed: Adds per-turn metadata: latency, input/output token splits, and each tool call with its arguments, result, latency, and status.

If any turns were rebuilt from LLM message context after their original spans were evicted, those turns carry a reconstructed marker. For the mechanics, see Reconstructed transcript history.

Check agent health

Use the Transcripts tab to verify your agent is healthy. Recent conversations should show Completed status, duration within your expected range, and stable token usage without unexpected growth.

Several warning signs indicate problems. Conversations with Error status need investigation. When duration increases over time, your context window may be growing or tool calls could be slowing down. Many LLM calls for simple requests often signal that the agent is stuck in loops or making unnecessary iterations. If you see no new transcripts, the agent may be stopped or encountering deployment issues.

Pay attention to patterns across multiple conversations. When all recent transcripts show errors, start by checking agent status, MCP server connectivity, and system prompt configuration. A list that alternates between success and error typically points to intermittent tool failures or external API issues. If duration increases steadily over a session, your context window is likely filling up. Clear the conversation history to reset it. High token usage combined with relatively few LLM calls usually means tool results are large or your system prompts are verbose.

Debug with transcripts

Use the Transcripts tab to diagnose specific issues:

If the agent is not responding:

  1. Check the list for recent conversations. If none appear, the agent may be stopped.

  2. Verify agent status in the main Agents view.

  3. Look for error transcripts with deployment or initialization failures.

If the agent fails during execution:

  1. Set the status dropdown to Error and open the failed conversation.

  2. Switch to the Detailed view and find the turn carrying the error.

  3. Check the tool call’s arguments and result for error messages.

  4. Cross-reference with MCP server status.

If performance is slow:

  1. Compare the Duration column across recent conversations.

  2. Open a slow conversation in the Detailed view and scan per-turn latency to find the bottleneck.

  3. Check if LLM calls are taking longer than expected.

  4. Verify tool execution time on the nested tool calls.

Track token usage and costs

View token consumption in the conversation detail view. The Detailed view breaks each turn into input tokens (everything sent to the LLM including system prompt, conversation history, and tool results) and output tokens (what the LLM generates in agent responses); the header chip shows the conversation total.

Calculate cost per request:

Cost = (input_tokens x input_price) + (output_tokens x output_price)

Example: GPT-5.2 with 4,302 input tokens and 1,340 output tokens at $0.00000175 per input token and $0.000014 per output token costs $0.026 per request.

For cost optimization strategies, see Cost calculation.

Test agent behavior with Playground

The Playground tab provides real-time conversation testing. Use it to test agent responses interactively and verify behavior before deploying changes.

Playground test loop. A user sends a prompt, the agent streams responses, artifacts, and MCP tool calls, and Playground shows context and token usage. The user can retry or clear context, while completed runs remain available in transcripts and activity views.
Figure 1. Playground closes the loop between a test prompt and production evidence

Access Playground

  1. Open Agents in the sidebar.

  2. Click your agent name.

  3. Open the Playground tab.

  4. Enter test queries and review responses.

  5. Check the conversation panel to see tool calls.

  6. Start a new session to test fresh conversations or click Clear context to reset history.

Review MCP tools and subagents

For a managed agent, the Playground composer has a Tools control, and a Subagents control when the agent delegates to subagents. Use them to review the capabilities available to the agent before you send a test message.

To review the agent’s direct MCP servers and their tools:

Playground tool discovery. The Tools control displays direct MCP servers, and the Subagents control displays the agent’s subagents. Direct servers expose their tools to the agent. Each subagent exposes only the tools from the MCP servers configured for that subagent. Authentication and availability affect which tools are visible.
Figure 2. Playground shows tool access through direct MCP servers and subagents

  1. Open the agent’s Playground tab.

  2. Click Tools in the composer.

  3. Under Direct MCP servers, expand a server to see its type, description, and exposed tools. Each tool entry shows its name and description when the server provides one.

The list reflects the agent’s current configuration and the tools each direct MCP server exposes. It does not guarantee that a later tool call succeeds. Server availability, access policy, or an external connection can still affect a call at runtime.

Some remote MCP servers require you to connect your own account before their tools are shown. Expand the server and click Connect, complete the authorization in the new browser tab, then return to Playground. The list refreshes when you return. If it does not, click Refresh tools.

Use the status shown under a server to resolve common issues:

Status What to do

Authentication required

Click Connect and complete the authorization flow. Protected tools remain hidden until the connection grants the required access.

Unavailable

Ask an administrator to confirm that the MCP server exists and is enabled for the agent.

Couldn’t load tools

Click Try again. If the error continues, check the MCP server’s availability and authentication settings.

No tools exposed by this MCP server

Confirm that the server publishes tools and that the agent is connected to the intended server.

When the agent delegates to subagents, the composer also shows a Subagents control next to Tools. Click it to review the delegation targets this agent can orchestrate. Each entry lists the subagent’s name, its description, and the MCP servers configured on it.

Context-window usage

As you test, the composer tracks how much of the model’s context window the conversation consumes. When the agent uses a model from Redpanda’s catalog, a context-usage indicator appears in the composer after the first response and shows the share of the context window in use. Click it to open a breakdown of the window size, the input, output, reasoning, and cached tokens for the session, the Session total, and the estimated Total cost.

The indicator counts input tokens plus cached tokens, so it reflects the full context occupancy under prompt caching, not fresh input alone. As a conversation grows and the indicator approaches the window size, start a new session or click Clear context to reset the history before responses slow down or the agent starts dropping earlier context.

For models that aren’t in the catalog, such as OpenAI-compatible providers or hand-entered model IDs, the composer shows a plain input and output token count instead, because the context-window size isn’t known.

Long-running tasks

The Playground live view streams a test run as it happens. If a task is still running after about five minutes, Playground stops the live view and shows a Still running notice: the task keeps running in the background, so you don’t need to hold the Playground open. Follow it to completion on the Activity tab of the Cost & Usage page under Governance.

Testing best practices

Test your agents systematically by exploring edge cases and potential failure scenarios. Begin with boundary testing. Requests at the edge of agent capabilities verify that scope enforcement works correctly. Error handling becomes clear when you request unavailable data and observe whether the agent degrades gracefully. Even with proper system prompt constraints, testing confirms that your agent responds appropriately to edge cases.

Monitor iteration counts during complex requests to ensure they complete within your configured limits. Ambiguous or vague queries reveal whether the agent asks clarifying questions or makes risky assumptions. Throughout testing, track token usage per request to estimate costs and identify which query patterns consume the most resources.

Next steps