Monitor and Manage an LLM Provider

Open LLM providers in the sidebar, then click the provider’s name.

The detail page organizes everything about the provider into tabs: Overview, Connect, Cost and usage, Playground (when enabled for your organization), and Settings. The Cost and usage tab charts this provider’s spend and usage, which Analyze Cost and Usage covers across every provider. The page header carries the provider’s logo, its display name with an Enabled or Disabled badge beside it, its resource identifier, and its description if it has one.

The provider’s model list sits in the Models section of the Settings tab, alongside the rest of the provider’s configuration. A saved link that ends in tab=models opens Settings instead.

The Overview tab reports over a single time range, which you select with the tab’s range selector. The range covers everything on the tab: the metric cards, the Gateway traffic chart, Who calls what, and the Callers table. It opens on the last seven days, and you can pick a preset or an exact window of your own.

Three metric cards give total spend, requests, and tokens over that range. Each card gives the total, how it changed against the range of the same length immediately before it, a trend line, and a breakdown of the figure by cost type, request outcome, or token type. The token card also reports what share of prompt tokens came from cache. Click View more on a card to open a detail panel for that metric, where you can group and filter the figure.

A Gateway traffic chart plots token volume as bars and spend as a line, with the totals for the range in its header. Drag across the chart, or pinch it, to zoom into a narrower window, which then becomes the range for the whole tab. While a window you selected that way is in force, click Reset time range to go back to the last seven days. The reset also clears any preset you picked from the selector. Click View all to open the Cost and usage page with this provider charted against the gateway total over the same range. The Connection card lists the provider type, the authentication passthrough state, and any attached guardrail, then the endpoint values with a copy control each: the proxy URL, the upstream base URL, the API key secret reference, and, for an Amazon Bedrock provider, the region and its credential references.

A Who calls what card then draws the caller-to-model traffic for that range. Hover over a node in that graph to isolate its connections, which the graph’s hint reminds you of on a machine with a mouse.

A Callers table lists, over the same range, each agent or user that called the provider, with its request count, the models it used, its error count, spend, and when it was last seen. Sort the table by requests, errors, spend, or last seen. It opens on the busiest caller first. Search the table by caller name, slug, or email. Select an agent’s row to open the Activity tab of the Cost and usage page for that agent, or a user’s row to open that page with this provider charted against the gateway total. For analysis across providers, see View cost and usage.

When a caller has failed requests in the selected range, Who calls what shows a warning counting the callers that need attention, and says so when some callers aren’t shown. Expand it to see each failing caller with its count of failed requests, then click Inspect to open the details. An affected caller’s node in the graph and a caller’s error count in the Callers table open the same details.

The Failed provider calls panel lists that caller’s failed calls over the range, newest first. Each entry carries a label for what went wrong, such as Rate limited, Authentication failed, Permission denied, Resource not found, Timed out, Provider unavailable, Safety policy blocked, Request too large, Invalid request, Provider internal error, or Provider error, along with when the call started, a summary of the error, the model, and the latency. When the provider reported no specific cause, the summary says so. Click Diagnostics on an entry to see its trace ID, span ID, and response ID, each with a copy control. An entry that recorded none of the three has no Diagnostics control. Click Load more to page through a longer list. The panel reports what the gateway recorded about the call itself, not the prompt or the model’s output.

Each entry offers the next steps that fit its failure:

The Connect tab generates ready-made configuration for this provider, with the provider’s proxy URL already filled in. It offers two paths:

For a Bedrock provider, the Claude Code guide runs against the gateway without local AWS credentials, because the gateway signs the upstream AWS requests for you, and the HTTP example uses the Bedrock Converse API. For a Gemini Enterprise Agent Platform provider, the Claude Code guide needs a Claude model that the provider allows. See Run Claude Code for that path. See Connect your app to AI Gateway for the underlying flow.

Which client guides appear depends on the provider type and on the models the provider allows:

On an Amazon Bedrock, Gemini Enterprise Agent Platform, Google Gemini API, or Fireworks provider, the tab loads model metadata before it offers a client guide. If the tab can’t load the model catalog, it reports Model catalog unavailable with Try again, which reloads every model at once. On a Google Gemini API provider, the generic client guides stay available while the catalog is unavailable, and only the Antigravity CLI guide waits for the metadata. A configured model that the catalog doesn’t list appears without metadata, and the tab doesn’t flag it. The code examples and agent framework recipes under Use the API directly report the same state.

On a Fireworks provider, the tab offers a guide only for a client that one of the provider’s models can serve, and it reports No CLI integration for this provider yet when none can. When a configured model qualifies, each guide leads with the rpk ai run command and names the minimum client version and the provider read access that the command needs. The manual Claude Code setup starts a session that excludes your user and project settings and sends main and background requests to the same model. Claude Code might warn that it doesn’t recognize the model identifier. The manual Codex setup passes the provider to Codex as per-session options. Under Use the API directly, select a Request format of Messages, Chat Completions, or Responses to get the matching curl, Python, TypeScript, Go, and Java examples. The tab doesn’t offer Agent SDK recipes for a Fireworks provider. Use the HTTP and provider SDK examples instead.

With authorization passthrough on, each client sends its own upstream credential in the Authorization header and its API key in the X-Redpanda-Api-Key header. These are separate credentials. The API key authenticates to AI Gateway, not to your upstream account. The tab adjusts its guides to match:

The Cost and usage page reports spend, requests, and tokens for a time window against the period before it, breaks spend down by agent, model, provider, user, or tag, charts any one of them against the gateway total, and exports CSV. See Analyze Cost and Usage.

When the Playground tab is available, use it to test one of the provider’s models before you connect application code. Choose a model, send a prompt, and optionally adjust response controls such as reasoning effort and maximum output tokens. The response streams into the conversation while the Playground shows the request, response events, status, and timing.

Your browser saves Playground conversations, separately for each signed-in user, organization, and environment, so switching organization or environment opens a different set. Signing out deletes every saved Playground conversation from that browser. AI Gateway still proxies each Playground request, so the provider’s Transcripts settings decide whether it records the prompts and responses (see Configure message recording). If the Playground can’t match the saved conversations to your sign-in and environment, it reports Playground session unavailable and offers Try again.

On a Gemini Enterprise Agent Platform provider, the Playground can send requests to Google and Anthropic models. It tells them apart by the publisher the model catalog records for each one. It refuses a model from any other publisher, or one the catalog has no entry for. In that case it explains why and offers no cURL command. To set up the provider, see Set up Gemini Enterprise Agent Platform as an LLM provider.

On a Fireworks provider, the Playground sends each request on the API the catalog lists first for the model. It offers only the controls the catalog declares for that model, such as reasoning effort levels and a maximum output length within the model’s limit. It also rejects an image sent to a model that doesn’t accept images. If the catalog has no details for the model, the Playground says so and asks you to reload the page.

The Settings tab is where you edit the provider after you create it. It is organized into sections:

AI Gateway rejects a request for a model that isn’t enabled on the provider with HTTP 403 and the error type model_not_allowed. This applies to every provider type.

On an Amazon Bedrock or Fireworks provider, disabling a model also affects agents. Agentic Data Plane rejects an agent that references a disabled model, at agent creation or when the model reference changes. Agents that already reference a model you disable stay saveable until someone edits their model. See Create an Agent.

When you change a field, a save bar appears with Save changes and Discard. An enable/disable control on the tab stops or resumes routing without deleting anything, and a Delete this provider danger zone permanently removes the provider after you confirm.

Each model enabled on a provider has a detail page, which gathers what the catalog knows about that model on this provider in one place. The provider’s model list doesn’t link to it, so open it by URL: https://ai.redpanda.com/llm-providers/<provider-name>/models/<model-id>, where <provider-name> is the provider’s resource name and <model-id> is a model enabled on it. The page shows:

The LLM providers list page is the at-a-glance home for every provider in your Agentic Data Plane environment. Open it from the sidebar’s LLM providers entry.

Click the 24h requests or 30d spend column header to order the list by that figure. A metric sort replaces any other column’s sort. Use the search box to find a provider by name, and the Filter button to narrow the list by provider type, model, or status. The Add provider button opens the create flow described in Open the Create LLM provider page. Each row’s actions menu can enable or disable the provider, copy its proxy URL, or delete it, and offers to copy the upstream base URL for an OpenAI or OpenAI-compatible provider. Disabling asks you to confirm, and deleting asks you to type the provider’s display name. The list paginates, with a rows-per-page selector in the footer.