Configure an LLM Provider

The first card on the page collects identity fields. Enter a Display name; the form auto-derives the Resource ID from it as you type.

The Summary panel labels the Resource ID as Name.

The Provider type card shows five cards. Pick the one that matches your upstream.

Selecting a type reveals the type-specific configuration fields.

Each API key reference and credential field points at a secret-store entry, not the secret value itself. Use the Existing tab to pick a secret already in your dataplane’s secret store, or the New tab to create one inline.

Models you select on this form become the catalog the provider exposes. Leave the list empty to allow every model the upstream catalog returns.

For OpenAI, Anthropic, Google AI, and AWS Bedrock, the form shows a picker backed by the provider’s catalog. Each model in the picker shows its input and output price per million tokens. Pick from the list, or type a model identifier the catalog doesn’t show. For OpenAI-compatible, the form takes a freeform list: type the exact identifiers your upstream serves.

Redpanda maintains the catalog of available models in the picker. When an upstream provider publishes a new model, it usually appears in the picker within a day or two; admins don’t have to wait for a Redpanda release. New models aren’t enabled automatically: an admin still selects the model in the catalog to make it callable through this provider.

For Bedrock, the picker exposes inference profiles, not raw foundation-model IDs. See AWS Bedrock: Inference profiles and IAM.

The Transcripts card controls whether AI Gateway records the message bodies this provider proxies. It has two independent toggles, both off by default:

Because both toggles default to off, AI Gateway does not retain message bodies for a new provider until you turn them on. Enable them to power turn-by-turn investigation and per-conversation drill-down in the Transcripts view. Leave them off for workloads where the message body must not be retained, such as regulated PII or customer secrets.

These are per-provider settings, not per-request: applications cannot opt in or out at call time. To split sensitive from non-sensitive traffic, create one provider with recording on and another with it off, and route each application to whichever proxy URL matches its data class.

Recording settings do not affect cost and usage telemetry. Token counts, latency, and provider/model attribution are always recorded, so the Cost & Usage page reports spend for traffic on the provider regardless of these toggles; only the message bodies are withheld when the toggles are off.

A successful Test Connection result confirms that the provider’s credentials, region (Bedrock), and network path are all correct. If the call fails, see Troubleshooting.

Bedrock has three concepts that affect how you configure a provider: foundation models, cross-region inference profiles, and IAM. Get these right and the Test connection check passes. Get them wrong and you see AccessDenied or ValidationException errors.

If you want each client to authenticate against Anthropic with its own subscription (Claude Pro, Max, Team, or enterprise), enable Authorization passthrough instead of configuring a server-side API key. In this mode:

The provider detail page shows whether Authorization passthrough is enabled in the Connection card.

The LLM Providers list page is the at-a-glance home for every provider in your dataplane. Open it from the sidebar’s LLM Providers entry.

The Filter button narrows the list by provider type, status, or name. The Create provider button opens the create flow described in Open the Create LLM provider page. The list paginates, with a rows-per-page selector in the footer.

The Cost & Usage page tracks spend, request volume, and token volume over time across providers and models. Open Cost & Usage under Governance in the sidebar. Use it when you want to understand which provider or model generated usage during a selected time window.

The page includes these charts:

Use Group by to switch the chart breakdown between providers, models, and token type. Group by provider to see which upstream consumed the most budget. Group by model to see which model drove spend inside one or more providers. Group by token type to separate input, output, cached, cache-write, and reasoning usage where those buckets apply.

Use Filter to narrow the charts by provider, model, cost type, token type, user, or agent. Each filter appears as a chip above the chart, and you can combine them. For example, filter to one Anthropic provider, drill into claude-opus-4-7, then limit the spend view to input tokens. Selecting an agent also narrows the provider options to the providers that agent used.

The date-range picker supports last 7 days, last 14 days, last 30 days, last 90 days, month to date, quarter to date, year to date, and custom ranges. The chart subtitle shows the selected date range and bucket size.

A custom range writes customStart and customEnd ISO-8601 timestamps to the page URL, so the view is shareable: copy the URL after picking a custom range and any teammate who opens it lands on the same window.

The chart renders empty buckets in the selected range as zero-height bars rather than gaps, so quiet days line up with their date label and the trend stays readable when traffic is bursty.

The chart palette is colorblind-safe. When multiple providers of the same type exist (for example, two OpenAI providers), the chart renders each one with a distinct hatched pattern so the series stay visually distinguishable.

The spend chart footer summarizes the selected view by cost bucket, including total, input, output, cached, cache writes, and reasoning when the selected traffic includes those categories.

AI Gateway does not provide these capabilities. For current status, consult the ADP release notes.