Set Up Gemini Enterprise Agent Platform as an LLM Provider
Register Google’s Gemini Enterprise Agent Platform (formerly Vertex AI) as an LLM provider in Redpanda Agentic Data Plane to run Gemini and Claude models through your own Google Cloud Platform (GCP) project. Google bills the usage to that project, and AI Gateway authenticates with a service account key from it.
After reading this page, you will be able to:
-
Create a GCP service account that has only the
roles/aiplatform.userrole, and a key for it -
Enable Claude models and Claude structured outputs in your GCP project and organization
-
Register Gemini Enterprise Agent Platform as an LLM provider in Agentic Data Plane at a location that serves your models
Some Google pages and SDKs still use the old name, and the API’s service name is still aiplatform.googleapis.com. In Agentic Data Plane, the provider type is Gemini Enterprise Agent Platform.
|
Prerequisites
-
A GCP project with billing enabled. See Enable, disable, or change billing for a project.
-
The
gcloudCLI, signed in as a user with theroles/ownerrole on that project. See Install the gcloud CLI. -
The Gemini Enterprise Agent Platform API enabled on the project. Replace
<project-id>with your project ID:gcloud services enable aiplatform.googleapis.com --project=<project-id> -
For Claude models: the
roles/consumerprocurement.entitlementManagerrole on the project. For Claude Fable models, also theaiplatform.consents.updatepermission, which theroles/aiplatform.adminrole includes. See Enable Claude models in Model Garden. -
For Claude models in a GCP organization: access to someone with the
roles/orgpolicy.policyAdminrole on the organization. See Allow Claude structured outputs. -
Access to the Agentic Data Plane UI.
Before you register the provider, set up each item in this table in GCP. The Model Garden and organization policy items apply only to Claude models. Model Garden is the catalog of models in the Google Cloud console. A missing item causes the failure in its What fails without it column.
| What you set up | Section | What fails without it |
|---|---|---|
A service account with the single role |
Every call returns |
|
A service account key |
You have no credential to give Agentic Data Plane. |
|
Each Claude model enabled in Model Garden |
Calls to that Claude model return |
|
An organization policy entry that allows Claude structured outputs |
Claude requests that ask for structured output return |
Create a service account
The service account needs one role, roles/aiplatform.user, which covers both Gemini and Claude inference. Don’t grant the service account any other role.
The role also allows much more than inference, such as creating training, tuning, and pipeline jobs that Google bills to the project. See Gemini Enterprise Agent Platform roles and permissions. A leaked key can do all of that, so keep the key file safe. If the key leaks, see Respond to a leaked key.
| Don’t reuse an existing service account. A dedicated one lets you rotate or revoke the key without affecting other workloads. |
Create the service account and grant it the role:
gcloud iam service-accounts create redpanda-vertex \
--project=<project-id> \
--display-name="Redpanda AI Gateway"
gcloud projects add-iam-policy-binding <project-id> \
--member="serviceAccount:redpanda-vertex@<project-id>.iam.gserviceaccount.com" \
--role="roles/aiplatform.user"
Create a service account key
A service account key is the only credential a Gemini Enterprise Agent Platform provider accepts. Agentic Data Plane doesn’t read Application Default Credentials and can’t use Workload Identity Federation.
Create a JSON key for the service account:
gcloud iam service-accounts keys create redpanda-vertex-key.json \
--project=<project-id> \
--iam-account=redpanda-vertex@<project-id>.iam.gserviceaccount.com
Google enforces the constraints/iam.managed.disableServiceAccountKeyCreation constraint by default for organizations created on or after May 3, 2024, and might enforce it for some organizations created between February and April 2024. If the command fails because of that constraint or the older constraints/iam.disableServiceAccountKeyCreation constraint, ask someone with the roles/orgpolicy.policyAdmin role to override the constraint for your project, then rerun the command. If your organization doesn’t allow an override for the project, you can’t connect Agentic Data Plane to Gemini Enterprise Agent Platform.
| The file holds a private key. Keep it only until you store it in Agentic Data Plane (see Register Gemini Enterprise Agent Platform as an LLM provider), then delete it. |
Enable Claude models in Model Garden
If you use only Gemini models, skip this section.
You enable each Claude model separately in your project. Enabling one Claude model doesn’t enable the others, and enabling a model in one project doesn’t enable it in another.
To enable models, you need the roles/consumerprocurement.entitlementManager role on the project. For a Claude Fable model, you also need the aiplatform.consents.update permission to accept the Advanced AI Safety Addendum, which the roles/aiplatform.admin role includes. The data sharing call for a Claude Fable model needs the aiplatform.endpoints.setPublisherModelConfig permission. Google’s predefined roles don’t include it, so a project owner makes that call. Grant these roles to whoever enables the models, not to the redpanda-vertex service account, which needs only roles/aiplatform.user.
Before you enable a model, check these terms and requirements:
- Enable form
-
The Enable form asks for company and use-case details, and has you accept Anthropic’s and Google Cloud Marketplace’s terms of service. These terms are between you, Google, and Anthropic. Redpanda doesn’t see this data and isn’t a party to the terms.
- Advanced AI Safety Addendum
-
Claude Fable models require Google’s Advanced AI Safety Addendum. The addendum lets Google store prompts and outputs for up to 30 days and review them with automated tools and, as needed, human review, only to monitor for abuse. Accepting it is a data-handling decision for your organization.
- Resellers
-
If a reseller that Anthropic prohibits manages your billing account, you can’t enable Claude models. Contact your reseller.
- Procurement service
-
If your organization restricts service usage with an organization policy, the policy must allow the
cloudcommerceconsumerprocurement.googleapis.comservice, or you can’t complete the Enable form. - Model access policy
-
If your organization restricts Model Garden models with the
vertexai.allowedModelsconstraint, the policy must allow partner models such as Claude.
| This configuration refers to a third-party product. Specific UI elements may not be up to date. |
For each Claude model you want to use:
-
In the Google Cloud console, open Model Garden and find the model’s card, for example Anthropic’s Claude Haiku 4.5 on Google Cloud.
-
For a Claude Fable model, check your project’s Fable quota first, because a project can have no Fable quota (see Troubleshooting). Then accept the Advanced AI Safety Addendum for the project in Model Garden. You accept it one time for each project, before you enable the first Claude Fable model. See Google’s Abuse monitoring page.
-
If you buy a Claude Fable model through a Private Offer, accept the Private Offer in Google Cloud Marketplace, then use the
enableModelAPI method to enable the model at each location you use (see Choose a location). Otherwise, click Enable and complete the form. For the Private Offer steps, see Google’s Claude Fable 5.1 on Google Cloud page. -
For a Claude Fable model, turn on data sharing with Anthropic. Without it, calls to the model return
403 PERMISSION_DENIEDnamingdata_sharing_enabled_provider. The setting applies to one model at one location, so make the call for each Claude Fable model at each location you use (see Choose a location). Google accepts the call only after someone accepts the Advanced AI Safety Addendum for the project.For example, at the
globallocation, replace<model-id>with a Claude Fable model ID, such asclaude-fable-5-1:curl -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ -d '{"publisherModelConfig": {"dataSharingEnabledProvider": "ANTHROPIC"}, "updateMask": "dataSharingEnabledProvider"}' \ "https://aiplatform.googleapis.com/v1beta1/projects/<project-id>/locations/global/publishers/anthropic/models/<model-id>:setPublisherModelConfig"At the
usoreulocation, the host isaiplatform.<location>.rep.googleapis.com, andlocations/globalin the path becomeslocations/<location>. KeepupdateMaskin the request. Without it, the call replaces the model’s whole configuration and clears any request-response logging set on it. See Google’s setPublisherModelConfig reference.The setting might not apply the moment the call returns. Before you test the model, send a
GETrequest to the same URL with:fetchPublisherModelConfigin place of:setPublisherModelConfig, and check that the response shows"dataSharingEnabledProvider": "ANTHROPIC":curl -H "Authorization: Bearer $(gcloud auth print-access-token)" \ "https://aiplatform.googleapis.com/v1beta1/projects/<project-id>/locations/global/publishers/anthropic/models/<model-id>:fetchPublisherModelConfig"While data sharing is on, Google shares the model’s logged requests and responses with Anthropic in real time. See Google’s Log and share requests and responses page.
If your project is inside a VPC Service Controls perimeter, Google blocks data sharing by default. To turn it on, file a support case with Google.
For Google’s full procedure, see Request predictions with Claude models.
Allow Claude structured outputs
If you use only Gemini models, or your project isn’t in a GCP organization, skip this section.
Google’s vertexai.allowedPartnerModelFeatures constraint controls Claude structured outputs. For any project in an organization, it denies them by default. Until an organization policy administrator allows structured outputs, agents that ask a Claude model for structured output fail with a 400 FAILED_PRECONDITION error, shown here with identifiers redacted:
400 FAILED_PRECONDITION
Organization Policy constraint constraints/vertexai.allowedPartnerModelFeatures violated for `projects/<redacted>` attempting to use a disallowed feature structured_outputs for Partner model claude-haiku-4-5. Please contact your organization administrator to fix this violation by adding `publishers/anthropic/models/claude-haiku-4-5:structured_outputs` to the allowed values. For more info, see https://docs.cloud.google.com/vertex-ai/generative-ai/docs/control-model-access
The error comes from Google. To fix it, add the allowed value that the message names.
The same constraint also turns off Claude web search by default. See Google’s Web search page.
Applying an organization policy for this constraint requires the roles/orgpolicy.policyAdmin role. A project owner usually doesn’t have it. If you don’t have this role, ask someone who has it to apply the policy for you.
To allow Claude structured outputs for your project:
-
Enable the Organization Policy API on the project. The
gcloud org-policiescommands need it:gcloud services enable orgpolicy.googleapis.com --project=<project-id> -
Check your project’s effective policy:
gcloud org-policies describe vertexai.allowedPartnerModelFeatures \ --project=<project-id> --effectiveIf the output shows any of these, skip the remaining steps. In this list,
<model-name>is the model ID, such asclaude-haiku-4-5.-
allowAll: true -
publishers/anthropicinallowedValues -
publishers/anthropic/models/<model-name>orpublishers/anthropic/models/<model-name>:structured_outputsinallowedValues, for each Claude model you use
-
-
Check whether the project has its own policy for this constraint. This time, leave out
--effective:gcloud org-policies describe vertexai.allowedPartnerModelFeatures \ --project=<project-id>If the command returns
NOT_FOUND, the project has no policy of its own, and the file in the next step needs no other rules.If the output lists rules, copy each one under
spec.rulesin the file in the next step. A rule holds only one ofvalues,allowAll, ordenyAll, so add your entries to theallowedValueslist of the rule that hasvalues. If a rule hasdenyAll: true, your organization blocks every value on purpose, so agree with whoever set it before you replace it with avaluesrule. Keep the policy’s owninheritFromParentvalue. Changingfalsetotruealso allows every value that the parent policies allow. See Google’s PolicyRule reference.The set-policycommand replaces the policy on the project with the policy in the file. The command removes any rule that the file leaves out. -
Save this policy as
policy.yaml, with one entry for each Claude model and feature you use:name: projects/<project-id>/policies/vertexai.allowedPartnerModelFeatures spec: inheritFromParent: true rules: - values: allowedValues: - publishers/anthropic/models/claude-haiku-4-5:structured_outputs - publishers/anthropic/models/claude-sonnet-5:structured_outputsReplace the example entries with your own. If your agents use Claude’s web search tool, also add
publishers/anthropic/models/<model-name>:web_searchfor each model. The single entrypublishers/anthropicallows every feature for every Claude model, web search included, so use it only if you mean to allow all of them.With
inheritFromParent: true, the project keeps the values that its folders and organization already allow. A value that a parent policy denies stays denied, because a denied value overrides an allowed one. See Google’s Understanding hierarchy evaluation page. -
Apply the policy:
gcloud org-policies set-policy policy.yaml -
Rerun the
describecommand with--effectiveto confirm that the output lists your entries.A policy change can take 15 minutes or longer to apply. Until it applies everywhere, the same request can succeed on one try and fail with the
400 FAILED_PRECONDITIONerror on the next.
To allow the entries for every project in the organization instead, use organizations/<organization-id> in place of projects/<project-id> in the name field, and leave out inheritFromParent, because an organization has no parent policy. To find your numeric organization ID, run gcloud projects get-ancestors <project-id> and use the ID of the entry whose type is organization. The set-policy command replaces the organization’s policy too, so first check it with --organization=<organization-id> in place of --project=<project-id>, and copy its rules into the file.
Choose a location
A Gemini Enterprise Agent Platform provider serves all its models from one location, so the location determines which models you can select:
-
The
globallocation serves every model that AI Gateway supports and is the best default. -
The
usandeulocations are multi-region locations. To learn what each one guarantees about where Google stores and processes data, see Data residency in the Google Cloud documentation. -
A named region, such as
us-east5, serves fewer models. Google offers Claude Opus 4.7 and every Claude model released after it, such as Claude Sonnet 5, Claude Opus 5, and the Claude Fable models, only atglobal,us, andeu. At a named region, the Models section doesn’t list these models, and a call to one of them there returns429. If data sharing is off for a Claude Fable model at that region, the model can return403namingdata_sharing_enabled_providerinstead. Don’t turn on data sharing there. Create the provider atglobal,us, oreu. See Google’s Deployments and endpoints page.
To use models from two locations, create a provider for each location.
Register Gemini Enterprise Agent Platform as an LLM provider
Create the provider in Agentic Data Plane with the key you created in Create a service account key.
-
Open LLM providers in the sidebar and click Add provider.
-
Enter a Display name, such as
GCP models. Agentic Data Plane derives the resource ID in the proxy URL from it. -
Select Gemini Enterprise Agent Platform as the provider type.
-
In the Credential section, set the Service account key field. If the key is already in the secret store, click Existing and select its secret. Otherwise, create a secret:
-
Click New and enter a secret name, such as
VERTEX_SERVICE_ACCOUNT_KEY. Agentic Data Plane converts the name to uppercase, replaces characters it doesn’t allow with_, and removes leading characters that aren’t letters. -
Paste the whole contents of
redpanda-vertex-key.jsoninto Secret value, or drag the file onto the area labeled Or drag and drop a .json file. The value must be the JSON key file exactly as Google created it. -
Click Create secret.
Agentic Data Plane rejects a value that isn’t a service account key, such as a JSON key whose
typeisn’tservice_account. It stores the secret in its secret store and lists it under Provider keys. -
-
Delete the local key file:
rm redpanda-vertex-key.json -
In the Connection settings section, enter your Project ID, such as
my-gcp-project. Use the project ID, not the numeric project number. -
Select a Location. The list shows how many models each location serves. Choose
globalunless you need a specific location. See Choose a location. -
In the Models section, select the models to allow. The form lists only the models that AI Gateway supports at the location. It preselects the Gemini models there, and Claude models start unselected. If you choose another location, the selection resets to that location’s Gemini models. The section shows how many models are selected, such as
9 of 23 selected. Select at least one model, because Agentic Data Plane doesn’t create the provider without one. Agents and developers can use only the models you select. For Claude, select only models that you enabled (see Enable Claude models in Model Garden) and that Google offers at your location (see Choose a location). -
Optional: In the Connection settings section, click Test connection in the Verify connection box. The check sends a one-token request to each selected model. If you selected no models, it checks one model served at the location.
Google bills these requests to your project. If a request fails, the check tries that model up to two more times, but not after a 400,401, or403error.The result tells you which models answered:
-
A successful result, such as
Connected (840ms)., means every selected model answered. -
A result such as
2 of 3 models answered.names each model that didn’t answer and why. Most often, the model that didn’t answer is a Claude model you haven’t enabled yet, listed as<model-id>: request access in Model Garden. -
A result that starts with
No model answered.names the cause for each model. Check the service account role (see Create a service account), the Project ID you entered, and the Claude models you enabled (see Enable Claude models in Model Garden). -
A result that starts with
mint Google access tokenmeans Google refused the key, for example because someone deleted or disabled it. See Troubleshooting.
-
-
Click Create provider.
-
Optional: On the provider’s Playground tab, send a prompt to a Gemini model and to each Claude model you selected.
-
Optional: To rerun the check on the saved provider, open its Settings tab and click Test connection in the Connection section.
Agentic Data Plane lists the key under Provider keys, where you can replace it later. The provider’s Settings tab links to that page.
Replace the service account key
Replacing the value under Provider keys doesn’t revoke the old key in GCP. The old key stays valid, and service account keys don’t expire by default. Disabling a key doesn’t revoke access tokens already issued from it, and those tokens last up to one hour by default. See Service account credentials. You can turn a disabled key back on, but you can’t restore a deleted key. See Disable and enable service account keys.
A saved provider reloads its key about every 5 minutes, and its first check after that can still use the old key. If the new value fails to load, the provider keeps the old key. Its own Test connection therefore doesn’t prove that a new key works. Check a new key on the create form instead, which reads the secret fresh.
These commands list the keys of the redpanda-vertex service account and disable one. The DISABLED column reads True for a disabled key. To turn a key back on, or to delete it, run the second command with enable or delete in place of disable:
gcloud iam service-accounts keys list --managed-by=user \
--project=<project-id> \
--iam-account=redpanda-vertex@<project-id>.iam.gserviceaccount.com
gcloud iam service-accounts keys disable <key-id> \
--project=<project-id> \
--iam-account=redpanda-vertex@<project-id>.iam.gserviceaccount.com
Put the new key in place
The Rotate the key on a schedule and Respond to a leaked key procedures both use these steps.
-
Create a new key with the command in Create a service account key. It writes
redpanda-vertex-key.jsonagain. -
Under Provider keys, open the menu on the key’s row and click Replace value…. Set Value to Multiline, paste the whole contents of the new
redpanda-vertex-key.json, and click Save changes. -
Click Add provider on the LLM providers page and select Gemini Enterprise Agent Platform. Enter the same project and location, select the key’s existing secret under Service account key, select the same models as the saved provider, and click Test connection. Then leave the form without creating the provider.
A successful result, or a result such as
2 of 3 models answered., means the new key works. In the second case, the models that the result doesn’t name answered with the new key. If no model answered, look up the error in Troubleshooting, paste the newredpanda-vertex-key.jsonagain under Provider keys, and repeat this step.
Rotate the key on a schedule
-
Follow Put the new key in place.
-
List the keys. The new key is the one whose ID is the
private_key_idin the newredpanda-vertex-key.json.Any other provider whose Project ID is this project might still use an old key, in this environment or another one, even when the list shows only one other key. Two providers can each hold a copy of the same key in their own secrets. If more than one other key isn’t disabled, a tool outside Agentic Data Plane might also use one. Before you disable anything, move every such provider to the new key:
-
In this environment, for each other provider whose Project ID is this project, open its Settings tab, switch Service account key to this provider’s secret, and click Save changes. Wait half a minute, then click Test connection. If the check fails, look up the error in Troubleshooting, or switch the provider back. Its old secret still holds its old key, which is still enabled.
-
In another environment, a provider can’t select this environment’s secret. For each provider there whose Project ID is this project, replace its key’s value with the new
redpanda-vertex-key.jsonand check it on that environment’s create form, as steps 2 and 3 of Put the new key in place show.
-
-
When every provider on this project uses the new key, and nothing outside Agentic Data Plane uses another key, disable every key whose ID isn’t the new key’s. Don’t delete them yet.
-
Wait at least one hour. Anything that still uses an old key can keep working for up to an hour and then fail. If something fails, turn its key back on and move it to the new key. When nothing has failed for an hour, delete each key you disabled, one command for each key ID.
-
Delete the new key’s local file:
rm redpanda-vertex-key.json
Respond to a leaked key
A leaked key can do everything the roles/aiplatform.user role allows, and it stays valid until you disable it.
-
List the keys and disable the leaked one right away. It’s the key whose ID is the
private_key_idin the leaked key file. If you can’t read that file, disable every key in the list. Every provider that uses a disabled key then fails until it has the new key. -
To cut off the access tokens already issued from the key, also disable the service account. Every provider that uses it then fails until you turn it back on. Wait one hour so those tokens expire, then run the same command with
enablein place ofdisable:gcloud iam service-accounts disable redpanda-vertex@<project-id>.iam.gserviceaccount.com \ --project=<project-id> -
Check what the key did. In the project’s audit logs, find the calls made as
redpanda-vertex@<project-id>.iam.gserviceaccount.comsince the leak, and cancel or delete any jobs, endpoints, or other resources it created. See Google’s Respond to compromised Google Cloud credentials page. Google always logs calls that create jobs or endpoints. It logs model calls only if Data Access audit logs were already on for the project, so a log with no model calls doesn’t prove that nobody used the key. See Google’s audit logging page. -
Follow Put the new key in place.
-
Move each other provider whose Project ID is this project to the new key, in this environment and in every other one, as step 2 of Rotate the key on a schedule shows. Its old key might be one you disabled, so don’t switch a provider back, and never turn a disabled key back on. If a check still fails, look up the error in Troubleshooting.
-
Delete every key you disabled as soon as a check proves the new key, without waiting an hour. Never turn one of them back on. See Google’s Best practices for managing service account keys.
-
Delete the new key’s local file:
rm redpanda-vertex-key.json
Troubleshooting
AI Gateway replaces your project ID and service account email with <redacted> before it passes Google’s error messages to the caller. It also replaces your project number where the message labels it as one, such as in the resource name projects/<redacted>.
| Symptom | What to check |
|---|---|
Running |
Your organization enforces the constraint. Ask an organization policy administrator to override the constraint for your project. Without a key, you can’t connect Agentic Data Plane to Gemini Enterprise Agent Platform. See Create a service account key. |
|
The Claude model isn’t enabled in Model Garden for this project. See Enable Claude models in Model Garden. Google returns the same message for a model ID it doesn’t recognize or a model that the location doesn’t serve, so if the model is enabled, check the model ID and the location. At the |
|
The model is enabled, but Google doesn’t serve it at the provider’s location. Create the provider at |
|
The model isn’t one of the provider’s selected models, so AI Gateway refuses the request before it calls Google. Use a selected model, or add the model to the provider. |
|
The service account doesn’t have the |
|
The Gemini Enterprise Agent Platform API is disabled on the project. See Prerequisites. |
|
Data sharing with Anthropic isn’t on for that model at the provider’s location. See step 4 of Enable Claude models in Model Garden. |
|
Your organization policy denies the Claude feature that the message names, such as |
|
Your project has no quota for that model at that region. For Claude Opus 4.7 and every later Claude model, this is expected, because Google offers them only at |
|
Your project has no Fable quota available at that location. A project can have no Fable quota at all. In the Google Cloud console, open Quotas and system limits and filter by |
|
The location in the request URL differs from the provider’s location. Send requests to the provider’s own location, or create a second provider for the other location. On the second provider, select the first provider’s key secret under Service account key. |
Saving the provider fails with a message that a model is not served at the location |
A selected model isn’t served at the chosen location. The message lists the locations that serve each model. Choose one of them, or |
Test connection fails with a message that the location serves no Vertex AI model, or that no model is served at the location |
You didn’t select a model, and the chosen location serves only retired models, or none that AI Gateway supports. Choose |
Test connection fails right away with a credential error, such as a message that starts with |
The secret doesn’t hold a valid service account key, or someone deleted or disabled the key in GCP. If you disabled the old key in a planned rotation, turn it back on with the same command, |
For other errors that can affect any provider, such as a missing secret reference, see the provider troubleshooting table.