Cloud

CREATE STORAGE

The CREATE STORAGE statement creates a named connection to external object storage such as Amazon S3 or Google Cloud Storage (GCS).

In Redpanda Cloud, the storage connection and Iceberg catalog used to query Iceberg-enabled topics are configured and managed automatically when you enable Redpanda SQL on a cluster that has an Iceberg REST catalog configured. You don’t run these statements yourself for that workflow. This page documents the statement’s syntax and options so you can understand what Redpanda Cloud configures. See Query Iceberg-enabled topics.

Syntax

CREATE STORAGE [IF NOT EXISTS] storage_name
TYPE = S3 | GCS
[WITH (option = 'value' [, ...])];

  • storage_name: Name for the new storage connection.

  • TYPE: Storage type. Redpanda SQL supports S3 and GCS.

  • IF NOT EXISTS: Optional. Prevents an error if a storage connection with the same name already exists.

  • WITH (…​): Optional. Omit it to create a storage connection with no credentials or configuration options set (for example, to configure them later with ALTER STORAGE, or to rely on default credential discovery such as Application Default Credentials for GCS). If you do include WITH, see the notes under each table for which options are required.

S3 options

Option Type Required Description

region

STRING

See note

Cloud region for the storage bucket (for example, us-west-2).

endpoint

STRING

See note

Custom S3-compatible endpoint URL. Use this for S3-compatible storage providers other than AWS.

access_key_id

STRING

See note

AWS access key ID.

secret_access_key

STRING

See note

AWS secret access key.

session_token

STRING

No

AWS session token for temporary credentials. Requires access_key_id and secret_access_key to also be set.

url

STRING

No

S3 bucket URL, in the form s3://<bucket>/<path>. If omitted, the connection holds credentials only and has no associated bucket.

path_style

BOOLEAN

No

Whether to use path-style addressing (https://endpoint/bucket) instead of virtual-hosted-style addressing. Defaults to false.

use_http

BOOLEAN

No

Whether to connect over plain HTTP instead of HTTPS. Defaults to false.

  • You must set region or endpoint (or both). At least one is required.

  • access_key_id and secret_access_key must be set together, or both omitted. If you omit both, Redpanda SQL falls back to the AWS default credential chain (environment variables, shared configuration, STS web identity, or an EC2/ECS instance-profile role).

GCS options

Option Type Required Description

url

STRING

No

GCS bucket URL, in the form gs://<bucket>/<path>. If omitted, the connection holds credentials only and has no associated bucket.

service_account_key

STRING

No

Contents of a GCP service account key JSON file. If omitted, Redpanda SQL authenticates using Application Default Credentials (ADC), which includes Workload Identity Federation on Google Kubernetes Engine (GKE).

endpoint

STRING

No

Override the default GCS endpoint.

Examples

Create an S3 storage connection

CREATE STORAGE archive_storage
TYPE = S3
WITH (
  region = 'us-west-2',
  access_key_id = 'AKIAIOSFODNN7EXAMPLE',
  secret_access_key = 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY'
);

Create an S3 storage connection using the default credential chain

Omit access_key_id and secret_access_key to authenticate using the AWS default credential chain instead of static keys:

CREATE STORAGE default_chain_storage
TYPE = S3
WITH (
  region = 'us-west-2'
);

Create a GCS storage connection with a service account key

CREATE STORAGE gcs_key_storage
TYPE = GCS
WITH (
  url = 'gs://archive-bucket/redpanda-sql',
  service_account_key = '{"type": "service_account", "project_id": "my-project", ...}'
);

Create a GCS storage connection using Application Default Credentials

Omit service_account_key to authenticate with Application Default Credentials (ADC). On GKE, this includes Workload Identity Federation.

CREATE STORAGE gcs_adc_storage
TYPE = GCS
WITH (
  url = 'gs://archive-bucket/redpanda-sql'
);

Create a storage connection without a WITH clause

Omit WITH entirely to create a storage connection with no options set. Use ALTER STORAGE to add credentials or configuration later:

CREATE STORAGE placeholder_storage
TYPE = S3;