# Use Iceberg Catalogs > For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [cloud-data-platform-full.txt](https://docs.redpanda.com/cloud-data-platform-full.txt) --- title: Use Iceberg Catalogs latest-operator-version: v26.1.4 latest-console-tag: v3.7.3 latest-connect-version: 4.93.0 latest-redpanda-tag: v26.1.9 docname: iceberg/use-iceberg-catalogs page-component-name: cloud-data-platform page-version: master page-component-version: master page-component-title: Cloud page-relative-src-path: iceberg/use-iceberg-catalogs.adoc page-edit-url: https://github.com/redpanda-data/cloud-docs/edit/main/modules/manage/pages/iceberg/use-iceberg-catalogs.adoc description: Learn how to access Redpanda topic data stored in Iceberg tables, using table metadata or a catalog integration. page-git-created-date: "2025-04-04" page-git-modified-date: "2026-05-26" --- To read from the Redpanda-generated [Iceberg table](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/about-iceberg-topics/), your Iceberg-compatible client or tool needs access to the catalog to retrieve the table metadata and know the current state of the table. The catalog provides the current table metadata, which includes locations for all the table’s data files. You can configure Redpanda to either connect to a REST-based catalog, or use a filesystem-based catalog. For production deployments, Redpanda recommends [using an external REST catalog](#rest) to manage Iceberg metadata. This enables built-in table maintenance, safely handles multiple engines and tools accessing tables at the same time, facilitates data governance, and maximizes data discovery. However, if it is not possible to use a REST catalog, you can [use the filesystem-based catalog](#object-storage) (`object_storage` catalog type), which does not require you to maintain a separate service to access the Iceberg data. In either case, you use the catalog to load, query, or refresh the Iceberg table as you produce to the Redpanda topic. See the documentation for your query engine or Iceberg-compatible tool for specific guidance on adding the Iceberg tables to your data warehouse or lakehouse using the catalog. After you have selected a catalog type at the cluster level and [enabled the Iceberg integration](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/about-iceberg-topics/#enable-iceberg-integration) for a topic, you cannot switch to another catalog type. ## [](#rest)Connect to a REST catalog > 📝 **NOTE** > > Redpanda connects to an Iceberg catalog that you provision and manage. Redpanda does not create or manage the catalog service, its databases, or any associated network configuration. Connect to an Iceberg REST catalog using the standard [REST API](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) supported by many catalog providers. Use this catalog integration type with REST-enabled Iceberg catalog services, such as [Databricks Unity](https://docs.databricks.com/en/data-governance/unity-catalog/index.html) and [Snowflake Open Catalog](https://other-docs.snowflake.com/en/opencatalog/overview). > 💡 **TIP** > > This section provides general guidance on using REST catalogs with Redpanda. For instructions on integrating with specific REST catalog services, see the following: > > - [AWS Glue Data Catalog](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/iceberg-topics-aws-glue/) > > - [Databricks Unity Catalog](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/iceberg-topics-databricks-unity/) > > - [Snowflake with Open Catalog](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/redpanda-topics-iceberg-snowflake-catalog/) ### [](#prerequisites)Prerequisites For BYOVPC clusters, you must: 1. Enable secrets management, which allows you to store and use secrets in your cluster’s Iceberg catalog authentication properties. Secrets management is enabled by default for AWS if you follow the guide to [creating a new BYOVPC cluster](https://docs.redpanda.com/cloud-data-platform/get-started/cluster-types/byoc/aws/vpc-byo-aws/). For GCP, follow the guides to enable secrets management for a [new BYOVPC cluster](https://docs.redpanda.com/cloud-data-platform/get-started/cluster-types/byoc/gcp/vpc-byo-gcp/) or an [existing BYOVPC cluster](https://docs.redpanda.com/cloud-data-platform/get-started/cluster-types/byoc/gcp/enable-secrets-byovpc-gcp/). 2. Ensure that your network security settings allow egress traffic from the Redpanda network to the catalog service endpoints. ### [](#limitations)Limitations The Iceberg integration for Redpanda Cloud supports multiple Iceberg catalogs across different cloud platforms, with progressive levels of release maturity. Each combination of cloud provider and catalog integration is tested and released independently. The following matrix shows the current status of Iceberg integrations across different cloud providers and catalogs. Check this matrix regularly as Redpanda Cloud continues to expand GA coverage for Iceberg topics. | | Databricks Unity Catalog | Snowflake Open Catalog | AWS Glue Data Catalog | Google BigQuery | | --- | --- | --- | --- | --- | | AWS | Supported | Beta | Beta | N/A | | GCP | Supported | Beta | N/A | Beta | | Azure | Beta | Beta | N/A | N/A | Other REST catalogs, such as Apache Polaris, Dremio Nessie (to be [merged with Polaris](https://www.dremio.com/newsroom/polaris-catalog-to-be-merged-with-nessie-now-available-on-github/)), and the Apache reference implementation, have been tested but are not regularly verified. For more information, contact [Redpanda Support](https://support.redpanda.com/hc/en-us/requests/new). ### [](#set-cluster-properties)Set cluster properties To connect to a REST catalog, set the following cluster configuration properties: - `[iceberg_catalog_type](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_catalog_type)`: `rest` - `[iceberg_rest_catalog_endpoint](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_rest_catalog_endpoint)`: The endpoint URL for your Iceberg catalog. You either manage this directly, or you have this managed by an external catalog service. > 📝 **NOTE** > > You must set `iceberg_rest_catalog_endpoint` at the same time that you set `iceberg_catalog_type` to `rest`. #### [](#configure-table-namespace)Configure table namespace Check if your REST catalog provider has specific requirements or recommendations for namespaces. For example, AWS Glue offers only a single global catalog per account, and each cluster that writes to the same Glue catalog must use a distinct namespace to avoid table name collisions. By default, Redpanda creates Iceberg tables in a namespace called `redpanda`. To use a unique namespace, configure the `[iceberg_default_catalog_namespace](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_default_catalog_namespace)` cluster property. You must set this property before enabling the Iceberg integration or at the same time. After you have enabled Iceberg, do not change this property value. #### [](#configure-authentication)Configure authentication To authenticate with the REST catalog, set the following cluster properties: - `[iceberg_rest_catalog_authentication_mode](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_rest_catalog_authentication_mode)`: The authentication mode to use for the REST catalog. Choose from `oauth2`, `aws_sigv4`, `bearer`, or `none` (default). You must use `aws_sigv4` for [AWS Glue Data Catalog](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/iceberg-topics-aws-glue/). Redpanda generally recommends using `oauth2` for REST catalogs. - For `oauth2`, also configure the following properties: - `[iceberg_rest_catalog_oauth2_server_uri](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_rest_catalog_oauth2_server_uri)`: The OAuth endpoint URI used to retrieve tokens for REST catalog authentication. If left unset, the deprecated catalog endpoint `/v1/oauth/tokens` is used as the token endpoint instead. - `[iceberg_rest_catalog_client_id](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_rest_catalog_client_id)`: The ID used to query the OAuth token endpoint for REST catalog authentication. - `[iceberg_rest_catalog_client_secret](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_rest_catalog_client_secret)`: The secret used with the client ID to query the OAuth token endpoint for REST catalog authentication. - For `bearer`, configure the `[iceberg_rest_catalog_token](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_rest_catalog_token)` property with your bearer token. Redpanda uses the bearer token unconditionally and does not attempt to refresh the token. Only use the bearer authentication mode for ad hoc or testing purposes. For REST catalogs that use self-signed certificates, also configure these properties: - `[iceberg_rest_catalog_trust](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_rest_catalog_trust)`: The contents of a certificate chain to trust for the REST catalog. - `[iceberg_rest_catalog_crl](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_rest_catalog_crl)`: The contents of a certificate revocation list for `iceberg_rest_catalog_trust`. See [Cluster Configuration Properties](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/) for the full list of cluster properties to configure for a catalog integration. ### [](#store-a-secret-for-rest-catalog-authentication)Store a secret for REST catalog authentication To store a secret that you can reference in your catalog authentication cluster properties, you must create the secret using `rpk` or the Data Plane API. Secrets are stored in the secret management solution of your cloud provider. Redpanda retrieves the secrets at runtime. For more information, see [Introduction to rpk](https://docs.redpanda.com/cloud-data-platform/manage/rpk/intro-to-rpk/) and [Cloud API Overview](https://docs.redpanda.com/api/doc/cloud-dataplane/topic/topic-cloud-api-overview). If you need to configure any of the following properties, you must set their values using secrets: - `iceberg_rest_catalog_client_secret` - `iceberg_rest_catalog_crl` - `iceberg_rest_catalog_token` - `iceberg_rest_catalog_trust` To create a new secret: #### rpk Run the following `rpk` command: ```bash rpk security secret create --name --value --scopes redpanda_cluster ``` Replace the placeholders with your own values: - ``: The name of the secret you want to add. The secret name is also its ID. Use only the following characters: `^[A-Z][A-Z0-9_]*$`. - ``: The value of the secret. #### Cloud API 1. Authenticate and make a `GET /v1/clusters/{id}` request to [retrieve the Data Plane API URL](https://docs.redpanda.com/cloud-data-platform/manage/api/cloud-dataplane-api/#get-data-plane-api-url) for your cluster. 2. Make a request to [`POST /v1/secrets`](https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-secretservice_createsecret). You must use a Base64-encoded secret. ```bash curl -X POST "https:///v1/secrets" \ -H 'accept: application/json'\ -H 'authorization: Bearer '\ -H 'content-type: application/json' \ -d '{"id":"","scopes":["SCOPE_REDPANDA_CLUSTER"],"secret_data":""}' ``` You must include the following values: - ``: The base URL for the Data Plane API. - ``: The API key you generated during authentication. - ``: The name of the secret you want to add. The secret name is also its ID. Use only the following characters: `^[A-Z][A-Z0-9_]*$`. - ``: The Base64-encoded secret. - This scope: `"SCOPE_REDPANDA_CLUSTER"`. The response returns the name and scope of the secret. You can now [reference the secret in your cluster configuration](#use-a-secret-in-cluster-configuration). ### [](#use-a-secret-in-cluster-configuration)Use a secret in cluster configuration To set the cluster property to use the value of the secret, use `rpk` or the Control Plane API. For example, to use a secret for the `iceberg_rest_catalog_client_secret` property, run: #### rpk ```bash rpk cluster config set iceberg_rest_catalog_client_secret '${secrets.}' ``` #### Cloud API Make a request to the [`PATCH /v1/clusters/`](https://docs.redpanda.com/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster) endpoint of the Control Plane API. ```bash curl -H "Authorization: Bearer " -X PATCH \ "https://api.cloud.redpanda.com/v1/clusters/" \ -H 'accept: application/json'\ -H 'content-type: application/json' \ -d '{"cluster_configuration": { "custom_properties": { "iceberg_rest_catalog_client_secret": "${secrets.}" } } }' ``` You must include the following values: - ``: The ID of the Redpanda cluster. - ``: The API key you generated during authentication. - ``: The name of the secret you created earlier. ### [](#example-rest-catalog-configuration)Example REST catalog configuration Suppose you configure the following Redpanda cluster properties for connecting to a REST catalog: ```yaml iceberg_catalog_type: rest iceberg_rest_catalog_endpoint: http://catalog-service:8181 iceberg_rest_catalog_authentication_mode: oauth2 iceberg_rest_catalog_oauth2_server_uri: iceberg_rest_catalog_client_id: iceberg_rest_catalog_client_secret: ``` If you use Apache Spark as a processing engine, your Spark configuration might look like the following. This example uses a catalog named `streaming`: ```spark spark.sql.catalog.streaming = org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.streaming.type = rest spark.sql.catalog.streaming.uri = http://catalog-service:8181 spark.sql.catalog.streaming.warehouse = # You may need to configure additional properties based on your object storage provider. # See https://iceberg.apache.org/docs/latest/spark-configuration/#catalog-configuration and https://spark.apache.org/docs/latest/configuration.html # For example, for AWS S3: # spark.sql.catalog.streaming.io-impl = org.apache.iceberg.aws.s3.S3FileIO # spark.sql.catalog.streaming.s3.endpoint = http:// ``` > 📝 **NOTE** > > Redpanda recommends setting credentials in environment variables so Spark can securely access your Iceberg data in object storage. For example, for AWS, use `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. The Spark engine can use the REST catalog to automatically discover the topic’s Iceberg table. Using Spark SQL, you can query the Iceberg table directly by specifying the catalog name, the namespace, and the table name: ```sql SELECT * FROM streaming.redpanda.; ``` The Iceberg table name is the name of your Redpanda topic. If you configured a different namespace using `[iceberg_default_catalog_namespace](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_default_catalog_namespace)`, replace `redpanda` with your configured namespace. > 💡 **TIP** > > You may need to explicitly create a table for the Iceberg data in your query engine. For an example, see [Query Iceberg Topics using Snowflake and Open Catalog](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/redpanda-topics-iceberg-snowflake-catalog/). ## [](#object-storage)Integrate filesystem-based catalog (`object_storage`) By default, Iceberg topics use the filesystem-based catalog (`[iceberg_catalog_type](https://docs.redpanda.com/cloud-data-platform/reference/properties/cluster-properties/#iceberg_catalog_type)` cluster property set to `object_storage`). Redpanda stores the table metadata in [HadoopCatalog](https://iceberg.apache.org/docs/latest/java-api-quickstart/#using-a-hadoop-catalog) format in the same object storage bucket or container as the data files. If using the `object_storage` catalog type, you provide the object storage URI of the table’s `metadata.json` file to an Iceberg client so it can access the catalog and data files for your Redpanda Iceberg tables. > 📝 **NOTE** > > The `metadata.json` file points to a specific Iceberg table snapshot. In your query engine, you must update your tables whenever a new snapshot is created so that they point to the latest snapshot. See the [official Iceberg documentation](https://iceberg.apache.org/docs/latest/maintenance/) for more information, and refer to the documentation for your query engine or Iceberg-compatible tool for specific guidance on Iceberg table update or refresh. ### [](#example-filesystem-based-catalog-configuration)Example filesystem-based catalog configuration To configure Apache Spark to use a filesystem-based catalog, specify at least the following properties: ```spark spark.sql.catalog.streaming = org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.streaming.type = hadoop # URI for table metadata: AWS S3 example spark.sql.catalog.streaming.warehouse = s3a:///redpanda-iceberg-catalog # You may need to configure additional properties based on your object storage provider. # See https://iceberg.apache.org/docs/latest/spark-configuration/#spark-configuration and https://spark.apache.org/docs/latest/configuration.html # For example, for AWS S3: # spark.hadoop.fs.s3.impl = org.apache.hadoop.fs.s3a.S3AFileSystem # spark.hadoop.fs.s3a.endpoint = http:// # spark.sql.catalog.streaming.s3.endpoint = http:// ``` > 📝 **NOTE** > > Redpanda recommends setting credentials in environment variables so Spark can securely access your Iceberg data in object storage. For example, for AWS, use `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. Depending on your processing engine, you may need to also create a new table to point the data lakehouse to the table location. ### [](#specify-metadata-location)Specify metadata location The base path for the filesystem-based catalog if using the `object_storage` catalog type is `redpanda-iceberg-catalog`. > 💡 **TIP** > > For an end-to-end example of using the filesystem-based catalog to access Iceberg topics, see the [Getting Started with Iceberg Topics on Redpanda BYOC](https://www.redpanda.com/blog/iceberg-topics-redpanda-cloud-byoc-setup) blog post. ## [](#next-steps)Next steps - [Query Iceberg Topics](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/query-iceberg-topics/) - [Query Iceberg Topics using AWS Glue](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/iceberg-topics-aws-glue/) - [Query Iceberg Topics using Databricks and Unity Catalog](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/iceberg-topics-databricks-unity/) - [Query Iceberg Topics using Snowflake and Open Catalog](https://docs.redpanda.com/cloud-data-platform/manage/iceberg/redpanda-topics-iceberg-snowflake-catalog/)