Set Iceberg Table Names and Namespaces
|
This feature requires an enterprise license. To get a trial license key or extend your trial period, generate a new trial license key. To purchase a license, contact Redpanda Sales. If Redpanda has enterprise features enabled and it cannot find a valid license, restrictions apply. |
Set a custom Iceberg table name and namespace for each topic. Without them, Redpanda names the table after the topic and creates it in the namespace that the iceberg_default_catalog_namespace cluster property sets.
After reading this page, you will be able to:
-
Choose a table name and namespace that meet the naming rules
-
Set a custom Iceberg table name and namespace for a topic
-
Check the table name and namespace that a topic uses
Prerequisites
-
Redpanda version 26.3 or later.
-
Iceberg enabled on the cluster. See Enable Iceberg integration.
-
rpkinstalled. See Install or Update rpk.
Choose a table name and namespace
Before you set a custom table name or namespace, check that it follows these rules:
-
Unique tables: Each Iceberg-enabled topic must write to a different table. Redpanda rejects a request to create a topic or change its configuration if the request gives two Iceberg-enabled topics the same namespace and table name. The error message contains
iceberg table identifier <identifier> is already used by topic <topic-name>.Redpanda also rejects a change to the
iceberg_default_catalog_namespaceoriceberg_topic_name_dot_replacementcluster property if the change gives two topics the same table. The error message containsiceberg table identifier <identifier> would be shared by topics <topic-a> and <topic-b>.Redpanda checks for duplicate tables on a best-effort basis. Two requests that run at the same time can both pass the check and give two topics the same table.
-
Valid values: The table name cannot be an empty string. The namespace cannot be an empty string, and each level between dots must contain at least one character. For example, Redpanda rejects the namespaces
analytics..web,.analytics, andanalytics.. -
Catalog limits: Your catalog can limit table names and namespaces. For example, Google Cloud Platform (GCP) Lakehouse does not accept table names that contain dots, and
iceberg_topic_name_dot_replacementdoes not apply to custom table names. If the namespace does not exist, Redpanda tries to create it.Some catalogs require you to create the namespace first, for example an existing catalog in Databricks Unity Catalog. See the page for your catalog in Integrate with REST Catalogs.
If the catalog rejects the table name or namespace, translation for the topic does not make progress. Redpanda retries every few seconds, logs each failure, and increments the catalog error metrics, such as
redpanda_iceberg_rest_client_num_create_table_requests_failedandredpanda_iceberg_rest_client_num_create_namespace_requests_failed. Check that your catalog accepts the name and namespace before you produce data to the topic. -
Dead-letter queue (DLQ) table: Redpanda creates the DLQ table in the same namespace as the main table. The DLQ table name is the custom table name with the
iceberg_dlq_table_suffixsuffix, for exampleevents~dlq. -
Topic deletion: When you delete the topic, Redpanda uses the custom table name and namespace to find the tables to drop. The
redpanda.iceberg.deletetopic property controls whether Redpanda drops the main table and the DLQ table. If you set it tofalse, Redpanda keeps both tables.
Set a custom table name and namespace
Use the following topic properties:
-
redpanda.iceberg.table.name: The name of the Iceberg table. Redpanda uses this value exactly as you set it. Theiceberg_topic_name_dot_replacementcluster property applies only to table names that Redpanda derives from the topic name. For example, if you set the table name tocustom.name, Redpanda does not replace the dot. -
redpanda.iceberg.table.namespace: The namespace of the Iceberg table. To set a namespace with more than one level, use dots to separate the levels, for exampleanalytics.web.
You can set one property or both. If you do not set a property, Redpanda uses the default for that part of the table identifier.
|
Set these properties before Redpanda writes data from the topic to Iceberg. Do not change them afterward. |
Redpanda does not block a change to these properties. If you change either property after Redpanda writes data to Iceberg, Redpanda writes new data to a new table. Redpanda stops tracking the existing table, and that table does not get new data.
For example, the following command creates a topic named clickstream and writes its data to the Iceberg table analytics.web.events:
rpk topic create clickstream \
-c redpanda.iceberg.mode=value_schema_latest \
-c redpanda.iceberg.table.name=events \
-c redpanda.iceberg.table.namespace=analytics.web
TOPIC STATUS
clickstream OK
To set the properties on an existing topic, use rpk topic alter-config:
rpk topic alter-config <topic-name> --set redpanda.iceberg.table.name=<table-name> --set redpanda.iceberg.table.namespace=<namespace>
Check the table name and namespace
To check the table name and namespace for a topic, do one of the following:
-
Run
rpk topic describe <topic-name> -c. The output shows the values ofredpanda.iceberg.table.nameandredpanda.iceberg.table.namespace. If a property is not set, its value is empty, and Redpanda uses the default.For example, for the
clickstreamtopic:rpk topic describe clickstream -c | grep redpanda.iceberg.tableredpanda.iceberg.table.name events DYNAMIC_TOPIC_CONFIG redpanda.iceberg.table.namespace analytics.web DYNAMIC_TOPIC_CONFIG -
List the tables in the namespace with your catalog or query engine. This shows the tables that Redpanda created, including the DLQ table.