Skip to main content
Version: 23.1

Create a Snowflake Sink Connector

You can use the Snowflake Sink connector to ingest and store Redpanda structured data into a Snowflake database for analytics and decision-making.


Before you can create a Snowflake Sink connector in the Redpanda Cloud, you must:

  1. Create a role for use by Kafka Connect.
  2. Create a key pair for authentication.
  3. Create a database to hold the data you intend to stream from Redpanda Cloud messages.


Refer to the Snowflake Kafka Connector Limitations documentation for details.

Create a Snowflake Sink connector

To create a Snowflake Sink connector:

  1. In Redpanda Cloud, click Connectors in the navigation menu, and then click Create Connector.

  2. Select Export to Snowflake.

  3. On the Create Connector page, specify the following required connector configuration options:

    Topics to exportA comma-separated list of the cluster topics you want to export to Snowflake.
    Snowflake URL nameThe Snowflake URL to be used for the connection.
    Snowflake database nameThe Snowflake database name to be used for the exported data.
    Snowflake user nameThe name of the user who created the key pair.
    Snowflake private keyThe private key name for the Snowflake user.
    Snowflake private key passphrase(Optional) If created and encrypted, the passphrase of the private key.
    Snowflake role nameThe name of the role created in Prerequisites.
    Kafka message value formatThe format of the value in the Kafka topic. The default is SNOWFLAKE_JSON.
    Max TasksMaximum number of tasks to use for this connector. The default is 1. Each task replicates exclusive set of partitions assigned to it.
    Connector nameGlobally-unique name to use for this connector.
  4. Click Next. Review the connector properties specified, then click Create.

Advanced Snowflake Sink connector configuration

In most instances, the preceding basic configuration properties are sufficient. If you require additional property settings, then specify any of the following optional advanced connector configuration properties by selecting Show advanced options on the Create Connector page:

Snowflake schema nameThe Snowflake database schema name. The default is PUBLIC.
Snowflake ingestion methodThe default, SNOWPIPE, allows for structured data, while SNOWPIPE_STREAMING is lower latency option.
Snowflake topic2table map(Optional) Map of topics to tables. Format is comma-separated tuples. For example, <topic-1>:<table-1>,<topic-2>:<table-2>.
Buffer count recordsNumber of records buffered in memory per partition before triggering Snowflake ingestion. Default is 10000.
Buffer flush timeThe time in seconds to flush cached data. Default is 120.
Buffer size bytesCumulative size of records buffered in memory per partition before triggering Snowflake ingestion. Default is 5000000.
Error toleranceError tolerance response during connector operation. Default value is none and signals that any error will result in an immediate connector task failure. Value of all changes the behavior to skip over problematic records.

Map data

Use the appropriate key or value converter (input data format) for your data as follows:

  • JSON formatted records should use SNOWFLAKE_JSON.

  • AVRO formatted records that use Kafka’s Schema Registry Service should use SNOWFLAKE_AVRO.

  • AVRO formatted records that contain the schema (and therefore do not need Kafka’s Schema Registry Service) should use SNOWFLAKE_AVRO_WITHOUT_SCHEMA_REGISTRY.

  • Plain text formatted records should use STRING.

Test the connection

After the connector is created, verify in your Snowflake worksheet that your table is populated:


It may take a couple of minutes for the records to be visible in Snowflake.


After submitting the connector for creation in Redpanda Console, the Snowflake Sink connector attempts to authenticate to the Snowflake database to validate the configuration. This validation must be successful before the connector is created. It can take up 10 seconds or more to respond.

Additional errors and corrective actions follow.

MessageAction is not a valid snowflake url`Check to make sure Snowflake URL name contains a valid Snowflake URL. Cannot connect to SnowflakeCheck to make sure Snowflake user name contains a valid Snowflake user.
snowflake.private.key must be a valid PEM RSA private key / java.lang.IllegalArgumentException: Last encoded character (before the padding, if any) is a valid base 64 alphabet but not a possible value. Expect the discarded bits to be zero.Snowflake private key is invalid. Provide a valid key. database does not existSpecify a valid database name in
Object does not exist, or operation cannot be performedSnowflake error that can have several causes: an invalide role is being used, there is no existing Snowflake table, or an incorrect schema name is specified. Verify that the connector configuration and Snowflake settings are valid.
Config:value.converter has provided value:com.snowflake.kafka.connector.records.SnowflakeJsonConverter. If ingestionMethod is:snowpipe_streaming, Snowflake Custom Converters are not allowed.Use STRING for the Kafka message value format.

Suggested reading

What do you like about this doc?

Optional: Share your email address if we can contact you about your feedback.

Let us know what we do well: