Docs Cloud Develop Managed Connectors Converters and serialization Converters and Serialization Connectors are a translation layer working between Redpanda and the remote system. For sink connectors the translation happens in the following phases: Converter deserializes data from Redpanda message format (for example JSON or Avro) to a universal in-memory connect data format. The in-memory connect data structure is translated by the connector to the data model of the remote system. For source connectors it is vice versa, the phases are: Connector translates the data model from remote system format to the in-memory connect data structure. Converter serializes the data from a universal in-memory connect format to a Redpanda message. Each Redpanda message is a key and value record. Record key and value converters are configured separately with the Redpanda message key format and Redpanda message value format properties. Key and value converters can be different. If an external system requires structured data (like BigQuery or a SQL database), then you must provide data with a schema. Use the Avro, Protobuf, or JSON converter with a schema. ByteArray converter The ByteArray converter is the most primitive and high-throughput converter. Schema is ignored. This is the default converter type for managed connectors. To use the converter, select the ByteArray option as a key or value message format. String converter The String converter is a high-throughput converter. Schema is ignored. All data is converted to a string. To use the converter, select the String option as a key or value message format. JSON converter The JSON converter supports a JSON schema embedded in the message, where each message contains a schema. It results in a bigger message size. The connector needs a message schema to check message format. To use the converter, select the JSON option as a key or value message format. Example JSON message with embedded schema: { "schema": { "type": "struct", "fields": [ { "type": "int64", "optional": false, "field": "person_id" }, { "type": "string", "optional": false, "field": "name" } ] }, "payload": { "person_id": 1, "name": "Redpanda" } } If you consume JSON data with no message schema, the schema check for the connector must be disabled with the Message key JSON contains schema or Message value JSON contains schema option. Avro converter The Avro converter requires a schema in Schema Registry. Avro supports primitive types and complex types, like records, enums, arrays, maps, and unions. To specify a timestamp in an Avro schema for use with Kafka Connect, use: { "name": "time1", "type": [ "null", { "type": "long", "connect.version": 1, "connect.name": "org.apache.kafka.connect.data.Timestamp", "logicalType": "timestamp-millis" } ], "default": null } See also: Redpanda Schema Registry Avro specification CloudEvents converter The CloudEvents converter is specific to Debezium PostgreSQL and MySQL source connectors. See also: CloudEvents Converter documentation Protobuf converter The Protobuf converter requires a schema in Schema Registry. The converter only supports sink connectors. Source connectors are not supported. To use the converter, select the Protobuf option as a key or value message format. See also: Redpanda Schema Registry Set property keys Redpanda Connectors use a set of <property.key>=<value> to set up its properties. For example if you want to set the property topic.creation.enable to true, use topic.creation.enable=true in the property settings page. Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Managed Connectors Monitor Connectors