Skip to main content
Version: 23.1

Create an HTTP Source Connector

You can use an HTTP Source connector to enable change data capture (CDC) from JSON/HTTP APIs into Redpanda. The HTTP Source connector imports data from HTTP services as batches or increments.


  • A request URL that is accessible by the connector
  • A user/password created in source so that the connector can authenticate


Only JSON or AVRO formats can be used as a value converter.

Create an HTTP Source connector

To create the HTTP Source connector:

  1. In Redpanda Cloud, click Connectors in the navigation menu, and then click Create Connector.

  2. Select Import from HTTP.

  3. On the Create Connector page, specify the following required connector configuration options:

    TopicName of the topic where the record will be sent.
    HTTP URLHTTP URL to use in the request.
    HTTP request paramsHTTP query parameters to use in the request. Use & separated list = separated pairs. For example, name=value&name2=value2.
    Authentication typeType of authentication. If using Basic, provide User and Password. Default is None.
    Kafka message key formatFormat of the key in the Kafka topic. Default is JSON. If choosing AVRO, specify the Kafka message value format.
    Message key JSON contains schemaSpecifies whether or not the message key contains schema in the schema field.
    Max TasksMaximum number of tasks to use for this connector. The default is 1. Each task replicates an exclusive set of partitions assigned to it.
    Connector nameGlobally-unique name to use for this connector.
  4. Click Next. Review the connector properties specified, then click Create.

Map data

Use JSON or AVRO converters to map data from an HTTP endpoint to a Redpanda topic.

Test the connection

After the connector is created:

  • Check the connector status and confirm that there are no errors in logs and in Redpanda Console.
  • Review the Redpanda topic to confirm that it contains the expected data.


Most HTTP Source connector issues are reported as a failed task. An invalid connection URL is reported in logs.

Additional errors and corrective actions follow.

RetriableException: Name or service not known / RetriableException: Connect timed outVerify that the HTTP URL is correct, the hostname and port are correct, and that the service accepts connections.
IllegalStateException: Policy failed for response code: 403Confirm that the service URL accepts requests from the connector, with HTTP code 200.
IllegalArgumentException: No node at '/items' (unmatched part: '/items')Confirm that HTTP response list pointer is a valid pointer in the HTTP result document.
IllegalArgumentException: No node at '/invalid-key' (unmatched part: '/invalid-key')Confirm that the HTTP response record offset pointer defines valid elements of the HTTP result document.

What do you like about this doc?

Optional: Share your email address if we can contact you about your feedback.

Let us know what we do well: