Cloud

gateway

Receive messages delivered over HTTP.

The gateway input is a Cloud-only component that receives messages over HTTP and injects them into a running Redpanda Connect pipeline.

Use this input to:

  • Receive webhook events from third-party services

  • Accept real-time telemetry or sensor data over HTTP

  • Build lightweight ingest endpoints for client apps

For on-premises or self-managed deployments, use the http_server input instead.

This component is fully managed and available in the following Redpanda Cloud deployment types:

  • Serverless

  • Dedicated

  • Bring Your Own Cloud (BYOC)

When you deploy a pipeline with a gateway input, Redpanda Cloud provisions a secure URL that you can use to send HTTP requests. You can post raw payloads, JSON messages, or stream events in real time.

Authentication and access control use standard Redpanda Cloud API tokens. For more information, see Redpanda Cloud API authentication.

Network access depends on the cluster type:

  • On public clusters (Serverless and Dedicated), the gateway URL is accessible over the public internet.

  • On private clusters (BYOC), the gateway is accessible only from within your configured VPC.

Rate limiting

The rate_limit field lets you specify an optional rate limit resource, which is applied to each HTTP request received. When the rate limit is breached, the input returns a 429 response with a Retry-After header.

Responses

You can return a response for each message received using synchronous responses. When doing so, you can customize headers using the sync_response.headers field, which supports function interpolation in the value based on the response message contents.

Metadata

This input adds the following metadata fields to each message:

  • http_server_user_agent

  • http_server_request_path

  • http_server_verb

  • http_server_remote_ip

  • All headers (only first values are taken)

  • All query parameters

  • All path parameters

  • All cookies

You can access these metadata fields using function interpolation.

  • Common

  • Advanced

input:
  label: ""
  gateway:
    path: /
    rate_limit: ""
input:
  label: ""
  gateway:
    path: /
    rate_limit: ""
    sync_response:
      status: "200"
      headers:
        Content-Type: "application/octet-stream"
      metadata_headers:
        include_prefixes: []
        include_patterns: []
    tcp:
      reuse_addr: false
      reuse_port: false

Fields

path

The endpoint path to listen for data delivery requests.

Type: string

Default: /

rate_limit

The name of an optional rate limit resource that the gateway applies to each request it receives. When the rate limit is reached, the gateway rejects the request with a 429 response and a Retry-After header.

Type: string

Default: ""

sync_response

Customize messages returned using synchronous responses.

Type: object

sync_response.headers

The headers that the gateway adds to synchronous responses. Header values are interpolated against the first response message.

This field supports interpolation functions.

Type: object<string>

Default:

Content-Type: "application/octet-stream"

sync_response.metadata_headers

Selects which metadata values of the response messages the gateway adds to the synchronous response as headers.

Type: object

sync_response.metadata_headers.include_patterns[]

A list of explicit metadata key regular expression (re2) patterns to match against.

Type: array<string>

Default: []

# Examples:
include_patterns:
  - .*

# ---

include_patterns:
  - _timestamp_unix$

sync_response.metadata_headers.include_prefixes[]

A list of explicit metadata key prefixes to match against.

Type: array<string>

Default: []

# Examples:
include_prefixes:
  - foo_
  - bar_

# ---

include_prefixes:
  - kafka_

# ---

include_prefixes:
  - content-

sync_response.status

The HTTP status code that the gateway returns when the pipeline produces a synchronous response. The value is interpolated against the first response message, so the code can depend on its contents and metadata.

This field supports interpolation functions.

Type: string

Default: 200

# Examples:
status: ${! json("status") }

# ---

status: ${! meta("status") }

tcp

TCP listener socket configuration.

Type: object

tcp.reuse_addr

Enable SO_REUSEADDR, allowing binding to ports in TIME_WAIT state. Useful for graceful restarts and config reloads where the server needs to rebind to the same port immediately after shutdown.

Type: bool

Default: false

tcp.reuse_port

Enable SO_REUSEPORT, allowing multiple sockets to bind to the same port for load balancing across multiple processes/threads.

Type: bool

Default: false

Examples

Ingest a real-time stream of sensor data

Use the gateway input to stream telemetry data from edge devices or browser clients that connect over HTTP.

Suppose a client connects and sends JSON-encoded sensor readings like this:

{ "sensor_id": "temp-001", "value": 22.5, "unit": "C" }
{ "sensor_id": "temp-001", "value": 22.8, "unit": "C" }
{ "sensor_id": "temp-001", "value": 23.1, "unit": "C" }

Redpanda Connect treats each line as an individual message.

The following pipeline sets up a gateway input to handle these connections and logs each message:

input:
  label: sensor_stream
  gateway:
    path: /ws/sensors
    rate_limit: ""

pipeline:
  processors:
    - log:
        level: INFO
        message: "Received reading from ${! json(\"sensor_id\") }: ${! json(\"value\") } ${! json(\"unit\") }"

This configuration:

  • Accepts HTTP connections on /ws/sensors

  • Receives a stream of messages over a single connection

  • Logs each message using Bloblang interpolation

You can replace the log processor with any downstream output, such as Redpanda or Amazon S3, to persist or analyze the data in real time.