aws_sqs

Type:

Available in: Cloud, Self-Managed

Consume messages from an AWS SQS URL.

  • Common

  • Advanced

# Common configuration fields, showing default values
input:
  label: ""
  aws_sqs:
    url: "" # No default (required)
    max_outstanding_messages: 1000
# All configuration fields, showing default values
input:
  label: ""
  aws_sqs:
    url: "" # No default (required)
    delete_message: true
    reset_visibility: true
    max_number_of_messages: 10
    max_outstanding_messages: 1000
    wait_time_seconds: 0
    message_timeout: 30s
    region: ""
    endpoint: ""
    credentials:
      profile: ""
      id: ""
      secret: ""
      token: ""
      from_ec2_role: false
      role: ""
      role_external_id: ""

Credentials

By default, Redpanda Connect uses a shared credentials file when connecting to AWS services. You can also set credentials explicitly at the component level, which allows you to transfer data across accounts. To find out more, see Amazon Web Services.

Metadata

This input adds the following metadata fields to each message:

  • sqs_message_id

  • sqs_receipt_handle

  • sqs_approximate_receive_count

  • All message attributes

You can access these metadata fields using function interpolation.

Fields

url

The SQS URL to consume from.

Type: string

delete_message

Whether to delete the consumed message when it’s acknowledged. Set to false to handle the deletion using a different mechanism.

Type: bool

Default: true

reset_visibility

Whether to set the visibility timeout of the consumed message to zero if Redpanda Connect receives a negative acknowledgement. Set to false to use the queue’s visibility timeout for each message rather than releasing the message immediately for reprocessing.

Type: bool

Default: true

Requires version 3.58.0 or newer

max_number_of_messages

The maximum number of messages that Redpanda Connect can return each time it polls the SQS URL. Enter values from 1 to 10 only.

Type: int

Default: 10

max_outstanding_messages

The maximum number of pending messages that Redpanda Connect can have in flight at the same time.

Type: int

Default: 1000

wait_time_seconds

Whether to set a wait time (in seconds). Enter values from 1 to 20 to enable wait times and to activate log polling for queued messages.

Type: int

Default: 0

message_timeout

The maximum time allowed to process a received message before Redpanda Connect refreshes the receipt handle, and the message becomes visible in the queue again. Redpanda Connect attempts to refresh the receipt handle after half of the timeout has elapsed.

Type: string

Default: 30s

region

The AWS region to target.

Type: string

Default: ""

endpoint

Allows you to specify a custom endpoint for the AWS API.

Type: string

Default: ""

credentials

Optional manual configuration of AWS credentials to use. More information can be found in Amazon Web Services.

Type: object

credentials.profile

A profile from ~/.aws/credentials to use.

Type: string

Default: ""

credentials.id

The ID of credentials to use.

Type: string

Default: ""

credentials.secret

The secret for the credentials being used.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Secrets.

Type: string

Default: ""

credentials.token

The token for the credentials being used, required when using short term credentials.

Type: string

Default: ""

credentials.from_ec2_role

Use the credentials of a host EC2 machine configured to assume an IAM role associated with the instance.

Type: bool

Default: false

Requires version 4.2.0 or newer

credentials.role

A role ARN to assume.

Type: string

Default: ""

credentials.role_external_id

An external ID to provide when assuming a role.

Type: string

Default: ""