---
title: aws_cloudwatch_logs
page-cloud: true
latest-operator-version: v26.2.6
latest-console-tag: v3.14.0
latest-connect-version: 4.118.0
latest-redpanda-tag: v26.2.7
docname: connect/components/inputs/aws_cloudwatch_logs
page-component-name: cloud-data-platform
page-version: master
page-component-version: master
page-component-title: Cloud
page-relative-src-path: connect/components/inputs/aws_cloudwatch_logs.adoc
page-edit-url: https://github.com/redpanda-data/cloud-docs/edit/main/modules/develop/pages/connect/components/inputs/aws_cloudwatch_logs.adoc
description: Consumes log events from AWS CloudWatch Logs.
page-self-managed-available: "true"
page-self-managed-available-url: https://docs.redpanda.com/connect/components/inputs/aws_cloudwatch_logs/
page-git-created-date: "2026-03-13"
page-git-modified-date: "2026-10-30"
page-generated-sources:
  - include: connect:components:partial$descriptions/inputs/aws_cloudwatch_logs.adoc
    generator: null
    version: null
    source: the component specs in https://github.com/redpanda-data/connect
page-generated-content: partial
---

# aws_cloudwatch_logs

> For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [cloud-data-platform-full.txt](https://docs.redpanda.com/cloud-data-platform-full.txt)

<!-- Source: https://docs.redpanda.com/cloud-data-platform/develop/connect/components/inputs/aws_cloudwatch_logs.md -->

Consumes log events from AWS CloudWatch Logs.

Polls CloudWatch Log Groups for log events. Supports filtering by log streams, CloudWatch filter patterns, and configurable start times.

Each log event becomes a separate message with metadata including the log group name, log stream name, timestamp, and ingestion time.

> ❗ **IMPORTANT**
>
> This input tracks its position in memory only. If the process restarts, it resumes from `start_time`, or from the beginning of the available logs if `start_time` is not set. With an RFC3339 `start_time`, or with none set, it can emit events it already delivered again (at-least-once). With `start_time: now`, it skips the events written while the pipeline was down. Make downstream processing idempotent or deduplicate the events.

## Credentials

By default, Redpanda Connect uses a shared credentials file when connecting to AWS services. You can also set credentials explicitly at the component level, which allows you to transfer data across accounts. For more information, see [AWS credentials](https://docs.redpanda.com/cloud-data-platform/develop/connect/guides/cloud/aws/).

## Metadata

This input adds the following metadata fields to each message:

-   `cloudwatch_log_group` - The name of the log group

-   `cloudwatch_log_stream` - The name of the log stream

-   `cloudwatch_timestamp` - The timestamp of the log event (Unix milliseconds)

-   `cloudwatch_ingestion_time` - The ingestion timestamp (Unix milliseconds)

-   `cloudwatch_event_id` - The unique event ID


You can access these metadata fields using [Bloblang](https://docs.redpanda.com/cloud-data-platform/develop/connect/guides/bloblang/about/).

### Common

```yml
input:
  label: ""
  aws_cloudwatch_logs:
    log_group_name: "" # No default (required)
    log_stream_names: [] # No default (optional)
    log_stream_prefix: "" # No default (optional)
    filter_pattern: "" # No default (optional)
    start_time: "" # No default (optional)
    poll_interval: 5s
    auto_replay_nacks: true
```

### Advanced

```yml
input:
  label: ""
  aws_cloudwatch_logs:
    log_group_name: "" # No default (required)
    log_stream_names: [] # No default (optional)
    log_stream_prefix: "" # No default (optional)
    filter_pattern: "" # No default (optional)
    start_time: "" # No default (optional)
    poll_interval: 5s
    limit: 1000
    structured_log: true
    api_timeout: 30s
    auto_replay_nacks: true
    region: "" # No default (optional)
    endpoint: "" # No default (optional)
    tcp:
      connect_timeout: 0s
      keep_alive:
        idle: 15s
        interval: 15s
        count: 9
      tcp_user_timeout: 0s
    credentials:
      profile: "" # No default (optional)
      id: "" # No default (optional)
      secret: "" # No default (optional)
      token: "" # No default (optional)
      from_ec2_role: false # No default (optional)
      role: "" # No default (optional)
      role_external_id: "" # No default (optional)
```

## Fields

### `api_timeout`

The maximum time to wait for an API request to complete.

**Type**: `string`

**Default**: `30s`

### `auto_replay_nacks`

Whether to automatically replay rejected messages (negative acknowledgements, or nacks) at the output level. If the cause of rejections persists, leaving this option enabled can result in back pressure.

Set `auto_replay_nacks` to `false` to delete rejected messages. Disabling auto replays can greatly improve memory efficiency of high throughput streams, as the original shape of the data is discarded immediately upon consumption and mutation.

**Type**: `bool`

**Default**: `true`

### `credentials`

Manually configure the AWS credentials to use (optional). For more information, see the [Amazon Web Services guide](https://docs.redpanda.com/cloud-data-platform/develop/connect/guides/cloud/aws/).

**Type**: `object`

### `credentials.from_ec2_role`

Use the credentials of a host EC2 machine configured to assume [an IAM role associated with the instance](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html).

**Type**: `bool`

### `credentials.id`

The ID of the AWS credentials to use.

**Type**: `string`

### `credentials.profile`

The profile from `~/.aws/credentials` to use.

**Type**: `string`

### `credentials.role`

The ARN of the role to assume.

**Type**: `string`

### `credentials.role_external_id`

An external ID to use when assuming a role.

**Type**: `string`

### `credentials.secret`

The secret for the AWS credentials in use.

> ⚠️ **CAUTION**
>
> This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see [Manage Secrets](https://docs.redpanda.com/cloud-data-platform/develop/connect/configuration/secret-management/) before adding it to your configuration.

**Type**: `string`

### `credentials.token`

The token for the AWS credentials in use. Required only when using short-term credentials.

**Type**: `string`

### `endpoint`

A custom endpoint URL for AWS API requests. Use this to connect to AWS-compatible services or local testing environments instead of the standard AWS endpoints.

**Type**: `string`

### `filter_pattern`

An optional CloudWatch Logs filter pattern to apply when querying log events. For syntax details, see the [CloudWatch Logs filter and pattern syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html) documentation.

**Type**: `string`

```yaml
# Examples:
filter_pattern: "[ERROR]"
```

### `limit`

The maximum number of log events to return in a single API call. Valid range: 1-10000.

**Type**: `int`

**Default**: `1000`

### `log_group_name`

The name of the CloudWatch Log Group to consume from.

**Type**: `string`

```yaml
# Examples:
log_group_name: my-app-logs
```

### `log_stream_names[]`

An optional list of log stream names to consume from. If not set, events from all streams in the log group will be consumed.

**Type**: `array<string>`

```yaml
# Examples:
log_stream_names:
  - stream-1
  - stream-2
```

### `log_stream_prefix`

An optional log stream name prefix to filter streams. Only streams starting with this prefix will be consumed.

**Type**: `string`

```yaml
# Examples:
log_stream_prefix: prod-
```

### `poll_interval`

The interval at which to poll for new log events.

**Type**: `string`

**Default**: `5s`

### `region`

The AWS region in which your resources are hosted.

**Type**: `string`

### `start_time`

The time to start consuming log events from. Can be an RFC3339 timestamp (for example, `2024-01-01T00:00:00Z`) or the string `now` to start consuming from the current time. If not set, starts from the beginning of available logs.

**Type**: `string`

```yaml
# Examples:
start_time: 2024-01-01T00:00:00Z

# ---

start_time: now
```

### `structured_log`

Whether to output log events as structured JSON objects with all metadata fields, or as plain text messages with metadata stored in Redpanda Connect message metadata.

**Type**: `bool`

**Default**: `true`

### `tcp`

Configure TCP socket-level settings to optimize network performance and reliability. These low-level controls are useful for:

-   **Unresponsive hosts**: Set `connect_timeout` to limit how long a connection attempt can take (the default `0s` sets no limit)

-   **Long-lived connections**: Configure `keep_alive` settings to detect and recover from stale connections

-   **Unstable networks**: Tune keep-alive probes to balance between quick failure detection and avoiding false positives

-   **Linux systems with specific requirements**: Use `tcp_user_timeout` (Linux 2.6.37+) to control data acknowledgment timeouts


Most users should keep the default values. Only modify these settings if you’re experiencing connection stability issues or have specific network requirements.

**Type**: `object`

### `tcp.connect_timeout`

Maximum amount of time a dial will wait for a connect to complete. Zero disables.

**Type**: `string`

**Default**: `0s`

### `tcp.keep_alive`

TCP keep-alive probe configuration.

**Type**: `object`

### `tcp.keep_alive.count`

Maximum unanswered keep-alive probes before dropping the connection. Zero defaults to 9.

**Type**: `int`

**Default**: `9`

### `tcp.keep_alive.idle`

Duration the connection must be idle before sending the first keep-alive probe. Zero defaults to 15s. Negative values disable keep-alive probes.

**Type**: `string`

**Default**: `15s`

### `tcp.keep_alive.interval`

Duration between keep-alive probes. Zero defaults to 15s.

**Type**: `string`

**Default**: `15s`

### `tcp.tcp_user_timeout`

Maximum time to wait for acknowledgment of transmitted data before killing the connection. Linux-only (kernel 2.6.37+), ignored on other platforms. When enabled, keep_alive.idle must be greater than this value per RFC 5482. Zero disables.

**Type**: `string`

**Default**: `0s`