gcp_pubsub

Type:

Available in: Cloud, Self-Managed

Consumes messages from a GCP Cloud Pub/Sub subscription.

  • Common

  • Advanced

# Common config fields, showing default values
input:
  label: ""
  gcp_pubsub:
    project: "" # No default (required)
    credentials_json: "" # No default (optional)
    subscription: "" # No default (required)
    endpoint: ""
    sync: false
    max_outstanding_messages: 1000
    max_outstanding_bytes: 1e+09
# All config fields, showing default values
input:
  label: ""
  gcp_pubsub:
    project: "" # No default (required)
    credentials_json: "" # No default (optional)
    subscription: "" # No default (required)
    endpoint: ""
    sync: false
    max_outstanding_messages: 1000
    max_outstanding_bytes: 1e+09
    create_subscription:
      enabled: false
      topic: ""

For information on how to set up credentials see this guide.

Metadata

This input adds the following metadata fields to each message:

  • gcp_pubsub_publish_time_unix - The time at which the message was published to the topic.

  • gcp_pubsub_delivery_attempt - When dead lettering is enabled, this is set to the number of times PubSub has attempted to deliver a message.

  • All message attributes

You can access these metadata fields using function interpolation.

Fields

project

The project ID of the target subscription.

Type: string

credentials_json

Optional field to set Google Service Account Credentials JSON.

This field contains sensitive information. Review your cluster security before adding it to your configuration.

Type: string

Default: ""

subscription

The target subscription ID.

Type: string

endpoint

An optional endpoint to override the default of pubsub.googleapis.com:443. This can be used to connect to a region specific pubsub endpoint. For a list of valid values, see this document.

Type: string

Default: ""

# Examples

endpoint: us-central1-pubsub.googleapis.com:443

endpoint: us-west3-pubsub.googleapis.com:443

sync

Enable synchronous pull mode.

Type: bool

Default: false

max_outstanding_messages

The maximum number of outstanding pending messages to be consumed at a given time.

Type: int

Default: 1000

max_outstanding_bytes

The maximum number of outstanding pending messages to be consumed measured in bytes.

Type: int

Default: 1000000000

create_subscription

Allows you to configure the input subscription and creates if it doesn’t exist.

Type: object

create_subscription.enabled

Whether to configure subscription or not.

Type: bool

Default: false

create_subscription.topic

Defines the topic that the subscription should be vinculated to.

Type: string

Default: ""