gcp_pubsub

Type:

Available in: Cloud, Self-Managed

Consumes messages from a GCP Cloud Pub/Sub subscription.

  • Common

  • Advanced

# Common config fields, showing default values
input:
  label: ""
  gcp_pubsub:
    project: "" # No default (required)
    credentials_json: "" # No default (optional)
    subscription: "" # No default (required)
    endpoint: ""
    sync: false
    max_outstanding_messages: 1000
    max_outstanding_bytes: 1e+09
# All config fields, showing default values
input:
  label: ""
  gcp_pubsub:
    project: "" # No default (required)
    credentials_json: "" # No default (optional)
    subscription: "" # No default (required)
    endpoint: ""
    sync: false
    max_outstanding_messages: 1000
    max_outstanding_bytes: 1e+09
    create_subscription:
      enabled: false
      topic: ""

For information on how to set up credentials see this guide.

Metadata

This input adds the following metadata fields to each message:

  • gcp_pubsub_publish_time_unix - The time at which the message was published to the topic.

  • gcp_pubsub_delivery_attempt - When dead lettering is enabled, this is set to the number of times PubSub has attempted to deliver a message.

  • All message attributes

You can access these metadata fields using function interpolation.

Fields

create_subscription

Allows you to configure the input subscription and creates if it doesn’t exist.

Type: object

create_subscription.enabled

Whether to configure subscription or not.

Type: bool

Default: false

create_subscription.topic

Defines the topic that the subscription should be vinculated to.

Type: string

Default: ""

credentials_json

Base64-encoded Google Service Account credentials in JSON format (optional). Use this field to authenticate with Google Cloud services. For more information about creating service account credentials, see Google’s service account documentation.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

Default: ""

endpoint

An optional endpoint to override the default of pubsub.googleapis.com:443. This can be used to connect to a region specific pubsub endpoint. For a list of valid values, see this document.

Type: string

Default: ""

# Examples:
endpoint: us-central1-pubsub.googleapis.com:443
endpoint: us-west3-pubsub.googleapis.com:443

max_outstanding_bytes

The maximum number of outstanding pending messages to be consumed measured in bytes.

Type: int

Default: 1000000000

max_outstanding_messages

The maximum number of outstanding pending messages to be consumed at a given time.

Type: int

Default: 1000

project

The project ID of the target subscription.

Type: string

subscription

The target subscription ID.

Type: string

sync

Enable synchronous pull mode.

Type: bool

Default: false