Cloud
Also in Self-Managed

google_drive_search

Searches Google Drive for files matching the provided query.

This processor searches for files in Google Drive using the provided query.

Search results are emitted as message batch, where each message is a Google Drive File

Authentication

By default, this connector will use Google Application Default Credentials (ADC) to authenticate with Google APIs.

To use this mechanism locally, run these gcloud commands:

# Log in for the application default credentials and add scopes for read-only Drive access
gcloud auth application-default login --scopes='openid,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/drive.readonly'
# When logging in with a user account, you may need to set the quota project for the application default credentials
gcloud auth application-default set-quota-project <project-id>

Replace <project-id> with your Google Cloud project ID.

Otherwise if using a service account, you can create a JSON key for the service account and set it in the credentials_json field. In order for a service account to access files in Google Drive either files need to be explicitly shared with the service account email, otherwise domain wide delegation can be used to share all files within a Google Workspace.

Try out the example pipeline on this page, which searches for and downloads all Google Drive files that match the specified query.

  • Common

  • Advanced

processor:
  label: ""
  google_drive_search:
    credentials_json: "" # No default (optional)
    query: "" # No default (required)
    projection:
      - "id"
      - "name"
      - "mimeType"
      - "size"
      - "labelInfo"
    include_label_ids: ""
    max_results: 64
    shared_drives: false
processor:
  label: ""
  google_drive_search:
    credentials_json: "" # No default (optional)
    query: "" # No default (required)
    projection:
      - "id"
      - "name"
      - "mimeType"
      - "size"
      - "labelInfo"
    include_label_ids: ""
    max_results: 64
    shared_drives: false

Fields

credentials_json

The JSON key for your service account (optional). If left empty, Application Default Credentials are used.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Manage Secrets before adding it to your configuration.

Type: string

include_label_ids

A comma delimited list of label IDs to include in the Google Drive search result.

This field supports interpolation functions.

Type: string

Default: ""

max_results

The maximum number of search results to return.

Type: int

Default: 64

projection[]

Partial fields to include in the Google Drive search result.

Type: array<string>

Default:

- "id"
- "name"
- "mimeType"
- "size"
- "labelInfo"

query

Specify a search query to locate matching files in Google Drive. This field supports:

Type: string

shared_drives

Whether or not to include shared drives in the result.

Type: bool

Default: false

Examples

Search & download files from Google Drive

This examples downloads all the files from Google Drive that are returned in the query

input:
  stdin: {}
pipeline:
  processors:
    - google_drive_search:
        query: "${!content().string()}"
    - mutation: 'meta path = this.name'
    - google_drive_download:
        file_id: "${!this.id}"
        mime_type: "${!this.mimeType}"
output:
  file:
    path: "${!@path}"
    codec: all-bytes