# gcp_bigquery_write_api

> For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [connect-full.txt](https://docs.redpanda.com/connect-full.txt)

---
title: gcp_bigquery_write_api
latest-connect-version: 4.93.0
latest-operator-version: v26.1.4
latest-console-tag: v3.7.3
latest-redpanda-tag: v26.1.9
docname: outputs/gcp_bigquery_write_api
page-component-name: connect
page-version: master
page-component-version: master
page-component-title: Connect
page-relative-src-path: outputs/gcp_bigquery_write_api.adoc
page-edit-url: https://github.com/redpanda-data/rp-connect-docs/edit/main/modules/components/pages/outputs/gcp_bigquery_write_api.adoc
description: Streams data into BigQuery using the Storage Write API.
page-git-created-date: "2026-05-11"
page-git-modified-date: "2026-05-26"
---

<!-- Source: https://docs.redpanda.com/connect/components/outputs/gcp_bigquery_write_api.md -->

**Available in:** Self-Managed

**License**: This component requires an [enterprise license](https://docs.redpanda.com/redpanda-connect/get-started/licensing/). You can either [upgrade to an Enterprise Edition license](https://www.redpanda.com/upgrade), or [generate a trial license key](http://redpanda.com/try-enterprise) that's valid for 30 days.

Streams data into BigQuery using the Storage Write API.

Introduced in version 4.90.0.

Writes messages to a BigQuery table using the Storage Write API. This provides higher throughput and lower latency than the legacy streaming API or load jobs.

Messages can be formatted as JSON (default) or raw protobuf bytes. When using JSON format the component automatically fetches the table schema and converts each message to the corresponding proto representation.

> ⚠️ **WARNING**
>
> The proto3 JSON mapping encodes int64 and uint64 values as strings. JSON messages with integer fields must use string values (e.g. `"age": "30"` not `"age": 30`). Otherwise the write will fail with an unmarshalling error.

When batching is enabled the table name is resolved from the first message in each batch. All messages in the same batch are written to that table.

#### Common

```yml
outputs:
  label: ""
  gcp_bigquery_write_api:
    project: ""
    dataset: "" # No default (required)
    table: "" # No default (required)
    message_format: json
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
      processors: [] # No default (optional)
    credentials_json: ""
```

#### Advanced

```yml
outputs:
  label: ""
  gcp_bigquery_write_api:
    project: ""
    dataset: "" # No default (required)
    table: "" # No default (required)
    message_format: json
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
      processors: [] # No default (optional)
    credentials_json: ""
    target_principal: ""
    delegates: []
    stream_idle_timeout: 5m
    stream_sweep_interval: 1m
    endpoint:
      http: ""
      grpc: ""
```

## [](#fields)Fields

### [](#batching)`batching`

Allows you to configure a [batching policy](https://docs.redpanda.com/connect/configuration/batching/).

**Type**: `object`

```yaml
# Examples:
batching:
  byte_size: 5000
  count: 0
  period: 1s

# ---

batching:
  count: 10
  period: 1s

# ---

batching:
  check: this.contains("END BATCH")
  count: 0
  period: 1m
```

### [](#batching-byte_size)`batching.byte_size`

An amount of bytes at which the batch should be flushed. If `0` disables size based batching.

**Type**: `int`

**Default**: `0`

### [](#batching-check)`batching.check`

A [Bloblang query](https://docs.redpanda.com/connect/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch.

**Type**: `string`

**Default**: `""`

```yaml
# Examples:
check: this.type == "end_of_transaction"
```

### [](#batching-count)`batching.count`

A number of messages at which the batch should be flushed. If `0` disables count based batching.

**Type**: `int`

**Default**: `0`

### [](#batching-period)`batching.period`

A period in which an incomplete batch should be flushed regardless of its size.

**Type**: `string`

**Default**: `""`

```yaml
# Examples:
period: 1s

# ---

period: 1m

# ---

period: 500ms
```

### [](#batching-processors)`batching.processors[]`

A list of [processors](https://docs.redpanda.com/connect/components/processors/about/) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.

**Type**: `processor`

```yaml
# Examples:
processors:
  - archive:
      format: concatenate

# ---

processors:
  - archive:
      format: lines

# ---

processors:
  - archive:
      format: json_array
```

### [](#credentials_json)`credentials_json`

An optional JSON string containing GCP credentials. If empty, credentials are loaded from the environment.

> ⚠️ **CAUTION**
>
> This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see [Secrets](https://docs.redpanda.com/connect/configuration/secrets/).

**Type**: `string`

**Default**: `""`

### [](#dataset)`dataset`

The BigQuery dataset ID.

**Type**: `string`

### [](#delegates)`delegates[]`

Optional delegation chain for chained service account impersonation. Each service account must be granted roles/iam.serviceAccountTokenCreator on the next in the chain.

**Type**: `array`

**Default**: `[]`

### [](#endpoint)`endpoint`

Optional endpoint overrides for the BigQuery and Storage Write API clients.

**Type**: `object`

### [](#endpoint-grpc)`endpoint.grpc`

Override the BigQuery Storage gRPC endpoint. Useful for local emulators.

**Type**: `string`

**Default**: `""`

### [](#endpoint-http)`endpoint.http`

Override the BigQuery HTTP endpoint. Useful for local emulators.

**Type**: `string`

**Default**: `""`

### [](#max_in_flight)`max_in_flight`

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

**Type**: `int`

**Default**: `64`

### [](#message_format)`message_format`

The format of input messages. Use 'json' to have the component convert JSON to proto automatically. Use 'protobuf' to supply raw proto-encoded bytes.

**Type**: `string`

**Default**: `json`

**Options**: `json`, `protobuf`

### [](#project)`project`

The GCP project ID. If empty, the project is auto-detected from the environment.

**Type**: `string`

**Default**: `""`

### [](#stream_idle_timeout)`stream_idle_timeout`

How long a cached stream can remain unused before being closed. Relevant when the table field uses interpolation to route to many tables.

**Type**: `string`

**Default**: `5m`

### [](#stream_sweep_interval)`stream_sweep_interval`

How often to check for idle streams to close.

**Type**: `string`

**Default**: `1m`

### [](#table)`table`

The BigQuery table ID. Supports interpolation functions. When batching, resolved from the first message in each batch.

**Type**: `string`

### [](#target_principal)`target_principal`

Service account email to impersonate. When set, the output obtains tokens acting as this service account. Requires the caller to have roles/iam.serviceAccountTokenCreator on the target.

**Type**: `string`

**Default**: `""`