Connect

Self-Managed Only

couchbase

Performs operations against Couchbase for each message, allowing you to store or delete data.

Introduced in version 4.37.0.

This output turns each message into one operation on a Couchbase database. It sends the operations of a batch concurrently and waits for all of them to complete.

When inserting, replacing or upserting documents, each must have the content property set.

Performance

This output benefits from sending multiple messages in flight in parallel for improved performance. You can tune the max number of in flight messages (or message batches) with the field max_in_flight.

This output benefits from sending messages as a batch for improved performance. Batches can be formed at both the input and output level. You can find out more in this doc.

Common
Advanced

output:
  label: ""
  couchbase:
    url: "" # No default (required)
    username: "" # No default (optional)
    password: "" # No default (optional)
    bucket: "" # No default (required)
    id: "" # No default (required)
    content: "" # No default (optional)
    operation: upsert
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""

output:
  label: ""
  couchbase:
    url: "" # No default (required)
    username: "" # No default (optional)
    password: "" # No default (optional)
    bucket: "" # No default (required)
    collection: "" # No default (optional)
    scope: "" # No default (optional)
    transcoder: legacy
    timeout: 15s
    id: "" # No default (required)
    content: "" # No default (optional)
    ttl: "" # No default (optional)
    operation: upsert
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
      processors: [] # No default (optional)

Fields

`batching`

Configure a batching policy.

Type: object

# Examples:
batching:
  byte_size: 5000
  count: 0
  period: 1s

# ---

batching:
  count: 10
  period: 1s

# ---

batching:
  check: this.contains("END BATCH")
  count: 0
  period: 1m

`batching.byte_size`

The maximum total size (in bytes) that a batch can reach before it is flushed. When the combined size of all messages in the batch reaches or exceeds this limit, the batch is immediately sent to the next stage (such as a processor or output).

Set to 0 to disable size-based batching. When disabled, messages are flushed based on other conditions (such as count or period).

Type: int

Default: 0

`batching.check`

A Bloblang query that returns a boolean value indicating whether a message should end a batch.

Type: string

Default: ""

# Examples:
check: this.type == "end_of_transaction"

`batching.count`

The number of messages at which the batch is flushed. Set to 0 to disable count-based batching.

Type: int

Default: 0

`batching.period`

The length of time after which an incomplete batch is flushed regardless of its size. This field accepts Go duration format strings such as 100ms, 1s, or 5s. Supported time units are ns, us, ms, s, m, and h.

Type: string

Default: ""

# Examples:
period: 1s

# ---

period: 1m

# ---

period: 500ms

`batching.processors[]`

A list of processors to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. All resulting messages are flushed as a single batch, so splitting the batch into smaller batches with these processors has no effect.

Type: array<processor>

# Examples:
processors:
  - archive:
      format: concatenate


# ---

processors:
  - archive:
      format: lines


# ---

processors:
  - archive:
      format: json_array

`bucket`

The name of the Couchbase bucket to access.

Type: string

`collection`

The name of the collection within the bucket to access.

Type: string

`content`

The document content to update. When inserting, replacing, or upserting documents, you must set a content value.

Type: string

`id`

The document ID to use.

This field supports interpolation functions.

Type: string

# Examples:
id: ${! json("id") }

`max_in_flight`

The maximum number of messages to have in flight at a given time. For outputs that send messages in batches, this limit applies to message batches. Increase this value to improve throughput.

Type: int

Default: 64

`operation`

The Couchbase operation to perform.

Type: string

Default: upsert

Option Summary

Option	Summary
`insert`	insert a new document.
`remove`	delete a document.
`replace`	replace the contents of a document.
`upsert`	creates a new document if it does not exist, if it does exist then it updates it.

insert

insert a new document.

remove

delete a document.

replace

replace the contents of a document.

upsert

creates a new document if it does not exist, if it does exist then it updates it.

`password`

A password to authenticate with the Couchbase cluster.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Secrets.

Type: string

`scope`

Bucket scope.

Requires version 4.56.0 or later.

Type: string

`timeout`

The timeout for Couchbase operations, including connecting to the cluster and waiting for the bucket to become ready.

Type: string

Default: 15s

`transcoder`

The Couchbase transcoder to use for converting the data format of documents written to and read from Couchbase.

Type: string

Default: legacy

Option Summary

Option	Summary
`json`	JSONTranscoder implements the default transcoding behavior and applies JSON transcoding to all values. This will apply the following behavior to the value: binary ([]byte) → error. default → JSON value, JSON Flags.
`legacy`	LegacyTranscoder implements the behavior for a backward-compatible transcoder. This transcoder implements behavior matching that of gocb v1.This will apply the following behavior to the value: binary ([]byte) → binary bytes, Binary expectedFlags. string → string bytes, String expectedFlags. default → JSON value, JSON expectedFlags.
`raw`	RawBinaryTranscoder implements passthrough behavior of raw binary data. This transcoder does not apply any serialization. This will apply the following behavior to the value: binary ([]byte) → binary bytes, binary expectedFlags. default → error.
`rawjson`	RawJSONTranscoder implements passthrough behavior of JSON data. This transcoder does not apply any serialization. It will forward data across the network without incurring unnecessary parsing costs. This will apply the following behavior to the value: binary ([]byte) → JSON bytes, JSON expectedFlags. string → JSON bytes, JSON expectedFlags. default → error.
`rawstring`	RawStringTranscoder implements passthrough behavior of raw string data. This transcoder does not apply any serialization. This will apply the following behavior to the value: string → string bytes, string expectedFlags. default → error.

json

JSONTranscoder implements the default transcoding behavior and applies JSON transcoding to all values. This will apply the following behavior to the value: binary ([]byte) → error. default → JSON value, JSON Flags.

legacy

LegacyTranscoder implements the behavior for a backward-compatible transcoder. This transcoder implements behavior matching that of gocb v1.This will apply the following behavior to the value: binary ([]byte) → binary bytes, Binary expectedFlags. string → string bytes, String expectedFlags. default → JSON value, JSON expectedFlags.

raw

RawBinaryTranscoder implements passthrough behavior of raw binary data. This transcoder does not apply any serialization. This will apply the following behavior to the value: binary ([]byte) → binary bytes, binary expectedFlags. default → error.

rawjson

RawJSONTranscoder implements passthrough behavior of JSON data. This transcoder does not apply any serialization. It will forward data across the network without incurring unnecessary parsing costs. This will apply the following behavior to the value: binary ([]byte) → JSON bytes, JSON expectedFlags. string → JSON bytes, JSON expectedFlags. default → error.

rawstring

RawStringTranscoder implements passthrough behavior of raw string data. This transcoder does not apply any serialization. This will apply the following behavior to the value: string → string bytes, string expectedFlags. default → error.

`ttl`

An optional TTL to set for items.

Requires version 4.73.0 or later.

Type: string

`url`

The Couchbase connection string.

Type: string

# Examples:
url: couchbase://localhost:11210

`username`

A username to authenticate with the Couchbase cluster.

Type: string

Was this helpful?

group Ask in the community

mail Share your feedback

What do you think of this page?

Let us know more:

Let us contact you about your feedback:

couchbase

Performance

Fields

batching

batching.byte_size

batching.check

batching.count

batching.period

batching.processors[]

bucket

collection

content

id

max_in_flight

operation

password

scope

timeout

transcoder

ttl

url

username

`batching`

`batching.byte_size`

`batching.check`

`batching.count`

`batching.period`

`batching.processors[]`

`bucket`

`collection`

`content`

`id`

`max_in_flight`

`operation`

`password`

`scope`

`timeout`

`transcoder`

`ttl`

`url`

`username`