couchbase

Beta

Type:

Available in: Self-Managed

Perform operations on a Couchbase database for each incoming message. This connector can store or delete data in Couchbase based on the message’s content.

Introduced in version 4.37.0.

  • Common

  • Advanced

# Common configuration fields, showing default values
output:
  label: ""
  couchbase:
    url: couchbase://localhost:11210 # No default (required)
    username: "" # No default (optional)
    password: "" # No default (optional)
    bucket: "" # No default (required)
    id: ${! json("id") } # No default (required)
    content: "" # No default (optional)
    operation: upsert
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
# All configuration fields, showing default values
output:
  label: ""
  couchbase:
    url: couchbase://localhost:11210 # No default (required)
    username: "" # No default (optional)
    password: "" # No default (optional)
    bucket: "" # No default (required)
    collection: _default
    transcoder: legacy
    timeout: 15s
    id: ${! json("id") } # No default (required)
    content: "" # No default (optional)
    operation: upsert
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
      processors: [] # No default (optional)

Performance

For improved performance, this output sends multiple messages in parallel. You can tune the maximum number of in-flight messages (or message batches), using the max_in_flight field.

You can configure batches at both the input and output level. For more information, see Message Batching.

Fields

url

The Couchbase connection string.

Type: string

# Examples
url: couchbase://localhost:11210

username

A username to authenticate with the Couchbase cluster.

Type: string

password

A password to authenticate with the Couchbase cluster.

This field contains sensitive information that usually shouldn’t be added to a configuration directly. For more information, see Secrets.

Type: string

bucket

The name of the Couchbase bucket you want the output to access.

Type: string

collection

The name of the specific collection you want the output to write to.

Type: string

Default: _default

transcoder

Choose from the following Couchbase transcoders to convert the messages' data format before they are written to Couchbase.

Type: string

Default: legacy

Option

Summary

json

JSONTranscoder implements the default transcoding behavior and applies JSON transcoding to all values. It applies the following behavior to the value: binary ([]byte) → error. default → JSON value, JSON Flags.

legacy

LegacyTranscoder implements the behavior for a backward-compatible transcoder. This transcoder implements behavior matching that of gocb v1. It applies the following behavior to the value: binary ([]byte) → binary bytes, Binary expectedFlags. string → string bytes, String expectedFlags. default → JSON value, JSON expectedFlags.

raw

RawBinaryTranscoder implements passthrough behavior of raw binary data. This transcoder does not apply any serialization. It applies the following behavior to the value: binary ([]byte) → binary bytes, binary expectedFlags. default → error.

rawjson

RawJSONTranscoder implements passthrough behavior of JSON data. This transcoder does not apply any serialization. It will forward data across the network without incurring unnecessary parsing costs. It applies the following behavior to the value: binary ([]byte) → JSON bytes, JSON expectedFlags. string → JSON bytes, JSON expectedFlags. default → error.

rawstring

RawStringTranscoder implements passthrough behavior of raw string data. This transcoder does not apply any serialization. It applies the following behavior to the value: string → string bytes, string expectedFlags. default → error.

timeout

Operation timeout if this output cannot connect to the Couchbase cluster.

Type: string

Default: 15s

id

The document ID to use.

This field supports interpolation functions.

Type: string

# Examples
id: ${! json("id") }

content

The document content to update. When inserting, replacing, or upserting documents, you must set a content value.

Type: string

operation

The Couchbase operation to perform.

Type: string

Default: upsert

Option

Summary

insert

Insert a new document.

remove

Delete a document.

replace

Replace the contents of a document.

upsert

Create a new document if it does not exist. If it does exist, then update.

max_in_flight

The maximum number of messages to have in flight at a given time. Increase this value to improve throughput.

Type: int

Default: 64

batching

Configure a batching policy.

Type: object

# Examples
batching:
  byte_size: 5000
  count: 0
  period: 1s
batching:
  count: 10
  period: 1s
batching:
  check: this.contains("END BATCH")
  count: 0
  period: 1m

batching.count

The number of messages after which the batch is flushed. Set to 0 to disable count-based batching.

Type: int

Default: 0

batching.byte_size

The amount of bytes at which the batch is flushed. Set to 0 to disable size-based batching.

Type: int

Default: 0

batching.period

The period of time after which an incomplete batch is flushed regardless of its size.

Type: string

Default: ""

# Examples
period: 1s
period: 1m
period: 500ms

batching.check

A Bloblang query that returns a boolean value indicating whether a message should end a batch.

Type: string

Default: ""

# Examples
check: this.type == "end_of_transaction"

batching.processors

For aggregating and archiving message batches, you can add a list of processors to apply to a batch as it is flushed. All resulting messages are flushed as a single batch even when you configure processors to split the batch into smaller batches.

Type: array

# Examples
processors:
  - archive:
      format: concatenate
processors:
  - archive:
      format: lines
processors:
  - archive:
      format: json_array