prometheus

Host endpoints (/metrics and /stats) for Prometheus scraping.

  • Common

  • Advanced

# Common config fields, showing default values
metrics:
  prometheus: {}
  mapping: ""
# All config fields, showing default values
metrics:
  prometheus:
    use_histogram_timing: false
    histogram_buckets: []
    summary_quantiles_objectives:
      - quantile: 0.5
        error: 0.05
      - quantile: 0.9
        error: 0.01
      - quantile: 0.99
        error: 0.001
    add_process_metrics: false
    add_go_metrics: false
    push_url: "" # No default (optional)
    push_interval: "" # No default (optional)
    push_job_name: benthos_push
    push_basic_auth:
      username: ""
      password: ""
    file_output_path: ""
  mapping: ""

Fields

use_histogram_timing

Whether to export timing metrics as a histogram, if false a summary is used instead. When exporting histogram timings the delta values are converted from nanoseconds into seconds in order to better fit within bucket definitions. For more information on histograms and summaries refer to: https://prometheus.io/docs/practices/histograms/.

Type: bool

Default: false Requires version 3.63.0 or newer

histogram_buckets

Timing metrics histogram buckets (in seconds). If left empty defaults to DefBuckets (https://pkg.go.dev/github.com/prometheus/client_golang/prometheus#pkg-variables). Applicable when use_histogram_timing is set to true.

Type: array

Default: [] Requires version 3.63.0 or newer

summary_quantiles_objectives

A list of timing metrics summary buckets (as quantiles). Applicable when use_histogram_timing is set to false.

Type: array

Default: [{"error":0.05,"quantile":0.5},{"error":0.01,"quantile":0.9},{"error":0.001,"quantile":0.99}] Requires version 4.23.0 or newer

# Examples

summary_quantiles_objectives:
  - error: 0.05
    quantile: 0.5
  - error: 0.01
    quantile: 0.9
  - error: 0.001
    quantile: 0.99

summary_quantiles_objectives[].quantile

Quantile value.

Type: float

Default: 0

summary_quantiles_objectives[].error

Permissible margin of error for quantile calculations. Precise calculations in a streaming context (without prior knowledge of the full dataset) can be resource-intensive. To balance accuracy with computational efficiency, an error margin is introduced. For instance, if the 90th quantile (0.9) is determined to be 100ms with a 1% error margin (0.01), the true value will fall within the [99ms, 101ms] range.)

Type: float

Default: 0

add_process_metrics

Whether to export process metrics such as CPU and memory usage in addition to Redpanda Connect metrics.

Type: bool

Default: false

add_go_metrics

Whether to export Go runtime metrics such as GC pauses in addition to Redpanda Connect metrics.

Type: bool

Default: false

push_url

An optional Push Gateway URL to push metrics to.

Type: string

push_interval

The period of time between each push when sending metrics to a Push Gateway.

Type: string

push_job_name

An identifier for push jobs.

Type: string

Default: "benthos_push"

push_basic_auth

The Basic Authentication credentials.

Type: object

push_basic_auth.username

The Basic Authentication username.

Type: string

Default: ""

push_basic_auth.password

The Basic Authentication password.

This field contains sensitive information that usually shouldn’t be added to a config directly, read our secrets page for more info.

Type: string

Default: ""

file_output_path

An optional file path to write all prometheus metrics on service shutdown.

Type: string

Default: ""

Push gateway

The field push_url is optional and when set will trigger a push of metrics to a Prometheus Push Gateway once Redpanda Connect shuts down. It is also possible to specify a push_interval which results in periodic pushes.

The Push Gateway is useful for when Redpanda Connect instances are short lived. Do not include the "/metrics/jobs/…​" path in the push URL.

If the Push Gateway requires HTTP Basic Authentication it can be configured with push_basic_auth.