# grok

> For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [connect-full.txt](https://docs.redpanda.com/connect-full.txt)

---
title: grok
latest-connect-version: 4.93.0
latest-operator-version: v26.1.4
latest-console-tag: v3.7.3
latest-redpanda-tag: v26.1.9
docname: processors/grok
page-component-name: connect
page-version: master
page-component-version: master
page-component-title: Connect
page-relative-src-path: processors/grok.adoc
page-edit-url: https://github.com/redpanda-data/rp-connect-docs/edit/main/modules/components/pages/processors/grok.adoc
page-git-created-date: "2024-05-24"
page-git-modified-date: "2026-05-26"
---

<!-- Source: https://docs.redpanda.com/connect/components/processors/grok.md -->

**Available in:** Self-Managed

Parses messages into a structured format by attempting to apply a list of Grok expressions, the first expression to result in at least one value replaces the original message with a JSON object containing the values.

#### Common

```yml
processors:
  label: ""
  grok:
    expressions: [] # No default (required)
    pattern_definitions: {}
    pattern_paths: []
```

#### Advanced

```yml
processors:
  label: ""
  grok:
    expressions: [] # No default (required)
    pattern_definitions: {}
    pattern_paths: []
    named_captures_only: true
    use_default_patterns: true
    remove_empty_values: true
```

Type hints within patterns are respected, therefore with the pattern `%\{WORD:first},%{INT:second:int}` and a payload of `foo,1` the resulting payload would be `\{"first":"foo","second":1}`.

## [](#performance)Performance

This processor currently uses the [Go RE2](https://golang.org/s/re2syntax) regular expression engine, which is guaranteed to run in time linear to the size of the input. However, this property often makes it less performant than PCRE based implementations of grok. For more information, see [https://swtch.com/~rsc/regexp/regexp1.html](https://swtch.com/~rsc/regexp/regexp1.html).

## [](#examples)Examples

### [](#vpc-flow-logs)VPC Flow Logs

Grok can be used to parse unstructured logs such as VPC flow logs that look like this:

```text
2 123456789010 eni-1235b8ca123456789 172.31.16.139 172.31.16.21 20641 22 6 20 4249 1418530010 1418530070 ACCEPT OK
```

Into structured objects that look like this:

```json
{"accountid":"123456789010","action":"ACCEPT","bytes":4249,"dstaddr":"172.31.16.21","dstport":22,"end":1418530070,"interfaceid":"eni-1235b8ca123456789","logstatus":"OK","packets":20,"protocol":6,"srcaddr":"172.31.16.139","srcport":20641,"start":1418530010,"version":2}
```

With the following config:

```yaml
pipeline:
  processors:
    - grok:
        expressions:
          - '%{VPCFLOWLOG}'
        pattern_definitions:
          VPCFLOWLOG: '%{NUMBER:version:int} %{NUMBER:accountid} %{NOTSPACE:interfaceid} %{NOTSPACE:srcaddr} %{NOTSPACE:dstaddr} %{NOTSPACE:srcport:int} %{NOTSPACE:dstport:int} %{NOTSPACE:protocol:int} %{NOTSPACE:packets:int} %{NOTSPACE:bytes:int} %{NUMBER:start:int} %{NUMBER:end:int} %{NOTSPACE:action} %{NOTSPACE:logstatus}'
```

## [](#fields)Fields

### [](#expressions)`expressions[]`

One or more Grok expressions to attempt against incoming messages. The first expression to match at least one value will be used to form a result.

**Type**: `array`

### [](#named_captures_only)`named_captures_only`

Whether to only capture values from named patterns.

**Type**: `bool`

**Default**: `true`

### [](#pattern_definitions)`pattern_definitions`

A map of pattern definitions that can be referenced within `patterns`.

**Type**: `string`

**Default**: `{}`

### [](#pattern_paths)`pattern_paths[]`

A list of paths to load Grok patterns from. This field supports wildcards, including super globs (double star).

**Type**: `array`

**Default**: `[]`

### [](#remove_empty_values)`remove_empty_values`

Whether to remove values that are empty from the resulting structure.

**Type**: `bool`

**Default**: `true`

### [](#use_default_patterns)`use_default_patterns`

Whether to use a [default set of patterns](#default-patterns).

**Type**: `bool`

**Default**: `true`

## [](#default-patterns)Default patterns

For summary of the default patterns on offer, see [https://github.com/Jeffail/grok/blob/master/patterns.go#L5](https://github.com/Jeffail/grok/blob/master/patterns.go#L5).