rpk topic consume

Text and numbers can be formatted in many different ways, and the default format can be changed within brace modifiers. %v prints a value, while %v\{hex} prints the value hex encoded. %T prints the length of a topic in ASCII, while %T\{big8} prints the length of the topic as an eight byte big endian.

All modifiers go within braces following a percent-escape.

Formatting number values can have the following modifiers:

All numbers are truncated as necessary per the modifier. Printing %V\{byte} for a length 256 value will print a single null, whereas printing %V\{big8} would print the bytes 1 and 0.

When writing number sizes, the size corresponds to the size of the raw values, not the size of encoded values. %T% t\{hex} for the topic foo will print 3 666f6f, not 6 666f6f.

By default, the timestamp field is printed as a millisecond number value. In addition to the number modifiers above, timestamps can be printed with either Go formatting or strftime formatting:

An arbitrary amount of brackets (or braces, or # symbols) can wrap your date formatting:

The above will print [YYYY-MM-DD], while the surrounding three #+ on each side are used to wrap the formatting. Further details on Go time formatting can be found at https://pkg.go.dev/time, while further details on strftime formatting can be read by checking +man strftime.

Each record (or batch of records) has a set of possible attributes. Internally, these are packed into bit flags. Printing an attribute requires first selecting which attribute you want to print, and then optionally specifying how you want it to be printed:

Compression is by default printed as text ("none", "gzip", …​). Compression can be printed as a number with ;number, where number is any number formatting option described above. No compression is 0, gzip is 1, etc.

The record’s timestamp type is printed as -1 for very old records (before timestamps existed), 0 for client generated timestamps, and 1 for broker generated timestamps. Number formatting can be controlled with ";number".

Prints 1 if the record a part of a transaction or 0 if it is not. Number formatting can be controlled with ;number.

Prints 1 if the record is a commit marker or 0 if it is not. Number formatting can be controlled with ";number".

Text fields without modifiers default to writing the raw bytes. Alternatively, there are the following modifiers:

The hex modifier hex encodes the text, the base64 modifier base64 encodes the text with standard encoding, and the base64raw modifier encodes the text with raw standard encoding. The unpack modifier has a further internal specification, similar to timestamps above:

Unpacking text can allow translating binary input into readable output. If a value is a big-endian uint32, %v will print the raw four bytes, while %v{unpack[>I]} will print the number in as ascii. If unpacking exhausts the input before something is unpacked fully, an error message is appended to the output.

Headers are formatted with percent encoding inside of the modifier:

will print all headers with a space before the key and after the value, an equals sign between the key and value, and with the value hex encoded. Header formatting actually just parses the internal format as a record format, so all of the above rules about %K, %V, text, and numbers apply.

The --offset flag allows for specifying where to begin consuming, and optionally, where to stop consuming. The literal words "start" and "end" specify consuming from the start and the end.

There are a few options for timestamps, with each option being evaluated until one succeeds:

Durations can be relative to the current time or relative to a timestamp. If a duration is used for t1, that duration is relative to now. If a duration is used for t2, if t1 is a timestamp, then t2 is relative to t1. If a duration is used for t2, if t1 is a duration, then t2 is relative to now.

Durations are parsed simply:

For example,