rpk topic

Text and numbers can be formatted in many different ways, and the default format can be changed within brace modifiers. %v prints a value, while %v{hex} prints the value hex encoded. %T prints the length of a topic in ascii, while %T{big8} prints the length of the topic as an eight byte big endian.

All modifiers go within braces following a percent-escape.

Formatting number values can have the following modifiers:

All numbers are truncated as necessary per the modifier. Printing %V{byte} for a length 256 value will print a single null, whereas printing %V{big8} would print the bytes 1 and 0.

When writing number sizes, the size corresponds to the size of the raw values, not the size of encoded values. %T% t{hex} for the topic foo will print 3 666f6f, not 6 666f6f.

By default, the timestamp field is printed as a millisecond number value. In addition to the number modifiers above, timestamps can be printed with either Go formatting:

Or strftime formatting:

An arbitrary amount of brackets (or braces, or # symbols) can wrap your date formatting:

The above will print [YYYY-MM-DD], while the surrounding three # on each side are used to wrap the formatting.

Further details on Go time formatting can be found at Go’s time documentation website.

Further details on strftime formatting can be read by running man strftime.

Text fields without modifiers default to writing the raw bytes. Alternatively, there are the following modifiers:

The hex modifier hex encodes the text, the base64 modifier base64 encodes the text with standard encoding, and the base64raw modifier encodes the text with raw standard encoding. The unpack modifier has a further internal specification, similar to timestamps above: |Match the end of the line (append error string if anything remains).

Unpacking text can allow translating binary input into readable output. If a value is a big-endian uint32, %v will print the raw four bytes, while %v{unpack[>I]} will print the number in as ascii. If unpacking exhausts the input before something is unpacked fully, an error message is appended to the output.

Headers are formatted with percent encoding inside of the modifier:

will print all headers with a space before the key and after the value, an equals sign between the key and value, and with the value hex encoded. Header formatting actually just parses the internal format as a record format, so all of the above rules about %K, %V, text, and numbers apply.

A key and value, separated by a space and ending in newline:

A key length as four big endian bytes, and the key as hex:

A little endian uint32 and a string unpacked from a value:

The --offset flag allows for specifying where to begin consuming, and optionally, where to stop consuming. The literal words start and end specify consuming from the start and the end.

There are a few options for timestamps, with each option being evaluated until one succeeds:

Durations are parsed simply:

For example:

Producing requires a topic to produce to. The topic can be specified either directly on as an argument, or in the input text through %t. A parsed topic takes precedence over the default passed in topic. If no topic is specified directly and no topic is parsed, this command will quit with an error.

The input format can parse partitions to produce directly to with %p. Doing so requires specifying a non-negative --partition flag. Any parsed partition takes precedence over the --partition flag; specifying the flag is the main requirement for being able to directly control which partition to produce to.

You can also specify an output format to write when a record is produced successfully. The output format follows the same formatting rules as the topic consume command. See that command’s help text for a detailed description.