rpk topic

You can use rpk topic to manage your topics, create new messages and consume them.

Formatting

Formatting is based on percent escapes and modifiers. Slashes can be used for common escapes:

Escape

Description

` \t `

Tabs.

\n

Newlines

\r

Carriage returns.

\\

Slashes.

\xNN

Hex encoded characters.

The percent encodings are represented like this:

Percent encoding

Description

%t

Topic.

%T

Topic length.

%k

Key.

%K

Key length.

%v

Value.

%V

Value length.

%h

Begin the header specification.

%H

Number of headers.

%p

Partition.

%o

Offset.

%e

Leader epoch.

%d

Timestamp (formatting described below).

%x

Producer id.

%y

Producer epoch.

%[

Partition log start offset.

`%

`

Partition last stable offset.

%]

Partition high watermark.

%%

Percent sign

%{

Left brace.

%}

Right brace.

%i

Modifiers

Text and numbers can be formatted in many different ways, and the default format can be changed within brace modifiers. %v prints a value, while %v{hex} prints the value hex encoded. %T prints the length of a topic in ascii, while %T{big8} prints the length of the topic as an eight byte big endian.

All modifiers go within braces following a percent-escape.

Numbers

Formatting number values can have the following modifiers:

Format

Description

ascii

Print the number as ascii (default).

hex64

Sixteen hex characters.

hex32

Eight hex characters.

hex16

Four hex characters.

hex8

Two hex characters.

hex4

One hex character.

big64

Eight byte big endian number.

big32

Four byte big endian number.

big16

Two byte big endian number.

big8

Alias for byte.

little64

Eight byte little endian number

little32

Four byte little endian number.

little16

Two byte little endian number.

little8

Alias for byte.

byte

One byte number.

All numbers are truncated as necessary per the modifier. Printing %V{byte} for a length 256 value will print a single null, whereas printing %V{big8} would print the bytes 1 and 0.

When writing number sizes, the size corresponds to the size of the raw values, not the size of encoded values. %T% t{hex} for the topic foo will print 3 666f6f, not 6 666f6f.

Timestamps

By default, the timestamp field is printed as a millisecond number value. In addition to the number modifiers above, timestamps can be printed with either Go formatting:

%d{go[2006-01-02T15:04:05Z07:00]}

Or strftime formatting:

%d{strftime[%F]}

An arbitrary amount of brackets (or braces, or # symbols) can wrap your date formatting:

%d{strftime### [%F] ###}

The above will print [YYYY-MM-DD], while the surrounding three # on each side are used to wrap the formatting.

Further details on Go time formatting can be found at Go’s time documentation website.

Further details on strftime formatting can be read by running man strftime.

Text

Text fields without modifiers default to writing the raw bytes. Alternatively, there are the following modifiers:

Modifier Description

%t{hex}

Hex encoding

%k{base64}

Base64 encoding

%v{unpack[<bBhH>iIqQc.$]}

The unpack modifier has a further internal specification, similar to timestamps above.

The hex modifier hex encodes the text, and the base64 modifier base64 encodes the text with standard encoding. The unpack modifier has a further internal specification, similar to timestamps above:

Character Description

x

Pad character (does not parse input).

<

Switch what follows to little endian.

>

Switch what follows to big endian.

b

Signed byte.

B

Unsigned byte.

h

int16 ("half word").

H

uint16 ("half word").

i

int32.

I

uint32.

q

int64 ("quad word").

Q

uint64 ("quad word").

c

Any character.

.

Alias for c.

s

Consume the rest of the input as a string