Optimize I/O
Redpanda relies on its own disk I/O scheduler, and by default, it tells the kernel to
use the noop scheduler. Additionally, rpk comes with an embedded database of I/O settings, which are specific combinations of CPUs, SSD types, and VM sizes. Because running software on four VCPUs isn’t the same as running on an EC2 i3.metal with 96 physical cores, Redpanda tries to predict the best known settings for VM cloud types.
It isn’t necessary to run rpk iotune each time Redpanda is started. The output properties for rpk iotune are written to a file that can be saved and reused in nodes running on the same type of hardware.
|
Currently, Redpanda has well-known-types for AWS and GCP. On startup, rpk tries to detect the cloud and
instance type from the cloud’s metadata API, setting the correct iotune properties.
If access to the metadata API isn’t allowed from the instance, you can hint the desired setup by passing the --well-known-io flag with the cloud vendor, VM type, and storage type:
rpk start --well-known-io 'aws:i3.xlarge:default'It can also be specified in the redpanda.yaml configuration file, under the rpk
object:
rpk:
well_known_io: 'gcp:c2-standard-16:nvme'If well-known-io is specified in the config file and also as a flag, then the flag takes precedence.
If a certain cloud vendor, machine type, or storage type isn’t
found, or if the metadata isn’t available and no hint is given, then rpk prints a
warning and continues to use the default values.