Manage Throughput
Redpanda supports applying throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. The purpose of this is to prevent unbounded network and disk usage of the broker by clients. Broker-wide limits apply to all clients connected to the broker and restrict total traffic on the broker. Client limits apply to a set of clients defined by their client_id
and help prevent a set of clients from starving other clients using the same broker.
Throughput throttling enforcement
Throughput limits are enforced by applying backpressure to clients. When a connection is in breach of the throughput limit, the throttler advises the client about the delay (throttle time) that would bring the rate back to the allowed level. Redpanda starts by adding a throttle_time_ms
field to responses. If that isn’t honored, delays are inserted on the connection’s next read operation. The throttling delay may not exceed the limit set by max_kafka_throttle_delay_ms
.
Broker-wide throughput limits
Broker-wide throughput limits account for all Kafka API traffic going in to or out of the broker. The limit values represent the allowed rate of data in bytes per second passing through in each direction. Redpanda also provides administrators the ability to exclude clients from throughput throttling and to fine-tune which Kafka request types are subject to throttling limits.
Broker-wide throughput limit properties
The properties for broker-wide throughput quota balancing are configured at the cluster level, for all brokers in a cluster:
Property | Description |
---|---|
A broker’s total throughput limit for ingress Kafka traffic. |
|
A broker’s total throughput limit for egress Kafka traffic. |
|
List of clients for whom broker-wide limits do not apply |
|
List of Kafka request types subject to broker-wide throughput limits; defaults to |
|
The maximum delay inserted in the data path of Kafka API requests to throttle them down. Configuring this to be less than the Kafka client timeout can ensure that the delay that’s inserted won’t be long enough to cause a client timeout by itself. |
|
Client throughput limits
Similar to Broker-wide throughput limits but for clients, Redpanda provides configurable throughput quotas that apply to an individual client or a group of clients.
The client throughput quotas limit the rates within each shard (logical CPU core) of a Redpanda broker’s node. The quotas are neither shared nor balanced between shards and brokers. |
Individual client throughput limit
To set throughput quota for a single client, configure the cluster properties target_quota_byte_rate
and target_fetch_quota_byte_rate
.
-
The
target_quota_byte_rate
property applies to a producer client that isn’t a member of a client group configured bykafka_client_group_byte_rate_quota
. It sets the maximum throughput quota of a client sending to a Redpanda broker. -
The
target_fetch_quota_byte_rate
property applies to a consumer client that isn’t a member of a client group configured bykafka_client_group_fetch_byte_rate_quota
. It sets the maximum throughput quota of a client fetching from a Redpanda broker.
The values of both target_quota_byte_rate
and target_fetch_quota_byte_rate
are throughput rate limits within a shard, in bytes per second.
Group of clients throughput limit
To set throughput quota for a group of clients, configure the cluster properties kafka_client_group_byte_rate_quota
and kafka_client_group_fetch_byte_rate_quota
.
-
The
kafka_client_group_byte_rate_quota
property applies to producer clients. It sets a maximum throughput quota of traffic sent to Redpanda from each producer in the group named by the property. -
The
kafka_client_group_fetch_byte_rate_quota
property applies to consumer clients. It sets a maximum throughput quota of traffic fetched from Redpanda by each consumer in the group named by the property.
Both kafka_client_group_byte_rate_quota
and kafka_client_group_fetch_byte_rate_quota
have the configuration value fields:
-
group_name
: a name for the group of clients. -
clients_prefix
: a client belonging to the group has this prefix in itsclient_id
property. -
quota
: the maximum throughput rate of each client in the group in bytes per second.
For kafka_client_group_fetch_byte_rate_quota , a group of consumer clients is not a Kafka consumer group, and the group_name field is not a Kafka consumer group ID. Instead, it is the set of clients fetching from a Redpanda broker that are configured by the property to be throughput limited.
|
An example configuration of kafka_client_group_byte_rate_quota
for two groups of producers:
"kafka_client_group_byte_rate_quota": [
{
"group_name": "group_1",
"clients_prefix": "producer_group_alone_producer",
"quota": 10240
},
{
"group_name": "group_2",
"clients_prefix": "producer_group_multiple",
"quota": 20480
},
]