# Manage Throughput > For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [streaming-full.txt](https://docs.redpanda.com/streaming-full.txt) --- title: Manage Throughput latest-operator-version: v26.1.4 # EOL = End-of-Life (support lifecycle status) page-is-nearing-eol: "false" page-is-past-eol: "true" page-eol-date: July 31, 2025 latest-console-tag: v3.7.3 latest-connect-version: 4.93.0 docname: cluster-maintenance/manage-throughput page-component-name: streaming page-version: "24.2" page-component-version: "24.2" page-component-title: Streaming page-relative-src-path: cluster-maintenance/manage-throughput.adoc page-edit-url: https://github.com/redpanda-data/docs/edit/v/24.2/modules/manage/pages/cluster-maintenance/manage-throughput.adoc description: Manage the throughput of Kafka traffic with configurable properties. page-git-created-date: "2023-08-03" page-git-modified-date: "2024-08-01" support-status: past end-of-life --- Redpanda supports applying throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. The purpose of this is to prevent unbounded network and disk usage of the broker by clients. Broker-wide limits apply to all clients connected to the broker and restrict total traffic on the broker. Client limits apply to a set of clients defined by their `client_id` and help prevent a set of clients from starving other clients using the same broker. ## [](#throughput-throttling-enforcement)Throughput throttling enforcement > 📝 **NOTE** > > As of v24.2, Redpanda enforces all throughput limits per broker, including client throughput. Throughput limits are enforced by applying backpressure to clients. When a connection is in breach of the throughput limit, the throttler advises the client about the delay (throttle time) that would bring the rate back to the allowed level. Redpanda starts by adding a `throttle_time_ms` field to responses. If that isn’t honored, delays are inserted on the connection’s next read operation. The throttling delay may not exceed the limit set by [`max_kafka_throttle_delay_ms`](https://docs.redpanda.com/streaming/24.2/reference/properties/cluster-properties/#max_kafka_throttle_delay_ms). ## [](#broker-wide-throughput-limits)Broker-wide throughput limits Broker-wide throughput limits account for all Kafka API traffic going into or out of the broker, as data is produced to or consumed from a topic. The limit values represent the allowed rate of data in bytes per second passing through in each direction. Redpanda also provides administrators the ability to exclude clients from throughput throttling and to fine-tune which Kafka request types are subject to throttling limits. ### [](#broker-wide-throughput-limit-properties)Broker-wide throughput limit properties The properties for broker-wide throughput quota balancing are configured at the cluster level, for all brokers in a cluster: | Cluster configuration property | Description | | --- | --- | | kafka_throughput_limit_node_in_bps | A broker’s total throughput limit for ingress Kafka traffic. | | kafka_throughput_limit_node_out_bps | A broker’s total throughput limit for egress Kafka traffic. | | kafka_throughput_control | List of clients for whom broker-wide limits do not apply | | kafka_throughput_controlled_api_keys | List of Kafka request types subject to broker-wide throughput limits; defaults to produce and fetch. | | max_kafka_throttle_delay_ms | The maximum delay inserted in the data path of Kafka API requests to throttle them down. Configuring this to be less than the Kafka client timeout can ensure that the delay that’s inserted won’t be long enough to cause a client timeout by itself. | > 📝 **NOTE** > > - By default, both `kafka_throughput_limit_node_in_bps` and `kafka_throughput_limit_node_out_bps` are disabled, and no throughput limits are applied. You must manually set them to enable throughput throttling. ## [](#client-throughput-limits)Client throughput limits Redpanda provides configurable throughput quotas that apply to an individual client or a group of clients. You can apply a quota for an individual client based on an exact match with its `client_id`, or a group of clients based on IDs that start with a given prefix. As of v24.2, client throughput quotas are compatible with the [AlterClientQuotas and DescribeClientQuotas](https://cwiki.apache.org/confluence/display/KAFKA/KIP-546%3A+Add+Client+Quota+APIs+to+the+Admin+Client) Kafka APIs, and are separate from quotas configured through cluster configuration in earlier Redpanda versions. The client throughput quotas no longer apply on a per-shard basis, and now limit the rates across a Redpanda broker’s node. The quotas are neither shared nor balanced between brokers. Redpanda supports the following Kafka API-based quota types on clients: | Quota type | Description | | --- | --- | | producer_byte_rate | Limit throughput of produce requests | | consumer_byte_rate | Limit throughput of fetch requests | | controller_mutation_rate | Limit rate of topic mutation requests, including create, add, and delete partition, in number of partitions per second | You can also apply a default quota for all other client requests that don’t have a specific quota based on an exact match or `client_id` prefix. It is possible to create conflicting quotas if you configure the same quotas through both the Kafka API and a cluster configuration. Redpanda resolves these conflicts by following an order of preference in finding a matching quota for a request: 1. Quota configured through the Kafka API for an exact match on `client_id` 2. Quota configured through the Kafka API for a prefix match on `client_id` 3. Quota configured through cluster configuration properties (`kafka_client_group_byte_rate_quota`, `kafka_client_group_fetch_byte_rate_quota`, [deprecated starting in v24.2](https://docs.redpanda.com/streaming/24.2/upgrade/deprecated/)) for a prefix match on `client_id` 4. Default quota configured through the Kafka API on `client_id` 5. Default quota configured through cluster configuration properties (`target_quota_byte_rate`, `target_fetch_quota_byte_rate`, `kafka_admin_topic_api_rate`, [deprecated starting in v24.2](https://docs.redpanda.com/streaming/24.2/upgrade/deprecated/)) on `client_id` Redpanda recommends [migrating](#migrate) over from cluster configuration-managed quotas to Kafka-compatible quotas. You can re-create the configuration-based quotas with `rpk`, and then remove the cluster configurations. ### [](#individual-client-throughput-limit)Individual client throughput limit To view current throughput quotas set through the Kafka API, run [`rpk cluster quotas describe`](https://docs.redpanda.com/streaming/24.2/reference/rpk/rpk-cluster/rpk-cluster-quotas-describe/). For example, to see the quotas for client ID `consumer-1`: ```bash rpk cluster quotas describe --name client-id=consumer-1 ``` ```bash client-id=consumer-1 producer_byte_rate=140000 ``` To set a throughput quota for a single client, use the [`rpk cluster quotas alter`](https://docs.redpanda.com/streaming/24.2/reference/rpk/rpk-cluster/rpk-cluster-quotas-alter/) command. ```bash rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id=consumer-1 ``` ```bash ENTITY STATUS client-id=consumer-1 OK ``` ### [](#group-of-clients-throughput-limit)Group of clients throughput limit Alternatively, you can view or configure throughput quotas for a group of clients based on a match on client ID prefix. The following example sets the `consumer_byte_rate` quota to client IDs prefixed with `consumer-`: ```bash rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer- ``` > 📝 **NOTE** > > A client group specified with `client-id-prefix` is not the equivalent of a Kafka consumer group. It is used only to match requests based on the `client_id` prefix. The `client_id` field is typically a configurable property when you create a client with Kafka libraries. ### [](#default-client-throughput-limit)Default client throughput limit You can apply default throughput limits to clients. Redpanda applies the default limits if no quotas are configured for a specific `client_id` or prefix. To specify a produce quota of 1 GBps through the Kafka API (which means a 1 GBps limit across all produce requests sent to a single Redpanda broker), run: ```bash rpk cluster quotas alter --default client-id --add producer_byte_rate=1000000000 ``` ### [](#bulk-manage-client-throughput-limits)Bulk manage client throughput limits To more easily manage multiple quotas, you can use the `cluster quotas describe` and [`cluster quotas import`](https://docs.redpanda.com/streaming/24.2/reference/rpk/rpk-cluster/rpk-cluster-quotas-import/) commands to do a bulk export and update. For example, to export all client quotas in JSON format: ```bash rpk cluster quotas describe --format json ``` `rpk cluster quotas import` accepts the output string from `rpk cluster quotas describe --format `: ```bash rpk cluster quotas import --from '{"quotas":[{"entity":[{"name":"foo","type":"client-id"}],"values":[{"key":"consumer_byte_rate","values":"12123123"}]},{"entity":[{"name":"foo-","type":"client-id-prefix"}],"values":[{"key":"producer_byte_rate","values":"12123123"},{"key":"consumer_byte_rate","values":"4444444"}]}]}' ``` You can also save the JSON or YAML output to a file and pass the file path in the `--from` flag. ### [](#migrate)Migrate cluster configuration quotas to Kafka API-based quotas 1. Use [`rpk cluster config get`](https://docs.redpanda.com/streaming/24.2/reference/rpk/rpk-cluster/rpk-cluster-config-get/) to view current client quotas managed with cluster configuration. The following example shows how to retrieve the `kafka_client_group_byte_rate_quota` for two groups of producers: ```bash rpk cluster config get kafka_client_group_byte_rate_quota ``` ```bash "kafka_client_group_byte_rate_quota": [ { "group_name": "group_1", "clients_prefix": "producer_group_alone_producer", "quota": 10240 }, { "group_name": "group_2", "clients_prefix": "producer_group_multiple", "quota": 20480 }, ] ``` 2. Each client quota cluster property ([deprecated in v24.2](https://docs.redpanda.com/streaming/24.2/upgrade/deprecated/)) corresponds to a quota type in Kafka. Check the corresponding `rpk` arguments to use when setting the new quota values: | Cluster configuration property | rpk cluster quotas arguments | | --- | --- | | target_quota_byte_rate | --default client-id --add producer_byte_rate= | | target_fetch_quota_byte_rate | --default client-id --add consumer_byte_rate= | | kafka_admin_topic_api_rate | --default client-id --add controller_mutation_rate= | | kafka_client_group_byte_rate_quota | --name client-id-prefix= --add producer_byte_rate= | | kafka_client_group_fetch_byte_rate_quota | --name client-id-prefix= --add consumer_byte_rate= | The client throughput quotas set through the Kafka API apply per broker, so you must convert the cluster configuration values that were applied on a per-shard (logical CPU core) basis. For example, if you set `target_fetch_quota_byte_rate` to 100 MBps/shard, and you run Redpanda on 16-core brokers, you can set the new consumer\_byte\_rate quota to 100 \* 16 = 1600 MBps. 3. Use `rpk cluster quotas alter` to set the corresponding client throughput quotas based on the Kafka API: ```bash rpk cluster quotas alter --name client-id-prefix=producer_group_alone_producer --add producer_byte_rate= rpk cluster quotas alter --name client-id-prefix=producer_group_multiple --add producer_byte_rate= ``` Replace the placeholder values with the new quota values, accounting for the conversion to per-broker limits. For example, 10240 \* broker core count = new quota. 4. Use [`rpk cluster config set`](https://docs.redpanda.com/streaming/24.2/reference/rpk/rpk-cluster/rpk-cluster-config-set/) to remove the configuration-based quotas: ```bash rpk cluster config set kafka_client_group_byte_rate_quota= ``` ### [](#view-throughput-limits-in-redpanda-console)View throughput limits in Redpanda Console You can also use Redpanda Console to view enforced limits. In the menu, go to **Quotas**. ### [](#monitor-client-throughput)Monitor client throughput The following metrics are available on both the `/public_metrics` and `/metrics` endpoints to provide insights into client throughput quota usage: - Client quota throughput per rule and quota type: - `/public_metrics` - [`redpanda_kafka_quotas_client_quota_throughput`](https://docs.redpanda.com/streaming/24.2/reference/public-metrics-reference/#redpanda_kafka_quotas_client_quota_throughput) - `/metrics` - [`vectorized_kafka_quotas_client_quota_throughput`](https://docs.redpanda.com/streaming/24.2/reference/internal-metrics-reference/#vectorized_kafka_quotas_client_quota_throughput) - Client quota throttling delay per rule and quota type, in seconds: - `/public_metrics` - [`redpanda_kafka_quotas_client_quota_throttle_time`](https://docs.redpanda.com/streaming/24.2/reference/public-metrics-reference/#redpanda_kafka_quotas_client_quota_throttle_time) - `/metrics` - [`vectorized_kafka_quotas_client_quota_throttle_time`](https://docs.redpanda.com/streaming/24.2/reference/internal-metrics-reference/#vectorized_kafka_quotas_client_quota_throttle_time) The `kafka_quotas` logger provides details at the trace level on client quota throttling: ```bash TRACE 2024-06-14 15:36:05,240 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns TRACE 2024-06-14 15:36:05,240 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:51256] throttle request:{snc:0, client:184}, enforce:{snc:-365123762, client:-365123762}, key:0, request_size:1316 TRACE 2024-06-14 15:37:44,835 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 119, delay:0ns, capped_delay:0ns TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:58636] throttle request:{snc:0, client:184}, enforce:{snc:-14359, client:-14359}, key:0, request_size:1316 ```