Configure Client Connections

Optimize the availability of your clusters by configuring and tuning properties.

Limit client connections

A malicious Kafka client application may create many network connections to execute its attacks. A poorly configured application may also create an excessive number of connections. To mitigate the risk of a client creating too many connections and using too many system resources, you can configure a Redpanda cluster to impose limits on the number of created client connections.

The following Redpanda cluster properties limit the number of connections:

  • kafka_connections_max_per_ip: Similar to Kafka’s max.connections.per.ip, this sets the maximum number of connections accepted per IP address by a broker.

  • kafka_connections_max_overrides: A list of IP addresses for which kafka_connections_max_per_ip is overridden and doesn’t apply.

  • kafka_connections_max: Similar to Kafka’s max.connections, this sets the maximum number of connections per broker.

Redpanda also provides properties to manage the rate of connection creation:

  • These connection limit properties are disabled by default. You must manually enable them.

  • The total number of connections is not equal to the number of clients, because a client can open multiple connections. As a conservative estimate, for a cluster with N brokers, plan for N + 2 connections per client.

Configure client reconnections

You can configure the Kafka client backoff and retry properties to change the default behavior of the clients to suit your failure requirements.

Set the following Kafka client properties on your application’s producer or consumer to manage client reconnections:

  • reconnect.backoff.ms: Amount of time to wait before attempting to reconnect to the broker. The default is 50 milliseconds.

  • reconnect.backoff.max.ms: Maximum amount of time in milliseconds to wait when reconnecting to a broker. The backoff increases exponentially for each consecutive connection failure, up to this maximum. The default is 1000 milliseconds (1 second).

Additionally, you can use Kafka properties to control message retry behavior. Delivery fails when either the delivery timeout or the number of retries is met.

  • delivery.timeout.ms: Amount of time for message delivery, so messages are not retried forever. The default is 120000 milliseconds (2 minutes).

  • retries: Number of times a producer can retry sending a message before marking it as failed. The default value is 2147483647 for Kafka >= 2.1, or 0 for Kafka <= 2.0.

  • retry.backoff.ms: Amount of time to wait before attempting to retry a failed request to a given topic partition. The default is 100 milliseconds.

See also: Configure Producers