Configure Client Connections

Optimize the availability of your clusters by configuring and tuning properties.

Limit client connections

A malicious Kafka client application may create many network connections to execute its attacks. A poorly configured application may also create an excessive number of connections. To mitigate the risk of a client creating too many connections and using too many system resources, you can configure a Redpanda cluster to impose limits on the number of created client connections.

The following Redpanda cluster properties limit the number of connections:

  • kafka_connections_max: Similar to Kafka’s max.connections, this sets the maximum number of connections per broker.

  • kafka_connections_max_per_ip: Similar to Kafka’s max.connections.per.ip, this sets the maximum number of connections accepted per IP address by a broker.

  • kafka_connections_max_overrides: A list of IP addresses for which kafka_connections_max_per_ip is overridden and doesn’t apply.

Redpanda also provides properties to manage the rate of connection creation:

  • These connection limit properties are disabled by default. You must manually enable them.

  • Typically, a client opens two or three connections, so the total number of connections is not equal to the number of clients. For example, to support 100 clients, you might set your connection limit to 300.

Configure client reconnections

You can configure the Kafka client backoff and retry properties to change the default behavior of the clients to suit your failure requirements.

The following Kafka properties let you manage client reconnections:

  • reconnect.backoff.ms: Amount of time to wait before attempting to reconnect to the broker. The default is 50 milliseconds.

  • reconnect.backoff.max.ms: Maximum amount of time in milliseconds to wait when reconnecting to a broker. The backoff increases exponentially for each consecutive connection failure, up to this maximum. The default is 1000 milliseconds (1 second).

Additionally, you can use Kafka properties to control message retry behavior. Delivery fails when either the delivery timeout or the number of retries is met.

  • delivery.timeout.ms: Amount of time for message delivery, so messages are not retried forever. The default is 120000 milliseconds (2 minutes).

  • retries: Number of times a producer can retry sending a message before marking it as failed. The default value is 2147483647 for Kafka >= 2.1, or 0 for Kafka <= 2.0.

  • retry.backoff.ms: Amount of time to wait before attempting to retry a failed request to a given topic partition. The default is 100 milliseconds.

See also: Configure Producers

Prevent crash loops

A Redpanda broker may create log segments at startup. If a broker crashes after startup, and if it gets stuck in a crash loop, it could produce progressively more stored state that uses more disk space and takes more time for each restart to process.

To prevent infinite crash loops, the Redpanda node property crash_loop_limit sets an upper limit on the number of consecutive crashes that can happen within one hour of each other. After it reaches the limit, a broker cannot restart until its internal consecutive crash counter is reset to zero by one of the following conditions:

  • The redpanda.yaml configuration file is updated.

  • The startup_log file in the broker’s data_directory is manually deleted.

  • One hour has elapsed since the last crash.

  • The broker is properly shut down. (This is not possible after crash_loop_limit has been reached and the broker cannot be restarted.)

  • The crash_loop_limit property is disabled by default. You must manually enable it by setting it to a non-zero value.

  • If the limit is less than two, the broker is blocked from restarting after every crash, until one of the reset conditions is met.