Configure Client Connections
Optimize the availability of your clusters by configuring and tuning properties.
A malicious Kafka client application may create many network connections to execute its attacks. A poorly configured application may also create an excessive number of connections. To mitigate the risk of a client creating too many connections and using too many system resources, you can configure a Redpanda cluster to impose limits on the number of created client connections.
The following Redpanda cluster properties limit the number of connections:
kafka_connections_max: Similar to Kafka’s
max.connections, this sets the maximum number of connections per broker.
kafka_connections_max_per_ip: Similar to Kafka’s
max.connections.per.ip, this sets the maximum number of connections accepted per IP address by a broker.
kafka_connections_max_overrides: A list of IP addresses for which
kafka_connections_max_per_ipis overridden and doesn’t apply.
Redpanda also provides properties to manage the rate of connection creation:
You can configure the Kafka client backoff and retry properties to change the default behavior of the clients to suit your failure requirements.
The following Kafka properties let you manage client reconnections:
reconnect.backoff.ms: Amount of time to wait before attempting to reconnect to the broker. The default is 50 milliseconds.
reconnect.backoff.max.ms: Maximum amount of time in milliseconds to wait when reconnecting to a broker. The backoff increases exponentially for each consecutive connection failure, up to this maximum. The default is 1000 milliseconds (1 second).
Additionally, you can use Kafka properties to control message retry behavior. Delivery fails when either the delivery timeout or the number of retries is met.
delivery.timeout.ms: Amount of time for message delivery, so messages are not retried forever. The default is 120000 milliseconds (2 minutes).
retries: Number of times a producer can retry sending a message before marking it as failed. The default value is 2147483647 for Kafka >= 2.1, or 0 for Kafka <= 2.0.
retry.backoff.ms: Amount of time to wait before attempting to retry a failed request to a given topic partition. The default is 100 milliseconds.
See also: Configure Producers
A Redpanda broker may create log segments at startup. If a broker crashes after startup, and if it gets stuck in a crash loop, it could produce progressively more stored state that uses more disk space and takes more time for each restart to process.
To prevent infinite crash loops, the Redpanda node property
crash_loop_limit sets an upper limit on the number of consecutive crashes that can happen within one hour of each other. After it reaches the limit, a broker cannot restart until its internal consecutive crash counter is reset to zero by one of the following conditions:
redpanda.yamlconfiguration file is updated.
startup_logfile in the broker’s data_directory is manually deleted.
One hour has elapsed since the last crash.
The broker is properly shut down. (This is not possible after
crash_loop_limithas been reached and the broker cannot be restarted.)