23.2 23.1 22.3 22.2 22.1 21.11

Frequently Asked Questions

Getting started with Redpanda

The FAQs in this section are related to the Redpanda platform.

Redpanda is Kafka API compatible so any client that works with Kafka should work out of the box with Redpanda. However here is a list of clients which we have tested with:

Language Client Link

Java

Apache Kafka Java Client

https://github.com/apache/kafka

C/C++

librdkafka

https://github.com/edenhill/librdkafka

Go

franz-go

https://github.com/twmb/franz-go

Python

kafka-python

https://pypi.org/project/kafka-python

JS

KafkaJS

https://kafka.js.org

PHP

php-simple-kafka-client

https://github.com/php-kafka/php-simple-kafka-client

Is Redpanda Fully Kafka API compatible?

Redpanda works to ensure all Kafka-based application are compatible with the Redpanda platform. The transactions API is in technical preview. but can be enabled by setting both enable_idempotence and enable_transactions to true in the Redpanda config.

If you run into any issues while working with a Kafka tool, please let us know! File an issue

Does Redpanda use Zookeeper?

No, Redpanda is a modern streaming platform that has been built using C++ and Raft for consensus. Since we use Raft we have no need for an external consensus system like Zookeeper.

Can I run Redpanda directly on Windows or MacOS?

Unfortunately, you can only run Redpanda directly on Linux. However, you can use Docker to run Redpanda on any system supported by Docker. Please see our Quick Start Docker Guide for more information.

Consumers and topics

How do consumer groups and topics interact?

  • A consumer group allows many topics to be consumed.

  • A consumer within a group can subscribe to multiple topics.

  • For each topic that a group subscribes to, the records are sent to one consumer within the group at least once.

  • Consumer groups are separate. Each group receives each record from each subscribed topic at least once.

Topics are split into partitions.

  • Every partition is handled by exactly one leader node in the cluster, so if you want to fully use your cluster you need at least the same amount of partitions as you have nodes.

  • The records within a partition are ordered.

  • There is no ordering between records in different partitions in the same topic.

The partition is the unit of scale.

  • Only one consumer in a group can receive records from a given partition at a specific time. This might change as the group is rebalanced.

  • If a consumer cannot keep up, reduce the work per consumer by adding additional consumers. You may also need to increase the number of partitions: because partitions are assigned exclusively to a single consumer, you should have at least three times the number of partitions as consumers to allow for a balanced assignment.