Glossary

Bring Your Own Cloud (BYOC) is a fully-managed Redpanda Cloud deployment where clusters run in your private cloud, so all data is contained in your own environment.

For more information, see Redpanda Cloud Overview.

A component of the Kafka Connect framework that enables Redpanda to integrate with external systems, such as databases.

Redpanda managed connectors offer a simpler way to integrate your data than manually creating a solution with the Kafka API. You can set up and manage connectors in Redpanda Console.

For more information, see Managed Connectors.

This part of Redpanda Cloud enforces rules in the data plane, including cluster management, operations, and maintenance.

This part of Redpanda Cloud contains Redpanda clusters and other components, such as Redpanda Console, Redpanda Operator, and rpk. It is managed by an agent that receives cluster specifications from the control plane. Sometimes used interchangeably with clusters.

A fully-managed Redpanda Cloud deployment option where clusters run in Redpanda-controlled accounts and regions in various cloud providers.

A fully-managed data streaming service deployed with Redpanda Console. It includes automated upgrades and patching, backup and recovery, data and partition balancing, built-in connectors, and 24x7 support. It is available in Serverless, Dedicated, and Bring Your Own Cloud (BYOC) deployment options to suit different data sovereignty and infrastructure requirements.

The web-based UI for managing and monitoring Redpanda clusters and streaming workloads. You can also set up and manage connectors in Redpanda Console. Redpanda Console is an integral part of Redpanda Cloud, but it also can be used as a standalone program in a self-hosted Redpanda deployment.

Exports data from a Redpanda cluster into a target system.

Imports data from a source system into a Redpanda cluster.

One or more data centers served by high-bandwidth links with low latency, typically within a close distance of one another.

An instance of Redpanda that stores and manages event streams. Multiple brokers join together to form a Redpanda cluster.

Sometimes used interchangeably with node, but a node is typically a physical or virtual server.

See also: node

A producer application that writes events to Redpanda, or a consumer application that reads events from Redpanda.

This could also be a client library, like librdkafka or franz-go.

One or more brokers that work together to manage real-time data streaming, processing, and storage.

A set of consumers that cooperate to read data for better scalability. As group members arrive and leave, partitions are re-assigned so each member receives a proportional share.

The position of a consumer in a specific topic partition, to track which records they have read. A consumer offset of 3 means it has read messages 0-2 and will next read message 3.

A client application that subscribes to Redpanda topics to asynchronously read events.

A broker that’s responsible for managing operational metadata for a Redpanda cluster. At any given time, one active controller exists in a cluster.

Controller responsibilities:

If a controller fails, another broker is automatically elected as the controller.

Containing all your data in your environment.

With BYOC, Redpanda handles provisioning, monitoring, and upgrades, but you manage your streaming data without Redpanda’s control plane ever seeing it. Additionally, with a customer-managed VPC, the Redpanda Cloud agent doesn’t create any new resources or alter any settings in your account.

A continuous flow of events in real time that are produced and consumed by client applications. Redpanda is a data streaming platform. Also known as event stream.

A record of something changing state at a specific time. Events can be generated by various sources, including sensors, applications, and devices. Producers write events to Redpanda, and consumers read events from Redpanda.

Producers and consumers interact with Redpanda using the Kafka API. It uses the default port 9092.

A broker that is a follower in a Raft group but is not part of quorum.

In a Raft group, a broker can be in learner status. Learners are followers that cannot vote and so do not count towards quorum (the majority). They cannot be elected to leader nor can they trigger leader elections. Brokers can be promoted or demoted between learner and voter. New Raft group members start as learners.

For more information, see Raft Group Reconfiguration.

Configuration on a broker that defines how it should accept client or inter-broker connections. Each listener is associated with a specific protocol, hostname, and port combination. The listener defines where the broker should listen for incoming connections.

For more information, see Configure Listeners.

One or more records representing individual events being transmitted. Redpanda transfers messages between producers and consumers.

Sometimes used interchangeably with record.

A machine, which could be a server, a virtual machine (instance), or a Docker container. Every node has its own disk. Partitions are stored locally on nodes. In Kubernetes, a Node is the machine that Redpanda runs on. Outside the context of Kubernetes, this term may be used interchangeably with broker, such as node_id.

See also: broker

An acknowledgement that the event has been read.

A unique integer assigned to each record to show its location in the partition.

Original name for the subsystem of Redpanda that allows access to your data through a REST API. This name still appears in the HTTP Proxy API and the Schema Registry API.

Every Redpanda partition forms a Raft group with a single elected leader. This leader handles all reads and writes, and it replicates data to followers to ensure that a majority of brokers store the data.

A subset of events in a topic, like a log file. It is an ordered, immutable sequence of records. Partitions allow you to distribute a stream, which lets producers write messages in parallel and consumers read messages in parallel. Partitions are made up of segment files on disk.

A client application that writes events to Redpanda. Redpanda stores these events in sequence and organizes them into topics.

For more information, see Configure Producers.

A failure zone that has one or more Redpanda brokers assigned to it.

The consensus algorithm Redpanda uses to coordinate writing data to log files and replicating that data across brokers.

For more details, see https://raft.github.io/

A self-contained data entity with a defined structure, representing a single event.

Sometimes used interchangeably with message.

Copies of partitions that are distributed across different brokers, so if one broker goes down, there is a copy of the data.

The mechanism for determining how long Redpanda stores data on local disk or in object storage before purging it.

For more information, see Manage Disk Space.

The number of copies of partitions in a cluster. By default, replication.factor is set to 1 (no replication). With a replication factor greater than 1, you ensure that each partition has a copy of its data on at least one other broker. One replica acts as the leader, and the other replicas are followers.

An external mechanism to describe the structure of data and its encoding. Schemas validate the structure and ensure that producers and consumers can connect with data in the same format.

An open-source thread-per-core C++ framework, which binds all work to physical cores. Redpanda is built on Seastar.

For more details, see https://seastar.io/

The initial set of brokers that a Redpanda broker contacts to join the cluster. Seed servers play a crucial role in cluster formation and recovery, acting as a point of reference for new or restarting brokers to understand the current topology of the cluster.

Discrete part of a partition, used to break down a continuous stream into manageable chunks. You can set the maximum duration (segment.ms) or size (segment.bytes) for a segment to be open for writes.

The process of converting a record into a format that can be stored. Deserialization is the process of converting a record back to the original state. Redpanda Schema Registry supports Avro and Protobuf serialization formats.

A CPU core.

A logical grouping or category for schemas. When data formats are updated, a new version of the schema can be registered under the same subject, allowing for backward and forward compatibility.

Programming model that allows Redpanda to pin each of its application threads to a CPU core to avoid context switching and blocking.

A logical stream of related events that are written to the same log. It can be divided into multiple partitions. A topic can have various clients writing events to it and reading events from it.

A REST API used to manage and monitor Redpanda clusters. It uses the default port 9644.

For more information, see Admin API.

Note: The Redpanda Admin API is different from the Kafka Admin API.

Feature that retains the latest value for each key within a partition while discarding older values.

For more information, see Compaction Settings.

Snapshot of the current cluster metadata state saved to disk, so broker startup is fast.

Framework to manipulate or enrich data written to Redpanda topics. You can develop custom data functions, which run asynchronously using a WebAssembly (Wasm) engine inside a Redpanda broker.

For more information, see Data Transforms.

Redpanda HTTP Proxy (pandaproxy) allows access to your data through a REST API. It is built into the Redpanda binary and uses the default port 8082.

For more information, see Use Redpanda with the HTTP Proxy API.

A state where a Redpanda broker temporarily doesn’t take any partition leaderships. It continues to store data as a follower. This is usually done for system maintenance or a rolling upgrade.

For more information, see Maintenance Mode.

Feature that lets you distribute replicas of the same partition across different racks to minimize data loss and improve fault tolerance in the event of a rack failure.

For more information, see Enable Rack Awareness.

Process of redistributing partitions and partition leadership for improved performance.

By default, Redpanda provides leadership balancing and partition rebalancing when brokers are added or decommissioned.

With an Enterprise license, you can additionally enable Continuous Data Balancing to continuously monitor broker and rack availability and disk usage.

For more information, see Cluster Balancing.

The process of upgrading each broker in a Redpanda cluster, one at a time, to minimize disruption and ensure continuous availability.

For more information, see Upgrade Redpanda.

Redpanda’s command-line interface tool for managing Redpanda clusters.

For more information, see rpk.

A read-only topic that mirrors a topic on a different cluster, using data from Tiered Storage.

For more information, see Remote Read Replicas.

Redpanda Schema Registry (pandaproxy) is the interface for storing and managing event schemas. Producers and consumers register and retrieve schemas they use from the registry. It is built into the Redpanda binary and uses the default port 8081.

For more information, see Schema Registry.

Feature that lets you offload log segments to object storage in near real-time, providing long-term data retention and topic recovery.

For more information, see Tiered Storage.

A Kubernetes controller that simplifies the process of obtaining, renewing, and using certificates.

For more details, see https://cert-manager.io/docs/

Generates and applies all the manifest files you need for deploying Redpanda in Kubernetes.

For more information, see Redpanda in Kubernetes.

Extends Kubernetes with custom resource definitions (CRDs), which allow Redpanda clusters to be treated as native Kubernetes resources.

For more information, see Redpanda in Kubernetes.

Redpanda software that is available under the Redpanda Business Source License (BSL). These core features are free and source-available.

Redpanda software that is available under the Redpanda Community License (RCL). It includes the free features licensed with the Redpanda Community Edition, as well enterprise features, such as Tiered Storage, Remote Read Replicas, and Continuous Data Balancing.

A deployment on Kubernetes or Linux where you host and manage Redpanda clusters yourself.

Sometimes used interchangeably with self-managed.

A security feature used to define and enforce granular permissions to resources, ensuring only authorized users or applications can perform specific operations. ACLs act on principals.

For more information, see Configure Authorization.

The address a Redpanda broker broadcasts to producers, consumers, and other brokers. It specifies the hostname and port for connections to different listeners. Clients and other brokers use advertised listeners to connect to services such as the Admin API, Kafka API, and HTTP Proxy API. The advertised address might differ from the listener address in scenarios where brokers are behind a NAT, in a Docker container, or in Kubernetes. Advertised addresses ensure clients can reach the Redpanda brokers even in complex network setups.

The process of verifying the identity of a principal, user, or service account.

For more information, see Configure Authentication.

The process of specifying access rights to resources. Access rights are enforced through access-control lists (ACLs).

For more information, see Configure Authorization.

An access token used for authentication and authorization in web applications and APIs. It holds user credentials, usually in the form of random strings of characters.

For more information, see Configure Authentication.

An entity (such as a user account or a service account) that accesses resources. Principals can be authenticated and granted permissions based on roles to perform operations.

An identity independent of the user who created it that can be used to authenticate and perform operations. This is especially useful for authentication of machines.