Glossary

Bring Your Own Cloud (BYOC) is a fully-managed Redpanda Cloud deployment where clusters run in your private cloud, so all data is contained in your own environment. Redpanda handles provisioning, operations, and maintenance.

Enables Redpanda to integrate with external systems, such as databases.

This part of Redpanda Cloud enforces rules in the data plane, including cluster management, operations, and maintenance.

This part of Redpanda Cloud contains Redpanda clusters and other components, such as Redpanda Console, Redpanda Operator, and rpk. It is managed by an agent that receives cluster specifications from the control plane. Sometimes used interchangeably with clusters.

A fully-managed Redpanda Cloud deployment option where you host your data in Redpanda’s VPC, and Redpanda handles provisioning, operations, and maintenance. Dedicated clusters are single-tenant deployments that support private networking (for example, VPC peering to talk over private IPs) for better data isolation.

A fully-managed data streaming service deployed with Redpanda Console. It includes automated upgrades and patching, backup and recovery, data and partition balancing, and built-in connectors. It is available in Serverless, Dedicated, and Bring Your Own Cloud (BYOC) deployment options to suit different data sovereignty and infrastructure requirements.

The web-based UI for managing and monitoring Redpanda clusters and streaming workloads. You can also set up and manage connectors in Redpanda Console. Redpanda Console is an integral part of Redpanda Cloud, but it also can be used as a standalone program as part of a Redpanda Self-Managed deployment.

A container for Redpanda Cloud resources, including clusters and networks. You can rename your default resource group, and you can create more resource groups. For example, you may want different resource groups for production and testing.

Serverless is a fully-managed Redpanda Cloud deployment where you host your data in Redpanda’s VPC, and Redpanda handles automatic scaling, provisioning, operations, and maintenance. A cluster is available instantly, there is no base cost, and you only pay for what you consume.

Exports data from a Redpanda cluster into a target system.

Imports data from a source system into a Redpanda cluster.

One or more data centers served by high-bandwidth links with low latency, typically within a close distance of one another.

An instance of Redpanda that stores and manages event streams. Multiple brokers join together to form a Redpanda cluster.

Sometimes used interchangeably with node, but a node is typically a physical or virtual server.

See also: node

A producer application that writes events to Redpanda, or a consumer application that reads events from Redpanda.

This could also be a client library, like librdkafka or franz-go.

One or more brokers that work together to manage real-time data streaming, processing, and storage.

A set of consumers that cooperate to read data for better scalability. As group members arrive and leave, partitions are re-assigned so each member receives a proportional share.

The position of a consumer in a specific topic partition, to track which records they have read. A consumer offset of 3 means it has read messages 0-2 and will next read message 3.

A client application that subscribes to Redpanda topics to asynchronously read events.

A broker that manages operational metadata for a Redpanda cluster and ensures replicas are distributed among brokers. At any given time, one active controller exists in a cluster. If the controller fails, another broker is automatically elected as the controller.

Containing all your data in your environment.

With BYOC, Redpanda handles provisioning, monitoring, and upgrades, but you manage your streaming data without Redpanda’s control plane ever seeing it. Additionally, with a customer-managed VPC, the Redpanda Cloud agent doesn’t create any new resources or alter any settings in your account.

A continuous flow of events in real time that are produced and consumed by client applications. Redpanda is a data streaming platform. Also known as event stream.

A record of something changing state at a specific time. Events can be generated by various sources, including sensors, applications, and devices. Producers write events to Redpanda, and consumers read events from Redpanda.

Producers and consumers interact with Redpanda using the Kafka API. It uses the default port 9092.

A broker that is a follower in a Raft group but is not part of quorum.

In a Raft group, a broker can be in learner status. Learners are followers that cannot vote and so do not count towards quorum (the majority). They cannot be elected to leader nor can they trigger leader elections. Brokers can be promoted or demoted between learner and voter. New Raft group members start as learners.

For more information, see Raft Group Reconfiguration.

Configuration on a broker that defines how it should accept client or inter-broker connections. Each listener is associated with a specific protocol, hostname, and port combination. The listener defines where the broker should listen for incoming connections.

For more information, see Configure Listeners.

One or more records representing individual events being transmitted. Redpanda transfers messages between producers and consumers.

Sometimes used interchangeably with record.

A machine, which could be a server, a virtual machine (instance), or a Docker container. Every node has its own disk. Partitions are stored locally on nodes. In Kubernetes, a Node is the machine that Redpanda runs on. Outside the context of Kubernetes, this term may be used interchangeably with broker, such as node_id.

See also: broker

An acknowledgement that the event has been read.

A unique integer assigned to each record to show its location in the partition.

Original name for the subsystem of Redpanda that allows access to your data through a REST API. This name still appears in the HTTP Proxy API and the Schema Registry API.

Every Redpanda partition forms a Raft group with a single elected leader. This leader handles all writes, and it replicates data to followers to ensure that a majority of brokers store the data.

A subset of events in a topic, like a log file. It is an ordered, immutable sequence of records. Partitions allow you to distribute a stream, which lets producers write messages in parallel and consumers read messages in parallel. Partitions are made up of segment files on disk.

A client application that writes events to Redpanda. Redpanda stores these events in sequence and organizes them into topics.

For more information, see Configure Producers.

A failure zone that has one or more Redpanda brokers assigned to it.

The consensus algorithm Redpanda uses to coordinate writing data to log files and replicating that data across brokers.

For more details, see https://raft.github.io/

A self-contained data entity with a defined structure, representing a single event.

Sometimes used interchangeably with message.

Copies of partitions that are distributed across different brokers, so if one broker goes down, there is a copy of the data.

The mechanism for determining how long Redpanda stores data on local disk or in object storage before purging it.

For more information, see Manage Disk Space.

The number of copies of partitions in a cluster. With a replication factor greater than 1, you ensure that each partition has a copy of its data on at least one other broker. One replica acts as the leader, and the other replicas are followers.

By default, replication.factor is set to 1 (no replication) in Redpanda Self-Managed deployments and set to 3 in Redpanda Cloud deployments.

An external mechanism to describe the structure of data and its encoding. Schemas validate the structure and ensure that producers and consumers can connect with data in the same format.

An open-source thread-per-core C++ framework, which binds all work to physical cores. Redpanda is built on Seastar.

For more details, see https://seastar.io/

The initial set of brokers that a Redpanda broker contacts to join the cluster. Seed servers play a crucial role in cluster formation and recovery, acting as a point of reference for new or restarting brokers to understand the current topology of the cluster.

Discrete part of a partition, used to break down a continuous stream into manageable chunks. You can set the maximum duration (segment.ms) or size (segment.bytes) for a segment to be open for writes.

The process of converting a record into a format that can be stored. Deserialization is the process of converting a record back to the original state. Redpanda Schema Registry supports Avro and Protobuf serialization formats.

A CPU core.

A logical grouping or category for schemas. When data formats are updated, a new version of the schema can be registered under the same subject, allowing for backward and forward compatibility.

Programming model that allows Redpanda to pin each of its application threads to a CPU core to avoid context switching and blocking.

A logical stream of related events that are written to the same log. It can be divided into multiple partitions. A topic can have various clients writing events to it and reading events from it.

A REST API used to manage and monitor Redpanda clusters. It uses the default port 9644.

For more information, see Admin API.

Note: The Redpanda Admin API is different from the Kafka Admin API.

Feature that retains the latest value for each key within a partition while discarding older values.

For more information, see Compaction Settings.

Snapshot of the current cluster metadata state saved to disk, so broker startup is fast.

Framework to manipulate or enrich data written to Redpanda topics. You can develop custom data functions, which run asynchronously using a WebAssembly (Wasm) engine inside a Redpanda broker.

For more information, see Data Transforms.

Redpanda HTTP Proxy (pandaproxy) allows access to your data through a REST API. It is built into the Redpanda binary and uses the default port 8082.

A state where a Redpanda broker temporarily doesn’t take any partition leaderships. It continues to store data as a follower. This is usually done for system maintenance or a rolling upgrade.

For more information, see Maintenance Mode.

Feature that lets you distribute replicas of the same partition across different racks to minimize data loss and improve fault tolerance in the event of a rack failure.

For more information, see Enable Rack Awareness.

Process of moving partition replicas and transferring partition leadership for improved performance.

Redpanda provides various topic-aware tools to balance clusters for best performance.

For more information, see Cluster Balancing.

The process of upgrading each broker in a Redpanda cluster, one at a time, to minimize disruption and ensure continuous availability.

For more information, see Upgrade Redpanda.

Redpanda’s command-line interface tool for managing Redpanda clusters.

A read-only topic that mirrors a topic on a different cluster, using data from Tiered Storage.

Redpanda Schema Registry (pandaproxy) is the interface for storing and managing event schemas. Producers and consumers register and retrieve schemas they use from the registry. It is built into the Redpanda binary and uses the default port 8081.

For more information, see Schema Registry.

Feature that lets you offload log segments to object storage in near real-time, providing long-term data retention and topic recovery.

For more information, see Use Tiered Storage.

A Kubernetes controller that simplifies the process of obtaining, renewing, and using certificates.

For more details, see https://cert-manager.io/docs/

Generates and applies all the manifest files you need for deploying Redpanda in Kubernetes.

For more information, see Redpanda in Kubernetes.

Extends Kubernetes with custom resource definitions (CRDs), which allow Redpanda clusters to be treated as native Kubernetes resources.

For more information, see Redpanda in Kubernetes.

Redpanda software that is available under the Redpanda Business Source License (BSL). These core features are free and source-available.

Redpanda software that is available under the Redpanda Community License (RCL). It includes the free features licensed with the Redpanda Community Edition, as well enterprise features, such as Tiered Storage, Remote Read Replicas, and Continuous Data Balancing.

Redpanda Self-Managed refers to the product offering that includes both the Enterprise Edition and the Community Edition of Redpanda.

Sometimes used interchangeably with self-hosted.

A security feature used to define and enforce granular permissions to resources, ensuring only authorized users or applications can perform specific operations. ACLs act on principals.

For more information, see Redpanda Authorization Mechanisms.

The address a Redpanda broker broadcasts to producers, consumers, and other brokers. It specifies the hostname and port for connections to different listeners. Clients and other brokers use advertised listeners to connect to services such as the Admin API, Kafka API, and HTTP Proxy API. The advertised address might differ from the listener address in scenarios where brokers are behind a NAT, in a Docker container, or in Kubernetes. Advertised addresses ensure clients can reach the Redpanda brokers even in complex network setups.

The process of verifying the identity of a principal, user, or service account.

For more information, see Configure Authentication.

The process of specifying access rights to resources. Access rights are enforced through roles or access-control lists (ACLs).

For more information, see Redpanda Authorization Mechanisms.

An access token used for authentication and authorization in web applications and APIs. It holds user credentials, usually in the form of random strings of characters.

For more information, see Configure Authentication.

An entity (such as a user account or a service account) that accesses resources. Principals can be authenticated and granted permissions based on roles to perform operations.

An identity independent of the user who created it that can be used to authenticate and perform operations. This is especially useful for authentication of machines.