Redpanda Cloud Overview

Redpanda Cloud is a complete data streaming platform delivered as a fully-managed service with automated upgrades and patching, data and partition balancing, and 24x7 support. It continuously monitors and maintains your clusters along with the underlying infrastructure to meet strict performance, availability, reliability, and security requirements. All Redpanda Cloud clusters are deployed with an integrated Redpanda Console.

Cluster types

Redpanda offers three types of fully-managed cloud clusters:

  • Serverless: Clusters hosted in Redpanda Cloud. This is the fastest and easiest way to start data streaming.

  • Dedicated Cloud: Single-tenant clusters hosted in Redpanda Cloud. This provides more control over your deployment.

  • Bring Your Own Cloud (BYOC): Clusters hosted in your private cloud. This provides more control over your deployment and offers full data sovereignty.

    With standard BYOC clusters, Redpanda manages security policies and resources for your VPC, including service accounts, subnetworks, IAM roles, firewall rules, and storage buckets. For the most security, you can manage these resources yourself with a customer-managed VPC.

Serverless

With Serverless clusters, you host your data in Redpanda’s VPC, and Redpanda handles automatic scaling, provisioning, operations, and maintenance. This is a production-ready deployment option with a cluster available instantly. This is no base cost, and with pay-as-you-go billing after the free trial, you only pay for what you consume.

Available on AWS.

Sign up for Serverless

To start using Serverless, sign up for a free trial. New trials receive $100 (USD) in free credits to spend in the first 14 days. This should be enough to run Redpanda with reasonable throughput. No credit card is required for a trial. To continue using Serverless, add a credit card and pay as you go. You can edit payment methods on the Redpanda Cloud Billing page.

Serverless is currently in a limited availability (LA) release with usage limits and no public SLA. During LA, existing clusters can scale to the usage limits, but new clusters may need to wait for availability.

Dedicated Cloud

With Dedicated clusters, you host your data in Redpanda’s VPC, and Redpanda handles provisioning, operations, and maintenance. Dedicated clusters are single-tenant deployments that support private networking (that is, VPC peering to talk over private IPs) for better data isolation.

Available on AWS or GCP.

Sign up for Dedicated

To start using Dedicated, contact Redpanda sales to request a private offer for monthly or annual committed use. With a usage-based billing commitment, you sign up for a monthly or an annual minimum spend amount. You can then use Google Cloud Marketplace or AWS Marketplace to quickly provision Dedicated Cloud clusters, and you can view invoices and manage your subscription in the marketplace.

Bring Your Own Cloud (BYOC)

With BYOC clusters, you deploy Redpanda in your own VPC, so all data is contained in your own environment. There are no inbound connections from the Redpanda control plane to the data plane. This provides an additional layer of security and isolation. Redpanda handles provisioning, operations, and maintenance.

Setting up a BYOC cluster is initiated from the Redpanda control plane. This is a two-step process performed using rpk:

  1. You bootstrap a virtual machine (VM) in your VPC. This VM spins up the agent and the required infrastructure.

  2. The agent communicates with the control plane to pull the cluster specifications.

Redpanda assigns the necessary IAM policies required to run the agent at the time of deployment.

Available on AWS or GCP.

Sign up for BYOC

To start using BYOC, contact Redpanda sales to request a private offer, and you can be billed either directly or through Google Cloud Marketplace or AWS Marketplace.

Serverless vs Dedicated/BYOC

Serverless clusters are a good fit for the following:

  • Starter and growing workloads

  • Spiky workloads (that is, development environments, systems that only occasionally get busy, or workloads that come and go)

  • Fast and dynamic cluster creation: you can use a Serverless cluster as an isolated container for topics

With Serverless, you only pay for what you consume, without any commitment. A cluster is created instantly, so you can surface it in your applications (for example, for tenant isolation). If your workload increases, you can migrate it to a Dedicated or BYOC cluster.

Dedicated and BYOC clusters offer the following features:

  • Single-zone or multi-zone availability (a multi-zone cluster provides higher resiliency in the event of a failure in one of the zones)

  • Private networking using VPC peering

  • Ability to export metrics to a 3rd-party monitoring system

  • Managed connectors

  • Higher limits and quotas (see Serverless limits)

Consider Dedicated or BYOC if you need any of the following:

  • Control over the deployment

  • Private networking

  • Workloads with consistently-high throughput

Cluster tiers

When you create a Dedicated or BYOC cluster, you select a throughput tier. The following table lists the current cluster-level AWS and GCP tier limits. Legacy tiers may have different limits. For more information, contact support.

Tier Maximum ingress Maximum egress Maximum logical partitions Maximum connections

Tier 1

20 MBps

60 MBps

1,000

1,500

Tier 2

50 MBps

150 MBps

2,800

3,750

Tier 3

100 MBps

200 MBps

5,600

7,500

Tier 4

200 MBps

400 MBps

11,300

15,000

Tier 5

400 MBps

800 MBps

22,800

30,000

Tier 6 (BYOC only)

800 MBps

1,600 MBps

45,000

60,000

Tier 7 (BYOC only)

1,200 MBps

2,400 MBps

50,000

90,000

Serverless clusters are not on tiers. See Serverless limits. To change your cluster type, contact support.

Shared responsibility model

The Redpanda Cloud shared responsibility model lists the security ownership areas for Redpanda and customers. Responsibilities depend on the type of deployment.

  • Dedicated

  • BYOC

  • BYOC with customer-managed VPC

Resource Redpanda responsibility Customer responsibility

Redpanda upgrades and hotfixes

Cost management and attribution

Software vulnerability remediation

Infrastructure vulnerability remediation

IAM (roles, service accounts, access segmentation)

Compute

VPC (subnets, routing, firewall)

VPC peering

VPC private links (service endpoint)

VPC private links (consumer endpoint)

Local storage

Tiered Storage

Control plane

Access controls and audit

Managed disaster recovery

Observability and monitoring (SLOs, SLIs, tracing, alerting, runbooks)

Availability SLA

Proactive threat detection

Static secret rotation

Incident response

Resilience verification

Resource Redpanda responsibility Customer responsibility

Redpanda upgrades and hotfixes

Cost management and attribution

Software vulnerability remediation

Infrastructure vulnerability remediation

IAM (roles, service accounts, access segmentation)

Compute

VPC (subnets, routing, firewall)

VPC peering

VPC private links (service endpoint)

VPC private links (consumer endpoint)

Local storage

Tiered Storage

Control plane

Access controls and audit

Managed disaster recovery

Observability and monitoring (SLOs, SLIs, tracing, alerting, runbooks)

Availability SLA

✓ (subject to required access to customer resources)

Proactive threat detection

Static secret rotation

Incident response

Resilience verification

Resource Redpanda responsibility Customer responsibility

Redpanda upgrades and hotfixes

Cost management and attribution

Software vulnerability remediation

Infrastructure vulnerability remediation

IAM (roles, service accounts, access segmentation)

Compute

VPC (subnets, routing, firewall)

VPC peering

VPC private links (service endpoint)

VPC private links (consumer endpoint)

Local storage

Tiered Storage

Control plane

Access controls and audit

Managed disaster recovery

Observability and monitoring (SLOs, SLIs, tracing, alerting, runbooks)

Availability SLA

✓ (subject to required access to customer resources)

Proactive threat detection

Static secret rotation

Incident response

Resilience verification

Cloud architecture

Redpanda Cloud uses a control plane and data plane architecture. The control plane configures and maintains cluster agents, which are self-contained and independent of the control plane.

Control plane

The control plane is a separate entity that enforces rules in the data plane. The control plane is where a good deal of your Redpanda cluster management, operations, and maintenance takes place. While Redpanda Cloud clusters are configured and maintained in the control plane, they are self-contained and remain available even if the network connection to the control plane is lost.

Data plane

The data plane is where your Redpanda cluster resides in Redpanda Cloud; it is self-contained and managed by an agent that receives cluster specifications from the control plane. Redpanda Cloud clusters are interchangeably referred to as data planes. All network connections into the data plane take place through either a public endpoint or a Redpanda Cloud VPC peering network connection.

Agent

The Redpanda Cloud agent, also known as the data plane agent, provisions, configures, and maintains all the resources in the cluster, from the network all the way up to the cluster and connectors infrastructure. Data plane resources include cloud infrastructure, Kubernetes, and data plane software artifacts.

Agents can be bootstrapped in two different ways: automatically by a machine or manually by a human. The human-driven agent bootstrap is designed for BYOC deployments. After the agent is up and running, it connects to the control plane and starts dequeuing and applying cluster specifications that provision, configure, and maintain Redpanda Cloud clusters.

The agent is in constant communication with the control plane, receiving and applying cluster specifications and exchanging cluster metadata. Agents are authenticated and authorized through opaque and ephemeral tokens, and have dedicated job queues in the control plane. In BYOC, you can provision, configure, and maintain agents using rpk.

Redpanda Cloud agents also manage VPC peering networks. However, only one agent manages each network resource at a time.

The following diagram shows the resources, operations, and connections in the control plane and data plane.

Redpanda Cloud Security Architecture

Redpanda Cloud vs self-hosted feature compatibility

Redpanda Cloud does not support the following self-hosted functionality:

  • mTLS

  • OpenID Connect (OIDC) authentication

  • Kerberos authentication

  • Data transforms

  • Remote Read Replicas

  • Redpanda Console topic documentation

  • Setting auto_create_topics_enabled=true for BYOC and dedicated clusters

  • Admin API

  • The following rpk commands (which use the Admin API):

    • rpk acl user

    • rpk cluster health

    • rpk cluster config

    • rpk cluster license

    • rpk cluster maintenance

    • rpk cluster partitions

    • rpk cluster self-test

    • rpk cluster storage

    • rpk generate app

    • rpk redpanda

    • rpk topic describe-storage (all other rpk topic commands are supported on both Redpanda Cloud and self-hosted)