Serverless

Serverless is the fastest and easiest way to start data streaming. With Serverless clusters, you host your data in Redpanda’s VPC, and Redpanda handles automatic scaling, provisioning, operations, and maintenance. This is a production-ready deployment option with a cluster available instantly, and you only pay for what you consume. You can view detailed billing activity for each cluster and edit payment methods on the Billing page.

Serverless usage limits

Each Serverless cluster has the following maximum usage limits:

  • Ingress: 100 MB/s

  • Egress: 300 MB/s

  • Partitions: 5,000

  • Message size: 20 MiB

  • Retention: unlimited

  • Storage: unlimited

  • Users: 30

  • ACLs: 120

  • Consumer groups: 200

  • Connections: 10,000

The partition limit is the number of logical partitions before replication occurs. Redpanda Cloud uses a replication factor of 3.

Prerequisites

Make sure you have the latest version of rpk, the Redpanda CLI. See Install or Update rpk.

Get started with Serverless

Choose the option that fits how you want to subscribe:

  • Free trial

  • Redpanda Sales

  • AWS Marketplace

A free trial on AWS is the fastest way to get started with Serverless. Each free-trial customer qualifies for $100 (USD) in credits to spend in the first 14 days. This should be enough to run Redpanda with reasonable throughput. No credit card is required. To continue using Serverless after your trial expires, you can enter a credit card and pay as you go. Any remaining credit balance is used before you are charged.

When either the credits expire or the days in the trial expire, the clusters move into a suspended state, and you won’t be able to access your data in either the Redpanda Cloud UI or with the Kafka API. There is a seven-day grace period following the end of the trial. After that, the data is permanently deleted. For questions about the trial, use the #serverless Community Slack channel.

After you start a trial, Redpanda instantly prepares an account for you. Your account includes a welcome cluster with a hello-world demo topic you can explore. It includes sample data so you can see how real-time messaging works before sending your own data.

Get started by following the wizard to create a Redpanda Connect pipeline, or by following the steps in the UI to use rpk to interact with your cluster from the command line:

  1. Log in with rpk cloud login.

  2. Consume from the hello-world topic with rpk topic consume hello-world.

  3. In the Redpanda Cloud UI, navigate to the Topics page and open the hello-world topic to see the included messages.

To request a private offer with possible discounts for annual committed use, contact Redpanda Sales. When you subscribe to Serverless through Redpanda Sales, you gain immediate access to Enterprise support.

Redpanda creates a cloud organization for you and sends you a welcome email.

New subscriptions to Redpanda Cloud through AWS Marketplace receive $300 (USD) in free credits to spend in the first 30 days. AWS Marketplace charges for anything beyond $300, unless you cancel the subscription. After your free credits have been used, you can continue using your cluster without any commitment, only paying for what you consume and canceling anytime.

When you subscribe to Redpanda through AWS Marketplace, you do not have immediate access to Enterprise support, only the Community Slack channel. For Enterprise support, contact Redpanda Sales

Redpanda creates a cloud organization for you and sends you a welcome email.

Create a Serverless cluster

To create a Serverless cluster:

  1. In the Redpanda Cloud UI, on the Clusters page, click Create cluster, then click Create Serverless cluster.

  2. Enter a cluster name, then select the resource group. If you don’t have an existing resource group, you can create one. Refresh the page to see newly-created resource groups.

  3. Select a cloud provider and region. For best performance, select the region closest to your applications. Redpanda expects your applications to be deployed in the same cloud provider and region as your Serverless cluster.

    Serverless clusters are not guaranteed to be pinned to a particular availability zone within the selected region.

  4. Add team members and grant them access with access control lists (ACLs) on the Security page.

Interact with your cluster

The cluster’s Overview page includes a Get Started wizard to quickly pipeline your data into and out of Redpanda. The wizard guides you to select an input data source (to produce data) or an output data sink (to consume data) and automatically populates the necessary permissions and configurations. See also: Redpanda Connect Quickstart

The Overview page lists your bootstrap server URL and security settings in the How to connect - Kafka API tab. Here you can add a Kafka client to interact with your cluster.

Or, Redpanda can generate a sample application to interact with your cluster. Run rpk generate app, and select Go as the language. Follow the commands in the terminal to run the application, create a demo topic, produce to the topic, and consume the data back.

Follow the steps in the UI to use rpk to interact with your cluster from the command line. Here are some helpful commands:

  • rpk cloud login: Use this to log in to Redpanda Cloud or to refresh the session.

  • rpk topic: Use this to manage topics, produce data, and consume data.

  • rpk profile print: Use this to view your rpk configuration and see the URL for your Serverless cluster.

  • rpk security user: Use this to manage users and permissions.

Redpanda Serverless is opinionated about Kafka configurations. For example, automatic topic creation is disabled. Some systems expect the Kafka service to automatically create topics when a message is produced to a topic that doesn’t exist. Create topics on the Topics page or with rpk topic create.

The How to connect - Kafka API tab lists your bootstrap server URL and security settings. This area includes code examples for creating a Kafka client to interact with your cluster. Or, Redpanda can generate a sample application to interact with your cluster. Run rpk generate app, and select your preferred programming language. Follow the commands in the terminal to run the application, create a demo topic, produce to the topic, and consume the data back.

Explore the rest of the UI:

  • Go to the Topics page to create new topics for data streams.

  • Under the Actions dropdown, produce messages to topics.

  • Add team members with permissions on the Security page.

Supported features

  • Redpanda Serverless supports the Kafka API. Serverless clusters work with all Kafka clients. See Kafka Compatibility.

  • Serverless clusters support all major Apache Kafka messages for managing topics, producing/consuming data (including transactions), managing groups, managing offsets, and managing ACLs. (User management is available in the Redpanda Cloud UI or with rpk security acl.)

Unsupported features

Not all features included in BYOC clusters are available in Serverless. For example, the following features are not supported:

  • HTTP Proxy API

  • Ability to export metrics to a third-party monitoring system

  • Private networking (VPC peering or AWS PrivateLink)

  • Multiple availability zones (AZs)

  • RBAC in the data plane and mTLS authentication for Kafka API clients

  • Kafka Connect

Next steps