Serverless

Serverless is the fastest and easiest way to start data streaming. With Serverless clusters, you host your data in Redpanda’s VPC, and Redpanda handles automatic scaling, provisioning, operations, and maintenance. This is a production-ready deployment option with a cluster available instantly, and with pay-as-you-go billing, you only pay for what you consume. You can view detailed billing activity for each cluster and edit payment methods on the Billing page.

Serverless is currently in a limited availability (LA) release. During LA, existing clusters can scale to the usage limits, but new clusters may need to wait for availability.

Prerequisites

Make sure you have the latest version of rpk. See Install or Update rpk.

Serverless usage limits

Each Serverless cluster has the following limits:

  • Ingress: 100 MiB/s

  • Egress: 300 MiB/s

  • Partitions: 5000

  • Message size: 20 MB

  • Retention: unlimited

  • Storage: unlimited

  • Users: 30

  • ACLs: 120

  • Consumer groups: 200

The partition limit is the number of logical partitions before replication occurs. Redpanda Cloud uses a replication factor of 3.

Get started with Serverless

  • Free trial

  • Redpanda Sales

  • AWS Marketplace

A trial is the fastest way to get started with Serverless, and it comes with a welcome cluster. New trials receive $100 (USD) in free credits to spend in the first 14 days. This should be enough to run Redpanda with reasonable throughput. No credit card is required. To continue using Serverless after your trial expires, you can enter a credit card and pay as you go. Any remaining credit balance is used before you are charged.

When either the 100 credits expire or the 14 days in the trial expire, the clusters move into a suspended state, and you won’t be able to access your data in either the Redpanda Cloud UI or with the Kafka API. There is a seven-day grace period following the end of the trial. After that, the data is permanently deleted.

  • To ask questions about the trial, use the #serverless Community Slack channel.

  • For information about billing after the trial ends, see Usage Metrics.

To request a private offer with possible discounts, contact Redpanda Sales. When you subscribe to Serverless through Redpanda Sales, you get immediate access to Enterprise support.

Redpanda creates a cloud organization for you and sends you a welcome email.

To start using Serverless, subscribe to Redpanda Cloud through AWS Marketplace. New subscriptions receive $300 (USD) in free credits to spend in the first 30 days. AWS Marketplace charges for anything beyond $300, unless you cancel the subscription. After your free credits have been used, you can continue using your cluster without any commitment, only paying for what you consume and canceling anytime.

When you subscribe to Redpanda through AWS Marketplace, you do not have immediate access to Enterprise support, only the Community Slack channel. To access Enterprise support, contact Redpanda Sales

Redpanda creates a cloud organization for you and sends you a welcome email.

Explore your trial cluster

After you click to start a trial, Redpanda instantly prepares an account for you. Your account includes a welcome cluster with a hello-world demo topic you can explore.

Follow the steps in the UI to use rpk to interact with your cluster from the command line:

  1. Log in:

    rpk cloud login

  2. Consume from the hello-world topic:

    rpk topic consume hello-world

In the Redpanda Cloud UI, you can navigate to the Topics page and open the hello-world topic to see the included messages. Under the Actions dropdown, you can produce messages to it. Add team members and grant them access with ACLs on the Security page.

Create a Serverless cluster

To create a Serverless cluster:

  1. In the Redpanda Cloud UI, on the Clusters page, click Create cluster, then click Create Serverless cluster.

  2. Enter a cluster name, then select the resource group. If you don’t have an existing resource group, you can create one. Refresh the page to see newly-created resource groups.

    Serverless clusters are currently available in certain AWS regions. Redpanda expects your applications to be deployed in the same AWS region. For best performance, select the region closest to your applications. Serverless is not guaranteed to be pinned to a particular availability zone within that region.

  3. Go to the Topics page to create a topic. Under the Actions dropdown, you can produce messages to it. Add team members and grant them access with ACLs on the Security page.

Connect with your cluster

Create a Kafka client to interact with your cluster. The Overview page lists your bootstrap server URL and security settings in the How to connect - Kafka API tab.

Or, Redpanda can generate a sample application to interact with your cluster. Run rpk generate app, and select your preferred programming language. Follow the commands in the terminal to run the application, create a demo topic, produce to the topic, and consume the data back.

Follow the steps in the UI to use rpk to interact with your cluster from the command line. Here are some helpful commands:

  • rpk cloud login: Use this to log in to Redpanda Cloud or to refresh the session.

  • rpk topic: Use this to manage topics, produce data, and consume data.

  • rpk profile print: Use this to view your rpk configuration and see the URL for your Serverless cluster.

  • rpk security user: Use this to manage users and permissions.

Redpanda Serverless is opinionated about Kafka configurations. For example, automatic topic creation is disabled. Some systems expect the Kafka service to automatically create topics when a message is produced to a topic that doesn’t exist. Create topics on the Topics page or with rpk topic create.

Supported features

  • Redpanda Serverless supports the Kafka API.

  • Serverless clusters work with all Kafka clients. For more information, see Kafka Compatibility.

  • Serverless clusters support all major Apache Kafka messages for managing topics, producing/consuming data (including transactions), managing groups, managing offsets, and managing ACLs. (User management is available in the Redpanda Cloud UI or with rpk security acl.)

  • Redpanda Connect is integrated with Serverless as a beta feature for testing and feedback. Choose from a range of connectors, processors, and other components to quickly build and deploy streaming data pipelines or AI applications.

Unsupported features

  • Data transforms

  • Redpanda Admin API

  • HTTP Proxy API

  • Kafka Connect

Next steps