Create a BYOVNet Cluster on Azure
|
BYOVPC/BYOVNet is an add-on feature that requires Premium support. To unlock this feature for your account, contact your Redpanda account team or Redpanda Sales. |
A Bring Your Own Virtual Network (BYOVNet) cluster allows you to deploy the Redpanda data plane into your existing VNet and manage the networking lifecycle. Compared to a standard Bring Your Own Cloud (BYOC) setup, where Redpanda manages the networking lifecycle for you, BYOVNet provides more control. To compare the two variants before you commit to one, see Choose Between BYOC and BYOVPC. For background on the architecture, see BYOC architecture.
When you create a BYOVNet cluster, you specify your VNet and managed identities. You create the networking and security resources in advance, so the Redpanda Cloud agent doesn’t create or change them. The agent still deploys and operates the Redpanda cluster infrastructure, using the managed identities you supply and only the permissions you grant them. With a customer-managed VNet:
-
You provide your own VNet in your Azure subscription.
-
You maintain more control over your account, because Redpanda requires fewer permissions than standard BYOC clusters.
-
You control your security resources and policies, including subnets, user-assigned identities, IAM roles and assignments, security groups, storage accounts, and key vaults.
The Redpanda BYOVNet Terraform Module contains Terraform code that deploys the resources required for a BYOVNet cluster on Azure. The Redpanda Terraform provider then creates the Redpanda network and cluster from the module outputs, and runs the Redpanda agent bootstrap for you. You can do the whole deployment in a single Terraform configuration, with no Cloud API calls or rpk commands. Variables are provided in the module so you can exclude resources that already exist in your environment, such as the VNet.
The module creates the following groups of customer-managed resources:
-
Resource groups
-
User-assigned identities
-
IAM roles and assignments
-
Network: VNet, subnets, NAT gateway, and network security group
-
Storage: management and Tiered Storage accounts and containers
-
Key vaults
Prerequisites
-
Access to an Azure subscription in which you create your cluster.
-
Permission in that subscription to create, modify, and delete the resources that the module creates, including custom role definitions and role assignments. For example, the Owner role, or the Contributor and User Access Administrator roles.
-
The Azure CLI, signed in to the subscription with
az login. Terraform and the Redpanda agent bootstrap use your Azure CLI credentials by default. To use a service principal instead, set theARM_CLIENT_ID,ARM_CLIENT_SECRET,ARM_TENANT_ID, andARM_SUBSCRIPTION_IDenvironment variables. With either sign-in method, the Redpanda provider still needs the subscription ID, so the example configuration setsazure_subscription_idon the provider. -
Enough vCPU quota in the region for the cluster. Check both the per-family quotas for the VM sizes that your throughput tier uses and the region’s Total Regional vCPUs quota. The Redpanda agent checks only the per-family quotas before it creates the cluster. Leave headroom: Redpanda Connect adds nodes when pipelines start. For example, a tier 1 cluster uses about 14 vCPUs before any pipeline runs.
-
Terraform version 1.8.5 or later.
-
The Redpanda Terraform provider version 2.5.0 or later, configured with valid credentials. For example, set the
REDPANDA_CLIENT_IDandREDPANDA_CLIENT_SECRETenvironment variables. For setup details, see Redpanda Terraform Provider.
Limitations
-
Existing clusters cannot be converted to BYOVNet clusters.
-
After creating a BYOVNet cluster, you cannot change to a different VNet.
-
Only primary CIDR ranges are supported for the VNet.
-
Every attribute under
customer_managed_resources.azureis immutable. Changing one plans a replacement of the network or cluster. See Configuration changes that replace a cluster.
|
For simplicity, these instructions assume that Terraform uses local state. You might want to configure remote state. |
Configure the Terraform providers
Create a directory for the Terraform configuration. In that directory, create a versions.tf file that declares the Redpanda and Azure providers:
terraform {
required_providers {
redpanda = {
source = "redpanda-data/redpanda"
version = ">= 2.5.0"
}
azurerm = {
source = "hashicorp/azurerm"
version = "~> 5.5"
}
}
}
Version 2 of the Redpanda BYOVNet module requires azurerm 5.5 or a later 5.x release.
|
Then create a main.tf file and configure both providers:
provider "azurerm" {
features {}
subscription_id = local.azure_subscription_id
# The module disables shared-key access on its storage accounts.
# Without this setting, the provider's post-create data-plane check
# fails with KeyBasedAuthenticationNotPermitted.
storage_use_azuread = true
}
# Reads REDPANDA_CLIENT_ID and REDPANDA_CLIENT_SECRET from the environment.
provider "redpanda" {
# The Redpanda provider does not read the subscription from the Azure
# CLI. It needs the subscription ID to run the agent bootstrap.
azure_subscription_id = local.azure_subscription_id
}
Configure the Redpanda BYOVNet Terraform module
Add the following to main.tf. This example uses the Redpanda BYOVNet Terraform module to create a new VNet, the subnets, and all the other resources required for a BYOVNet cluster.
locals {
azure_subscription_id = "<azure-subscription-id>"
azure_tenant_id = "<azure-tenant-id>"
# Use a region and zones from BYOC supported regions (Azure tab).
region = "eastus"
zones = ["eastus-az1", "eastus-az2", "eastus-az3"]
# Prefix added to every Azure resource name that the module creates.
# Storage account names use it with dashes removed, and must fit
# the 24-character storage account limit, so keep it short.
resource_name_prefix = "rp-"
# Prefix added to every resource group name that the module creates.
resource_group_name_prefix = "rp-"
# The module reserves this CIDR for AKS services inside the VNet, and
# the cluster declares the same value, so one local feeds both.
aks_service_cidr = "10.0.15.0/24"
tags = {
"environment" = "dev"
"managed-by" = "terraform"
}
}
module "redpanda_byovnet" {
source = "redpanda-data/redpanda-byovnet/azure"
version = "~> 2.0"
region = local.region
zones = local.zones
azure_tenant_id = local.azure_tenant_id
azure_subscription_id = local.azure_subscription_id
resource_name_prefix = local.resource_name_prefix
resource_group_name_prefix = local.resource_group_name_prefix
reserved_subnet_cidrs = { "k8s-service" = local.aks_service_cidr }
# The Redpanda agent bootstrap keeps its Terraform state in the
# management storage account, which allows only Microsoft Entra ID
# authentication. This grants the identity that runs Terraform the
# Storage Blob Data Contributor role on that storage account.
grant_caller_management_storage_access = true
tags = local.tags
}
By default, the module:
-
Creates four resource groups (
redpanda-rg,storage-rg,network-rg, andiam-rg), each prefixed withresource_group_name_prefix. To place resources in resource groups that already exist, setcreate_resource_groups = falseand provide the names inredpanda_resource_group_name,redpanda_storage_resource_group_name,redpanda_network_resource_group_name, andredpanda_iam_resource_group_name. -
Creates a VNet with the address space
10.0.0.0/20(vnet_addresses). -
Creates 13 private subnets (
private_subnets) and one egress subnet (egress_subnets), each a/24in the range10.0.0.0/24through10.0.13.0/24. Subnets must be at least/24. -
Reserves
10.0.15.0/24for the AKS service CIDR (reserved_subnet_cidrs). This range must not overlap any subnet. -
Creates a NAT gateway for outbound traffic from the private subnets. To provide egress another way, set
create_nat = false.
|
| For the Azure regions and zones that BYOC supports, see the Azure tab in BYOC supported regions. For throughput tiers, see BYOC Tiers and Regions. |
A BYOVNet deployment made with version 1.2 of the module moves to version 2 with no changes to its Azure resources. Set the module version to ~> 2.0 and azurerm to ~> 5.5, then run terraform init -upgrade. terraform plan reports no changes. If the deployment uses an earlier 1.x release, review the plan before you apply it.
|
Optional: Deploy with a pre-existing VNet
Skip this section to let the module create the VNet. If you already have a VNet in your Azure subscription, provide its name to the module instead of letting the module create one. The module still creates the subnets inside your VNet, so the CIDRs in private_subnets, egress_subnets, and reserved_subnet_cidrs must fall within the VNet’s address space and not overlap existing subnets.
module "redpanda_byovnet" {
# ...
vnet_name = "existing-vnet-name"
# Use resource groups that already exist. The module adds
# resource_group_name_prefix to these names, so set it to ""
# if you provide the full names.
create_resource_groups = false
resource_group_name_prefix = ""
redpanda_resource_group_name = "existing-redpanda-rg"
redpanda_storage_resource_group_name = "existing-storage-rg"
redpanda_network_resource_group_name = "existing-network-rg" # resource group that contains the VNet
redpanda_iam_resource_group_name = "existing-iam-rg"
private_subnets = {
"system-pod" = { "cidr" = "10.1.1.0/24", "name" = "snet-system-pods" }
"system-vnet" = { "cidr" = "10.1.2.0/24", "name" = "snet-system-vnet" }
"agent-private" = { "cidr" = "10.1.3.0/24", "name" = "snet-agent-private" }
"rp-0-pods" = { "cidr" = "10.1.4.0/24", "name" = "snet-rp-0-pods" }
"rp-0-vnet" = { "cidr" = "10.1.5.0/24", "name" = "snet-rp-0-vnet" }
"rp-1-pods" = { "cidr" = "10.1.6.0/24", "name" = "snet-rp-1-pods" }
"rp-1-vnet" = { "cidr" = "10.1.7.0/24", "name" = "snet-rp-1-vnet" }
"rp-2-pods" = { "cidr" = "10.1.8.0/24", "name" = "snet-rp-2-pods" }
"rp-2-vnet" = { "cidr" = "10.1.9.0/24", "name" = "snet-rp-2-vnet" }
"connect-pod" = { "cidr" = "10.1.10.0/24", "name" = "snet-connect-pods" }
"connect-vnet" = { "cidr" = "10.1.11.0/24", "name" = "snet-connect-vnet" }
"kafka-connect-pod" = { "cidr" = "10.1.12.0/24", "name" = "snet-kafka-connect-pods" }
"kafka-connect-vnet" = { "cidr" = "10.1.13.0/24", "name" = "snet-kafka-connect-vnet" }
}
egress_subnets = {
"agent-public" = { "cidr" = "10.1.0.0/24", "name" = "snet-agent-public" }
}
# Set local.aks_service_cidr to a range in the VNet that no subnet uses,
# for example "10.1.15.0/24".
reserved_subnet_cidrs = { "k8s-service" = local.aks_service_cidr }
# ...
}
Keep the subnet map keys as shown. The module outputs that the Redpanda network configuration in the next section reads are keyed on them.
Configure the Redpanda network and cluster
Add the Redpanda resource group, network, and cluster to main.tf. These resources read the names of the Azure resources from the module outputs.
locals {
# A Redpanda Cloud resource group, not an Azure resource group.
redpanda_cloud_resource_group_name = "byovnet"
throughput_tier = "tier-1-azure-v3-x86"
}
resource "redpanda_resource_group" "byovnet" {
name = local.redpanda_cloud_resource_group_name
}
resource "redpanda_network" "network" {
name = "${local.resource_name_prefix}network"
resource_group_id = redpanda_resource_group.byovnet.id
cloud_provider = "azure"
region = local.region
cluster_type = "byoc"
customer_managed_resources = {
azure = {
management_bucket = {
storage_account_name = module.redpanda_byovnet.management_bucket_storage_account_name
storage_container_name = module.redpanda_byovnet.management_bucket_storage_container_name
resource_group = { name = module.redpanda_byovnet.redpanda_resource_group_name }
}
vnet = {
name = module.redpanda_byovnet.vnet_name
resource_group = { name = module.redpanda_byovnet.network_resource_group_name }
}
subnets = {
rp_agent = { name = module.redpanda_byovnet.agent_private_subnet_name }
rp_0_pods = { name = module.redpanda_byovnet.rp_0_pods_subnet_name }
rp_0_vnet = { name = module.redpanda_byovnet.rp_0_vnet_subnet_name }
rp_1_pods = { name = module.redpanda_byovnet.rp_1_pods_subnet_name }
rp_1_vnet = { name = module.redpanda_byovnet.rp_1_vnet_subnet_name }
rp_2_pods = { name = module.redpanda_byovnet.rp_2_pods_subnet_name }
rp_2_vnet = { name = module.redpanda_byovnet.rp_2_vnet_subnet_name }
rp_connect_pods = { name = module.redpanda_byovnet.rp_connect_pods_subnet_name }
rp_connect_vnet = { name = module.redpanda_byovnet.rp_connect_vnet_subnet_name }
kafka_connect_pods = { name = module.redpanda_byovnet.kafka_connect_pods_subnet_name }
kafka_connect_vnet = { name = module.redpanda_byovnet.kafka_connect_vnet_subnet_name }
sys_pods = { name = module.redpanda_byovnet.system_pods_subnet_name }
sys_vnet = { name = module.redpanda_byovnet.system_vnet_subnet_name }
rp_egress_vnet = { name = module.redpanda_byovnet.egress_subnet_name }
}
}
}
depends_on = [module.redpanda_byovnet]
}
resource "redpanda_cluster" "cluster" {
name = "${local.resource_name_prefix}cluster"
resource_group_id = redpanda_resource_group.byovnet.id
network_id = redpanda_network.network.id
cloud_provider = "azure"
region = redpanda_network.network.region
zones = local.zones
cluster_type = "byoc"
connection_type = "private"
throughput_tier = local.throughput_tier
allow_deletion = false
tags = local.tags
customer_managed_resources = {
azure = {
resource_groups = {
redpanda_resource_group = { name = module.redpanda_byovnet.redpanda_resource_group_name }
storage_resource_group = { name = module.redpanda_byovnet.storage_resource_group_name }
iam_resource_group = { name = module.redpanda_byovnet.iam_resource_group_name }
}
user_assigned_identities = {
agent_user_assigned_identity = { name = module.redpanda_byovnet.agent_user_assigned_identity_name }
aks_user_assigned_identity = { name = module.redpanda_byovnet.aks_user_assigned_identity_name }
redpanda_cluster_assigned_identity = { name = module.redpanda_byovnet.cluster_user_assigned_identity_name }
cert_manager_assigned_identity = { name = module.redpanda_byovnet.cert_manager_user_assigned_identity_name }
external_dns_assigned_identity = { name = module.redpanda_byovnet.external_dns_user_assigned_identity_name }
redpanda_console_assigned_identity = { name = module.redpanda_byovnet.console_user_assigned_identity_name }
kafka_connect_assigned_identity = { name = module.redpanda_byovnet.kafka_connect_user_assigned_identity_name }
redpanda_connect_assigned_identity = { name = module.redpanda_byovnet.redpanda_connect_user_assigned_identity_name }
redpanda_connect_api_assigned_identity = { name = module.redpanda_byovnet.redpanda_connect_api_user_assigned_identity_name }
redpanda_operator_assigned_identity = { name = module.redpanda_byovnet.redpanda_operator_user_assigned_identity_name }
}
tiered_cloud_storage = {
storage_account_name = module.redpanda_byovnet.tiered_storage_account_name
storage_container_name = module.redpanda_byovnet.tiered_storage_container_name
resource_group = { name = module.redpanda_byovnet.storage_resource_group_name }
}
key_vaults = {
management_vault = { name = module.redpanda_byovnet.management_key_vault_name }
console_vault = { name = module.redpanda_byovnet.console_key_vault_name }
}
security_groups = {
redpanda_security_group = { name = module.redpanda_byovnet.redpanda_security_group_name }
}
cidrs = {
aks_service_cidr = local.aks_service_cidr
}
}
}
# Azure cluster creation and deletion can outlast the default timeouts.
timeouts = {
create = "150m"
delete = "150m"
}
depends_on = [redpanda_network.network]
}
To use a Redpanda resource group that already exists, replace the redpanda_resource_group resource with a redpanda_resource_group data source, and reference data.redpanda_resource_group.byovnet.id in the network and cluster.
To list the names of the Azure resources that the module creates, add an outputs.tf file. The filter keeps the module’s name outputs and leaves out its detailed JSON outputs:
output "byovnet" {
description = "Names of the Azure resources that the Redpanda BYOVNet module created"
value = { for k, v in module.redpanda_byovnet : k => v if endswith(k, "_name") }
}
Apply the Terraform configuration
Initialize, plan, and apply Terraform to create the Azure infrastructure, the Redpanda network, and the Redpanda cluster:
terraform init &&
terraform plan &&
terraform apply
Terraform creates the resources in order: first the Azure resources in the module, then the Redpanda network, and then the Redpanda cluster. When it creates the cluster, the Redpanda Terraform provider runs the Redpanda agent bootstrap in your subscription, using your Azure credentials. You don’t need to run rpk cloud byoc azure apply yourself.
Cluster provisioning can take about an hour. When provisioning completes, the cluster status updates to Running. If the cluster stays in Creating status, contact Redpanda Support.
To see the names of the Azure resources that the module created, run terraform output byovnet:
Show example output
{
"agent_private_subnet_name" = "rp-snet-agent-private"
"agent_user_assigned_identity_name" = "rp-agent-uai"
"aks_user_assigned_identity_name" = "rp-aks-uai"
"cert_manager_user_assigned_identity_name" = "rp-cert-manager-uai"
"cluster_user_assigned_identity_name" = "rp-cluster-uai"
"console_key_vault_name" = "rp-console-vault"
"console_user_assigned_identity_name" = "rp-console-uai"
"egress_subnet_name" = "rp-snet-agent-public"
"external_dns_user_assigned_identity_name" = "rp-external-dns-uai"
"iam_resource_group_name" = "rp-iam-rg"
"kafka_connect_pods_subnet_name" = "rp-snet-kafka-connect-pods"
"kafka_connect_user_assigned_identity_name" = "rp-kafka-connect-uai"
"kafka_connect_vnet_subnet_name" = "rp-snet-kafka-connect-vnet"
"management_bucket_storage_account_name" = "rpmanagementa"
"management_bucket_storage_container_name" = "rpmanagementc"
"management_key_vault_name" = "rp-rp-vault"
"network_resource_group_name" = "rp-network-rg"
"redpanda_connect_api_user_assigned_identity_name" = "rp-redpanda-connect-api-uai"
"redpanda_connect_user_assigned_identity_name" = "rp-redpanda-connect-uai"
"redpanda_operator_user_assigned_identity_name" = "rp-redpanda-operator-uai"
"redpanda_resource_group_name" = "rp-redpanda-rg"
"redpanda_security_group_name" = "rp-redpanda-nsg"
"rp_0_pods_subnet_name" = "rp-snet-rp-0-pods"
"rp_0_vnet_subnet_name" = "rp-snet-rp-0-vnet"
"rp_1_pods_subnet_name" = "rp-snet-rp-1-pods"
"rp_1_vnet_subnet_name" = "rp-snet-rp-1-vnet"
"rp_2_pods_subnet_name" = "rp-snet-rp-2-pods"
"rp_2_vnet_subnet_name" = "rp-snet-rp-2-vnet"
"rp_connect_pods_subnet_name" = "rp-snet-connect-pods"
"rp_connect_vnet_subnet_name" = "rp-snet-connect-vnet"
"storage_resource_group_name" = "rp-storage-rg"
"system_pods_subnet_name" = "rp-snet-system-pods"
"system_vnet_subnet_name" = "rp-snet-system-vnet"
"tiered_storage_account_name" = "rptieredstoragea"
"tiered_storage_container_name" = "rptieredstoragec"
"vnet_name" = "rp-rp-vnet"
}
Delete the cluster
To delete the cluster and all associated resources, first set allow_deletion = true on the redpanda_cluster resource and run terraform apply. Then run terraform destroy:
terraform destroy
Terraform deletes the cluster, runs the Redpanda agent teardown, and then deletes the Redpanda network and the Azure resources.
| This also deletes the customer-managed resources created by the module, including the Tiered Storage account and the data in it. |
| Redpanda Cloud does not support customer access or modifications to any of the internal data plane resources. This restriction allows Redpanda Data to manage all configuration changes internally to ensure a 99.99% service level agreement (SLA) for BYOC clusters. |
Manage custom tags
Your organization might require custom tags for cost allocation, audit compliance, or governance policies. A BYOVNet cluster has two sets of Azure resources, and you tag each set in a different place:
-
The
tagsargument on theredpanda_byovnetmodule tags the customer-managed resources that the module creates, such as the VNet, subnets, managed identities, storage accounts, and key vaults. -
The
tagsargument on theredpanda_clusterresource tags the Azure resources that the Redpanda agent creates for the cluster, such as the AKS cluster and its node pools. Redpanda does not apply these tags to the customer-managed resources.
The example configuration passes local.tags to both. To change the cluster’s tags after creation, edit the tags map and run terraform apply. The provider updates the tags in place, without replacing the cluster. The map replaces the existing custom tags, so include every tag you want to keep. To remove all custom tags, set tags = {}.
Before you change the tags of an existing cluster, refresh the Redpanda agent’s permissions in your subscription. terraform apply doesn’t do this for you. With Terraform 1.14 or later, the provider’s redpanda_byoc_agent_apply action runs the agent apply, the equivalent of rpk cloud byoc azure apply:
variable "cluster_id" {
type = string
}
action "redpanda_byoc_agent_apply" "cluster" {
config {
cluster_id = var.cluster_id
}
}
Set cluster_id to the cluster’s ID, for example in terraform.tfvars. Then invoke the action and apply the tag change:
terraform apply -invoke=action.redpanda_byoc_agent_apply.cluster
terraform apply
Redpanda Cloud allows up to 16 custom tags on an Azure cluster. Don’t use keys that start with redpanda-. Redpanda reserves these keys, and the provider drops them from Terraform state. For the full list of reserved keys, see Reserved tag keys.
Some Azure resources take custom tags only when the cluster is created, so changes to tags after creation don’t reach them:
-
Node pool application security groups (ASGs)
-
Private Link network interfaces for the Kubernetes API server, Tiered Storage, and the Private Link service