Skip to main content
Version: 22.3

Using Terraform and Ansible to Deploy Redpanda

You can deploy Redpanda for production on bare metal with the Redpanda installation binary.

If you use automation tools in your environment, like Terraform and Ansible, you can use those tools for your Redpanda production deployment.

Set up infrastructure with Terraform

  1. Install Terraform. See the Terraform documentation.

  2. To clone the deployment-automation repo:

    git clone https://github.com/redpanda-data/deployment-automation.git
  3. To change into the directory:

    cd deployment-automation
  4. Follow the specific instructions for AWS or GCP:

  1. Within the deployment-automation folder, change into the aws directory:

    cd aws
  2. Set AWS credentials. Terraform provides multiple ways to set the AWS secret and key. See the Terraform documentation.

  3. To initialize Terraform:

    terraform init
  4. To create the cluster:

    terraform apply

    Configuration options:

    PropertyDescription
    aws_regionThe AWS region to use for deploying the infrastructure. Default: us-west-2
    nodesThe number of nodes to base the cluster on. Default: 3
    enable_monitoringCreates a Prometheus/Grafana instance for monitoring the cluster. Default: true
    instance_typeThe instance type on which Redpanda is deployed. Default: i3.8xlarge
    prometheus_instance_typeThe instance type on which Prometheus and Grafana are deployed. Default: c5.2xlarge
    public_key_pathPath to the public key of the keypair used to access the nodes. Default: ~/.ssh/id_rsa.pub
    distroLinux distribution to install. (This affects the distro_ variables.) Default: ubuntu-focal
    distro_amiAWS AMI to use for each available distribution. These must be changed according to the chosen AWS region.
    distro_ssh_userUser used to ssh into the created EC2 instances.

    The following example creates a three-node cluster using i3.large instances.

    terraform apply -var="instance_type=i3.large"

Next steps

Now you can install Redpanda either with the Redpanda installation binary or with Ansible.

Install Redpanda with Ansible

Whether you are using bare metal servers or Terraform, you can use Ansible to install Redpanda.

Prerequisites

Install Ansible. See the Ansible documentation.

Clone the GitHub repository

To clone the repo:

git clone git@github.com:redpanda-data/deployment-automation.git

To change into the directory:

cd deployment-automation

To install the required roles needed by Ansible:

ansible-galaxy install -r ansible/requirements.yml

Configure the hosts.ini file

The hosts.ini file is in the deployment-automation directory. If you used Terraform to deploy the instances, this file is updated automatically. If you did not use Terraform, you must update it manually. When you open the file, and you see something like the following:

[redpanda]
ip ansible_user=ssh_user ansible_become=True private_ip=pip id=0
ip ansible_user=ssh_user ansible_become=True private_ip=pip id=1

[monitor]
ip ansible_user=ssh_user ansible_become=True private_ip=pip id=1

Under the [redpanda] section, replace the following:

PropertyDescription
ipThe public IP address of the machine.
ansible_userThe username for Ansible to use to ssh to the machine.
private_ipThe private IP address of the machine. This could be the same as the public IP address.
idThe node ID of the Redpanda instance. This must be unique for each host.

The [monitor] section is only relevant if you have Prometheus and Grafana installed on a given host. If you don't want to have this deployed, then remove the [monitor] section.

Run the Ansible playbook

To set up Redpanda on your selected nodes:

ansible-playbook --private-key <your_private_key> -i hosts.ini -v ansible/playbooks/provision-node.yml

After this completes, you have a fully running cluster.