As a Big Data Solutions Architect and InfraOps, I require development environments that are configurable, flexible, and high-performing in order to install and test software. For my use case of working with distributed systems, I have found that local virtualized clusters of multiple Linux instances are the best fit. I have been using HashiCorp’s Vagrant along with libvirt/KVM to manage these instances for several years, but I recently discovered a better setup for my needs: LXD for managing instances and Terraform (another HashiCorp tool) for operating LXD. In this article, I will explain the advantages of this setup and provide instructions on how to create such an environment.
Glossary:
– Vagrant: A tool used to create and configure lightweight, reproducible, and portable development environments, commonly used for provisioning virtual machines locally.
– Terraform: A widely used Infrastructure as Code tool that allows provisioning resources on various cloud platforms and self-hosted infrastructure.
– LXD: A Linux Container (LXC) hypervisor that can be used to manage containers and virtual machines.
– HashiCorp: The company behind Vagrant, Terraform, and other popular DevOps tools.
Advantages of Terraform + LXD over Vagrant + libvirt/KVM:
1. Live resizing of instances: LXD containers can be resized without rebooting, providing more flexibility compared to traditional virtual machines.
2. Unified tooling from development to production: LXD can be installed on multiple hosts to create a cluster that serves as the foundation for a self-hosted cloud. Terraform + LXD can then be used to manage environments ranging from local development to production.
3. LXD support in Ansible: Ansible, a tool commonly used for installing and configuring software, has plugins available for connecting to LXD containers. This eliminates the need for an OpenSSH server and simplifies SSH key management.
4. Configuration changes preview: Terraform allows users to preview the changes that a command would apply, preventing unwanted deployments and errors in configuration.
5. Configuration readability and modularity: Terraform’s declarative language makes configurations more readable compared to Vagrant’s Ruby language. Additionally, Terraform’s support for defining modules with inputs and outputs enables the splitting of configurations for better maintainability.
6. Performance gain: Using Terraform + LXD can speed up operations in local development environments, leading to a more enjoyable experience.
To setup a minimal Terraform + LXD environment, follow these steps:
Prerequisites:
– LXD: Install LXD on your computer (refer to the official installation guide).
– Terraform: Install Terraform version 0.13 or later (refer to the official installation guide).
– Linux cgroup v2: Ensure your host uses cgroup v2, which is necessary for running recent Linux containers. You can check this by running the command `stat -fc %T /sys/fs/cgroup`. If cgroup v2 is not enabled, refer to the instructions for enabling it.
1. Create a directory to work from: `mkdir terraform-lxd-xs && cd terraform-lxd-xs`
2. Create a file named `provider.tf` and add the following content:
“`
terraform {
required_providers {
lxd = {
source = “terraform-lxd/lxd”
version = “1.7.1”
}
}
}
provider “lxd” {
generate_client_certificates = true
accept_remote_certificate = true
}
“`
3. Create a file named `variables.tf` and add the variables defined for your environment. These variables allow users to configure the Terraform environment. Here is an example:
“`
variable “xs_storage_pool” {
type = object({
name = string
source = string
})
}
variable “xs_network” {
type = object({
ipv4 = object({
address = string
})
})
}
variable “xs_profiles” {
type = list(object({
name = string
limits = object({
cpu = number
memory = string
})
}))
}
variable “xs_image” {
type = string
default = “images:rocky/8”
}
variable “xs_containers” {
type = list(object({
name = string
profile = string
ip = string
}))
}
“`
4. Create a file named `main.tf` and define all the resources using the variables specified. Here is an example:
“`
resource “lxd_storage_pool” “xs_storage_pool” {
name = var.xs_storage_pool.name
driver = “dir”
config = {
source = “${path.cwd}/${path.module}/${var.xs_storage_pool.source}”
}
}
resource “lxd_network” “xs_network” {
name = “xsbr0”
config = {
“ipv4.address” = var.xs_network.ipv4.address
“ipv4.nat” = “true”
“ipv6.address” = “none”
}
}
resource “lxd_profile” “xs_profiles” {
depends_on = [lxd_storage_pool.xs_storage_pool]
for_each = {
for index, profile in var.xs_profiles : profile.name => profile.limits
}
name = each.key
config = {
“boot.autostart” = false
“limits.cpu” = each.value.cpu
“limits.memory” = each.value.memory
}
device {
type = “disk”
name = “root”
properties = {
pool = var.xs_storage_pool.name
path = “/”
}
}
}
resource “lxd_container” “xs_containers” {
depends_on = [
lxd_network.xs_network,
lxd_profile.xs_profiles
]
for_each = {
for index, container in var.xs_containers : container.name => container
}
name = each.key
image = var.xs_image
profiles = [each.value.profile]
device {
name = “eth0”
type = “nic”
properties = {
network = lxd_network.xs_network.name
“ipv4.address” = each.value.ip
}
}
}
“`
5. Create a file named `local.auto.tfvars` and provide the specific variables for your environment. For example:
“`
xs_storage_pool = {
name = “xs_storage_pool”
source = “lxd-xs-pool”
}
xs_network = {
ipv4 = {
address = “192.168.42.1/24”
}
}
xs_profiles = [
{
name = “xs_master”
limits = {
cpu = 1
memory = “1GiB”
}
},
{
name = “xs_worker”
limits = {
cpu = 2
memory = “2GiB”
}
}
]
xs_image = “images:rockylinux/8”
xs_containers = [
{
name = “xs-master-01”
profile = “xs_master”
ip = “192.168.42.11”
},
{
name = “xs-master-02”
profile = “xs_master”
ip = “192.168.42.12”
},
{
name = “xs-worker-01”
…
},
…
]
“`
6. Run `terraform init` to initialize the Terraform environment.
7. Run `terraform apply` to create the environment described in your Terraform files.
This is a basic guide to setting up a Terraform + LXD environment for managing development environments. Additional configuration and customization may be required based on your specific needs and infrastructure.