Terraform for Bare Metal with Matchbox

• By Dalton Hubble

This week, CoreOS released Matchbox v0.6.0 with new Terraform integrations, which enables you to create and share resources within teams for reproducible production infrastructure.

Matchbox, introduced by CoreOS, is an open source service for on-premise environments that matches bare metal machines to profiles in order to PXE boot Container Linux instances and automate cluster provisioning. It can create etcd clusters, Kubernetes clusters, and it powers Tectonic Bare Metal installs. Matchbox can be installed as a binary, as an RPM, as a container image, or run on a Kubernetes cluster. In a PXE-enabled network environment, profiles define: the kernel, initrd, iPXE config, Container Linux config, Cloud-Config, and other configurations to provision complete clusters.

New in the v0.6.0 release is the ability to use Terraform and the terraform-provider-matchbox plugin to push changes to Matchbox via its gRPC API. This enables you to use Terraform to plan, version, and apply (i.e. sync) Matchbox resources to manage bare metal infrastructure.

Getting started

Let’s walk through using Matchbox with Terraform to provision bare metal machines with a Container Linux disk install and an SSH key. You'll deploy a Matchbox instance, use Terraform to plan and apply configs, and setup a PXE network boot environment.

Matchbox

Install Matchbox on a server or Kubernetes cluster that your bare metal machines can reach (e.g. matchbox.example.com) using the official guides:

Enable the gRPC API and use the TLS generation script to create server and client certs. The client credentials will allow Terraform to write to Matchbox, so save the client.crt , client.key , and ca.crt locally (e.g. ~/.matchbox ).

Terraform

Install Terraform v0.9+ on your system.

$ terraform version Terraform v0.9.4

Download the terraform-provider-matchbox plugin binary onto your system.

$ wget https://github.com/coreos/terraform-provider-matchbox/releases/download/v0.1.0/terraform-provider-matchbox-v0.1.0-linux-amd64.tar.gz $ tar xzf terraform-provider-matchbox-v0.1.0-linux-amd64.tar.gz

Add the plugin to your ~/.terraformrc .

providers { matchbox = "/path/to/terraform-provider-matchbox" }

First cluster

Clone the Matchbox source and take a look at the provided Terraform examples.

$ git clone https://github.com/coreos/matchbox.git $ cd matchbox/examples/terraform/simple-install

With the simple-install example, any machines which PXE boot via Matchbox will install Container Linux to dev/sda , reboot, and be provisioned with a configured SSH authorized key.

Configure the variables in variables.tf by creating a terraform.tfvars file.

matchbox_http_endpoint = "http://matchbox.example.com:8080" matchbox_rpc_endpoint = "matchbox.example.com:8081" ssh_authorized_key = "YOUR_SSH_KEY"

Terraform can now interact with Matchbox to create resources.

$ terraform plan Plan: 4 to add, 0 to change, 0 to destroy.

Let's learn about the Matchbox objects, before running terraform apply .

Provider

Matchbox is configured as a provider platform for bare metal resources.

// Configure the Matchbox provider provider "matchbox" { endpoint = "${var.matchbox_rpc_endpoint}" client_cert = "${file("~/.matchbox/client.crt")}" client_key = "${file("~/.matchbox/client.key")}" ca = "${file("~/.matchbox/ca.crt")}" }

Profiles

Machine profiles specify the kernel, initrd, kernel args, Container Linux Config, Cloud-config, or other configs used to network boot and provision a bare metal machine. This profile will PXE boot machines using the current stable Container Linux kernel and initrd (see asset caching for speed) and supply a Container Linux Config specifying that a disk install and reboot should be performed. Learn more about Container Linux configs.

// Create a CoreOS-install profile resource "matchbox_profile" "coreos-install" { name = "coreos-install" kernel = "https://stable.release.core-os.net/amd64-usr/current/coreos_production_pxe.vmlinuz" initrd = [ "https://stable.release.core-os.net/amd64-usr/current/coreos_production_pxe_image.cpio.gz" ] args = [ "coreos.config.url=${var.matchbox_http_endpoint}/ignition?uuid=$${uuid}&mac=$${mac:hexhyp}", "coreos.first_boot=yes", "console=tty0", "console=ttyS0", ] container_linux_config = "${file("./cl/coreos-install.yaml.tmpl")}" }

Groups

Based on a machine’s labels (like MAC or UUID or others), groups match machines to different profiles and input template values. This group named “default” does not have a selector block, so any machines which network boot from Matchbox will be provisioned using the coreos-install profile. Machines are matched to the most specific matching group.

resource "matchbox_group" "default" { name = "default" profile = "${matchbox_profile.coreos-install.name}" # group with no selector matches all machines metadata { ignition_endpoint = "${var.matchbox_http_endpoint}/ignition" ssh_authorized_key = "${var.ssh_authorized_key}" } }

Apply

Apply the Terraform configuration.

$ terraform apply Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

PXE-enabled Network

Matchbox can integrate with many on-premise network setups. It does not seek to be the DHCP server, TFTP server, or DNS server for the network. Instead, Matchbox serves iPXE scripts and GRUB configs as the entrypoint for provisioning network booted machines. PXE clients are supported by chainloading iPXE firmware.

In the simplest case, an iPXE-enabled network can chain to Matchbox:

# /var/www/html/ipxe/default.ipxe chain http://matchbox.foo:8080/boot.ipxe

Read network-setup for the complete range of options. Network admins have a great amount of flexibility:

May keep using existing DHCP, TFTP, and DNS services

May configure subnets, architectures, or specific machines to delegate to Matchbox

May place Matchbox behind a menu entry (timeout and default to Matchbox)

If you've never set up a PXE-enabled network before, or you're trying to set up a home lab, check out the quay.io/coreos/dnsmasq container image examples.

Boot

It’s time to network boot your machines. Use the BMC's remote management capabilities (may be vendor-specific) to set the boot device (on the next boot only) to PXE and power on each machine.

$ ipmitool -H node1.example.com -U USER -P PASS power off $ ipmitool -H node1.example.com -U USER -P PASS chassis bootdev pxe $ ipmitool -H node1.example.com -U USER -P PASS power on

The example assumes machines boot from disk first and PXE only when requested, but you can write profiles for different cases. Once the Container Linux install completes and the machine reboots you can SSH:

$ ssh core@node1.example.com

To reprovision the machine for another purpose, run terraform apply and PXE boot it again.

Going Further

You can use Matchbox to provision multi-node clusters at one or many on-premise sites. Now that you know the basics, check out our cluster examples on provisioning a 3-node etcd cluster or a 3-node Kubernetes cluster! Or improve install times by caching images in the built-in HTTP assets server.

Tectonic is an enterprise Kubernetes offering by CoreOS that uses Matchbox to create bare metal clusters. You can try out the open source tectonic-installer, and Tectonic is free to use up to 10 nodes!

If you have questions about Matchbox, Tectonic, Kubernetes, or other projects, ask the team directly at CoreOS Fest! The Kubernetes and distributed systems conference takes place May 31 and June 1 in San Francisco - join us for two days of talks from the community on the latest developments in the open source container ecosystem. Register today!