In this era of cloud computing, many companies are adopting immutable infrastructure to simplify configuration management and reliability using infrastructure as code. In immutable infrastructure, instead of making changes on a running server, create new servers. Building immutable infrastructure is hard and needs a sophisticated process for building and testing them. Best way to implement immutable infrastructure is to have everything done through code which includes server images, infrastructure as a code, any specific configuration using code. We already looked in infrastructure as code using terraform, configuration management using ansible. In this post, we keep our focus on machine images using Hashicorp Packer.

When we launch a set of applications and services for accomplishing a specific purpose on an instance, going to the instance and setting them up can take some time. To solve this issue, we can use machine images which acts as a single static unit that contains installed software, applications, and operating system. In the context of AWS, AMI or Amazon Machine Image is a master image containing the information required to launch an instance server in the cloud. AMI is like a template for the root volume required to launch an instance. They include the operating systems, application services, and applications along with other files or configurations. Once we create an AMI and register it, then it can be used to launch a new instance.

Process for creating a machine image include below steps:

Boot an instance using the base image. Run configuration tools like ansible or scripts to configure the instance into the desired state. Create a new machine image from this server

Once a new machine image is created, then booting a new server from this new image will give all configuration already done on this instance. This helps us with a smooth deployment process. This also helps scale our services fast.

Hashicorp Packer is a lightweight, easy to use tool which automates the creation of any type of machine images for multiple platforms. Packer is not a replacement of configuration management tools like Ansible. Packer works with tools like ansible to install software while creating images. Packer uses a configuration file to create a machine image. Packer uses the concepts of builders to spin up an instance, run provisioners to configure applications or services. Once setup is done, it shuts the instance down and save new baked machine instance with any needed post-processing. Packer only builds images.

Advantages of using Packer:

Fast infrastructure deployment: Machine images allow us to launch provisioned and configured machines faster. Scalable: Packer install and configure all software for a machine during image creation time, so we don’t need to run any configuration management tool. With same AMI we can spawn any number of instances without doing extra configuration. Multi-provider support: Packer can be used to create images for multiple cloud providers like AWS, GCP, Digital Ocean etc. Provides testability: Machine images can be tested to verify that they are working.

Disadvantages of using Packer:

Manageability: No AMI manageability is provided by packer. You need to manage them yourself using tags or versions. Keep deleting old unused AMIs.

Packer configuration file has three main components:

User variable: User variables are used to parameterised packer configuration file template so we can keep secret, environment variables and other parameters out of the template. They help with the portability of configuration file. They help to separate out the part which can be modified in our template. Variables can be passed through command lines, environment variables, Vault or files. The variable section is a key value mapping with the variable name assigned to a default value.

Builder: Builders section contains a list of builders that packer uses to generate a machine image. Builders are responsible for creating an instance and generating machine images from them. A builder maps to a single machine image. Builder contains information including type which is the name of the builder, access keys which builder require to connect to the platform like AWS.

Provisioners: Provisioners section contains a list of provisioners that packer use to install and configure software with in running instance before creating a machine image. Provisioners are optional. Provisioner contains type which specifies the name of provisioner like Shell, Ansible etc.

Let’s understand with an example of creating a new AMI with nginx installed on top of base Amazon Linux 2 AMI provided by AWS. We are going to move all parameters which can be changed in future in the user variable section. We are going to use environment variables for access and secret keys. We set the default value to environment variable value using env function in user variable section.

"aws_access_key": "{{env `AWS_ACCESS_KEY_ID`}}",

"aws_secret_key": "{{env `AWS_SECRET_ACCESS_KEY`}}",

Other variables are defined with default values. You can change these values based on requirement.

"variables": {

"aws_access_key": "{{env `AWS_ACCESS_KEY_ID`}}",

"aws_secret_key": "{{env `AWS_SECRET_ACCESS_KEY`}}",

"region": "us-east-1",

"ssh_username": "ec2-user",

"base_ami": "ami-0e6d2e8684d4ccb3e",

"instance_type": "t2.micro"

}

We are going to generate EBS backed machine image, so need to define the type as ‘amazon-ebs’ in builder section. As we are going to create single AMI so only one JSON section comes in this. Need to define other parameters needed for backing a new image like source AMI name, access key, secret key, region etc.

"builders": [

{

"type": "amazon-ebs",

"access_key": "{{user `aws_access_key`}}",

"secret_key": "{{user `aws_secret_key` }}",

"region": "{{user `region` }}",

"source_ami": "{{user `base_ami`}}",

"instance_type": "{{user `instance_type` }}",

"ssh_username": "{{user `ssh_username`}}",

"ami_name": "packer-base-{{timestamp}}",

"associate_public_ip_address": true

}

]

Once the builder is ready, we jump to provisioners for defining what scripts or configuration management needs to run. We are going to run shell commands to install nginx. In Amazon Linux 2, we first need to enable nginx using amazon-linux-extras. Once it is enabled, then we can install using yum command and install nginx.

"provisioners": [

{

"type": "shell",

"inline": [

"echo ENABLE NGINX",

"sudo amazon-linux-extras enable nginx1.12",

"echo INSTALL NGINX",

"sudo yum -y install nginx",

"echo START NGINX",

"sudo systemctl start nginx"

]

}

]

We need to validate whether our packer file is valid using validate command.

packer validate example.json

We need to execute packer JSON file using build command to generate a new AWS AMI.

packer build example.json

If something goes wrong during the execution of provisioner file we get an error message in the end.

Build 'amazon-ebs' errored: Script exited with non-zero exit status: 1

If everything works fine then we get the below-mentioned message.

amazon-ebs output will be in this color. ==> amazon-ebs: Prevalidating AMI Name: packer-base-1539107879 amazon-ebs: Found Image ID: ami-0e6d2e8684d4ccb3e ==> amazon-ebs: Creating temporary keypair: packer_5bbcec27-767b-384c-c5d7-c047ebc099c6 ==> amazon-ebs: Creating temporary security group for this instance: packer_5bbcec2a-37e9-602c-9fdc-fd504f4d17a1 ==> amazon-ebs: Authorizing access to port 22 from 0.0.0.0/0 in the temporary security group... ==> amazon-ebs: Launching a source AWS instance... ==> amazon-ebs: Adding tags to source instance amazon-ebs: Adding tag: "Name": "Packer Builder" amazon-ebs: Instance ID: i-0b738eeecacf78658 ==> amazon-ebs: Waiting for instance (i-0b738eeecacf78658) to become ready... ==> amazon-ebs: Using ssh communicator to connect: 54.80.150.83 ==> amazon-ebs: Waiting for SSH to become available... ==> amazon-ebs: Connected to SSH! ==> amazon-ebs: Provisioning with shell script: /var/folders/bz/90_0_cjj71b9vmsl6qsp8dwc0000gp/T/packer-shell802152345 amazon-ebs: ENABLE NGINX amazon-ebs: 0 ansible2 available [ =2.4.2 =2.4.6 ] amazon-ebs: 1 emacs available [ =25.3 ] amazon-ebs: 2 httpd_modules available [ =1.0 ] .................

Some More Message

................. amazon-ebs: nginx-mod-mail.x86_64 1:1.12.2-1.amzn2.0.2 amazon-ebs: nginx-mod-stream.x86_64 1:1.12.2-1.amzn2.0.2 amazon-ebs: stix-fonts.noarch 0:1.1.0-5.amzn2 amazon-ebs: amazon-ebs: Complete! amazon-ebs: START NGINX ==> amazon-ebs: Stopping the source instance... amazon-ebs: Stopping instance, attempt 1 ==> amazon-ebs: Waiting for the instance to stop... ==> amazon-ebs: Creating unencrypted AMI packer-base-1539107879 from instance i-0b738eeecacf78658 amazon-ebs: AMI: ami-0c185c9c51c96be52 ==> amazon-ebs: Waiting for AMI to become ready... ==> amazon-ebs: Terminating the source AWS instance... ==> amazon-ebs: Cleaning up any extra volumes... ==> amazon-ebs: No volumes to clean up, skipping ==> amazon-ebs: Deleting temporary security group... ==> amazon-ebs: Deleting temporary keypair... Build 'amazon-ebs' finished. ==> Builds finished. The artifacts of successful builds are: --> amazon-ebs: AMIs were created: us-east-1: ami-0c185c9c51c96be52

Once packer file is executed successfully we get newly generated AMI name printed in the end. In this case, our latest AMI name is “ami-0c185c9c51c96be52”.

Hope this article helps you find enough motivation to use packer for backing AMI using code.

The complete code can be found in this git repository: https://github.com/MiteshSharma/PackerAmi

PS: If you liked the article, please support it with claps. Cheers