CloudFormation is the standard to provision AWS resources. But developing a template is a lot of work. Let’s speed up development and maintenance by working together on high-quality templates:

reviewed by experts

secure

highly available

scalable

easy to deploy and update

built-in monitoring, logging, and visibility

documented

automatically tested

As a maintainer of Free Templates for AWS CloudFormation, I recently reflected on the project. I came up with the following questions to see how things are going.

Many of our clients use our Free Templates for AWS CloudFormation. Sometimes they create a stack on their own; sometimes I help them to create the first stacks. But many of those stacks get never updated after creation. That’s a shame in three ways:

There are security fixes There are bugs There are improvements

I though about a way to make it super easy to create and update stacks. At the moment I experiment with pipelines (CodePipeline) for each template. But I have not yet found an easy way to

define acceptance and production environments in a general way that works for all templates

distribute the changes either from a repo or directly from S3 (should we host the templates or should the user clone them?)

How to assure that templates are working?

In February 2017, I added a test suite to the project. Before that, we maintained the tests in a private repository. But now the tests are also Open Source. What do we test? E.g.:

Is SSH working from the outside

Is the expected HTTP response returned

So each time the master branch changes, we run the entire test suite and create many CloudFormation stacks which take hours to complete and costs us hundreds of US-Dollars each month. But this has paid off. We can make changes to the templates and can be sure that everything is still working.

We are always trying to improve the tests and reduce the time it takes to run them. At the moment, we can not start the tests automatically on new pull requests, and we are a bit concerned because of potential abuse (each PR create AWS charges on our end). The test suite is written in Java, and large parts are developed only for this project. I’m thinking about launching a project to make it easy to test CloudFormatiom templates. Both by looking at the template but also by looking at the stack. Let me know if this is interesting to you!

What is a production-ready template?

I make mistakes. That’s why I always request a review by another expert. The posts on this blog, each template, all pages of AWS in Action were reviewed by my brother Andreas. He always catches something that I have missed. He adds a new perspective to the problem. He questions the whole approach and asks me to solve it differently. You can imagine that this drives my crazy from time to time. But in the end, the result is always better than before. Even if you don’t have a brother or sister with similar interests, you can still use Pull Request to ask a stranger for a review. Maybe you become friends one day! Pull Requests work pretty good

A production-ready template needs to be secure. Security on AWS is complex, but we always try to follow the principle of least privilege. Keep security groups as tight as possible, avoid * in IAM policies. It’s much easier to control Security Groups. The hard part is IAM policies. That’s why I started the Complete AWS IAM Reference. A place where anyone can find the information that is needed to create solid IAM policies. The two projects are closely linked together. I couldn’t maintain the one without the other. I did not understand that when I started them. Today, I know that the reference stays up-to-date because of the templates that I develop. And the templates give me the chance to test what’s in the reference.

Reliable infrastructure is king. One component to achieve this is a highly available architecture. No single point of failures means fewer troubles. This approach also enables things like rolling updates and deployments without downtime. Now you can patch your system at anytime, not only during short maintenance windows at night. AWS makes it super easy to run highly available infrastructure. Many services support it out of the box. Other services need more care. And sometimes you need to be aware of the limitations. In the Free Templates for AWS CloudFormation project, we document limitations. Otherwise, everything is highly available by default.