Recap

A few days ago, I wrote a post outlining how I use Hugo and Amazon S3 to create a severless blog hosting platform. While this solution works awesome for hosting the site, publishing is still a bit of a pain. After a few too many rounds of drag-and-drop uploading, I set out to find a better publishing workflow.

Intermediate Solution

My first breakthrough was a quick terminal command using the AWS CLI to automatically upload the public directory that Hugo generates. The sync parameter is used instead of upload to make sure that the source and target directories match, and the --delete modified is appended to make sure that any files removed from the source directory are also removed from the target (S3) directory. This ensures that any outdated images or documents are removed! Finally, the -- acl public-read modifier makes sure that the uploaded files are public readable (this should be redundant if you have a public bucket policy, but never hurts).

aws s3 sync PATH/public/ s3://YOUR_BUCKET_NAME/ --delete --acl public-read

CodePipeline Architecture

While my upload script was working great, I started learning more about AWS’s DevOps tools and wanted to see if I might be able to fit them into my writing workflow. For those who don’t know, AWS has a number of developer tools that can be put together a la carte to help build a DevOps delivery pipeline.

CodeCommit : provides a managed source control service which allows AWS users to host versioned source code repositories using Git

: provides a managed source control service which allows AWS users to host versioned source code repositories using Git CodeBuild : provides a managed build service which spins up virtual machines on-demand to build and test code

: provides a managed build service which spins up virtual machines on-demand to build and test code CodeDeploy : automates code deployment to EC2 instances

: automates code deployment to EC2 instances CodePipeline: provides a continuous integration (CI) and continuous delivery (CD) pipeline which connects CodeCommit, CodeBuild, and CodeDeploy to manage source code, build and test the artifacts, and deploy them to the cloud

Since my blog architecture is severless, I don’t need to use the CodeDeploy functionality, which is designed for EC2 instances. Instead, I can use a CodeCommit repository to hold my blog and a CodeBuild script to build it with Hugo and drop the generated code into S3. With those two mechanisms in place, we can stitch together a workflow in CodePipeline which waits for updates to the CodeCommit repository and triggers a Hugo build and S3 deployment automatically. Let’s take a look at how to put that together! Guidance and some code snippets borrowed from Ivan du Toit and Karel Bemelmans.

Building the Pipeline

CodeCommit

Create a new CodeCommit repository

Use directions from AWS to set up the command line on your computer, configure an IAM user for CodeCommit, and clone your new repository

Move all of your Hugo files into the repository

cd to the repository root on your computer and run git add . to add all of the files to your commit

Run git commit -m "YOUR_MESSAGE" to commit the files to your repository

Run git push origin master to push the files upstream

Now, if you access your repository through the CodeCommit console, you should be able to see your code

CodeBuild

Create a new Build Project within AWS CodeBuild Use your CodeCommit repository as the source Select Ubuntu 14.04 as the build environment (Ubuntu 16.04 would be better but is not supported by AWS at this time) Choose S3 as the destination for build artifacts and direct it to the S3 bucket which you are using to serve your site (for me, this is conormclaughlin.net) Create the Build Project

Navigate to the IAM Dashboard and select “Roles” from the sidebar

Attach the AmazonS3FullAccess and AmazonSNSFullAccess policies to the role to allow your build script to transfer the generated files to S3 and notify users of a build

Navigate to the Simple Notification Service (SNS) console within AWS and create a new topic, adding yourself as a subscriber as you see fit

On your computer, navigate to the root folder of your blog repository and create a file titled buildspec.yml version: 0.2 phases: install: commands: - pip install Pygments - wget https://github.com/gohugoio/hugo/releases/download/v0. 21 /hugo_0.21_Linux-64bit.deb - dpkg -i hugo_0.21_Linux-64bit.deb build: commands: - hugo - echo "S3 Upload Beginning" - aws s3 sync public/ s3://YOUR_S3_BUCKET/ --region us-east- 1 --delete --acl public-read - echo "S3 Upload Complete" post_build: commands: - echo "Build complete" - aws sns publish --topic-arn YOUR_TOPIC_ARN --subject 'AWS CodeBuild Complete' --message 'Your build using AWS CodeBuild has been completed. Check the CodeBuild console to see the details.'

Push the changes to your CodeCommit repository so that the buildspec.yml file is in the root of your repository

Preemptively delete all of the files in your server S3 bucket, as long as you are comfortable with some momentary downtime. If the CodeBuild process goes wrong, you can always re-copy the files from your computer

Navigate back the CodeBuild console and start a new build

With any luck, your build should be successful and look like the image below:

CodePipeline

CodePipeline is the final piece in the puzzle to automating your Hugo build and deployment to S3. With CodePipeline in place, simply pushing a new post from your computer to the CodeCommit repository will automatically trigger a CodeBuild operation, culminating with your updated blog being generated and automatically placed into S3.