Scala is great and all though I’m not familiar with it and the maintainer of the deployment tool I’ve been using since 2016 ended active support for s3_website earlier this year. That’s too bad because s3_website was a huge breath of fresh air for me given its support for deploying both Jekyll and Hugo, among others.

In addition to its support for various generators s3_website also has some novel features for deployments to AWS not trivial otherwise including:

Automated creation of S3 bucket

Automated creation of CloudFront distribution

Deployments with low-cost Reduced Redundancy storage

Multiple CNAME entries for sites on multiple domains

entries for sites on multiple domains Ability to set redirects and routing rules

There are other useful features in s3_website though the above are a few which jumped out at me as none were specifically called out, if available, with the introduction of hugo deploy in Hugo 0.56.0 in as far as I know.

That doesn’t mean to say loss of one or two of the above features would be a deal breaker for me, but I’ve been using AWS for several years and giving some of them up simply isn’t necessary. However that doesn’t mean they might not become available in Go CDK and exposed to the hugo CLI later on. But the CDK supports a broader array of cloud storage systems and being overly prescriptive is unwise.

Giving up some things with the move off s3_website Ruby Gem isn’t entirely unwelcome. For example, using the gem necessitates both Ruby and Java given its construction. And lessening the number of languages required to support deployments while gaining portability affordance is a win.

Here’s how I did it with After Dark. Follow along for the specifics or scan to learn how you can deploy to S3 using hugo deploy too. It’s actually fairly simple. Easier still after learning Zero to HTTP/2 with AWS and Hugo.

Getting Started

Unlike with s3_website , to get started with hugo deploy you’ll need to install the AWS CLI. If you’re running Arch Linux or Manjaro, you can download AWS CLI from the community repository using the following command:

sudo pacman -S aws-cli

Resulting in output like the following:

Expand to view output resolving dependencies... looking for conflicting packages... Packages ( 7 ) python-botocore-1.12.193-1 python-dateutil-2.8.0-1 python-docutils-0.14-2 python-jmespath-0.9.4-1 python-rsa-4.0-1 python-s3transfer-0.2.1-1 aws-cli-1.16.203-1 Total Download Size: 5,22 MiB Total Installed Size: 51,23 MiB :: Proceed with installation? [ Y/n ] :: Retrieving packages... python-dateutil-2.8... 259,3 KiB 92,3K/s 00:03 [ #####################] 100% python-jmespath-0.9... 35,0 KiB 422K/s 00:00 [ #####################] 100% python-docutils-0.1... 657,0 KiB 190K/s 00:03 [ #####################] 100% python-botocore-1.1... 3,2 MiB 421K/s 00:08 [ #####################] 100% python-rsa-4.0-1-any 46,0 KiB 130K/s 00:00 [ #####################] 100% python-s3transfer-0... 94,4 KiB 410K/s 00:00 [ #####################] 100% aws-cli-1.16.203-1-any 1008,0 KiB 700K/s 00:01 [ #####################] 100% ( 7/7 ) checking keys in keyring [ #####################] 100% ( 7/7 ) checking package integrity [ #####################] 100% ( 7/7 ) loading package files [ #####################] 100% ( 7/7 ) checking for file conflicts [ #####################] 100% ( 7/7 ) checking available disk space [ #####################] 100% :: Processing package changes... ( 1/7 ) installing python-dateutil [ #####################] 100% ( 2/7 ) installing python-jmespath [ #####################] 100% ( 3/7 ) installing python-docutils [ #####################] 100% ( 4/7 ) installing python-botocore [ #####################] 100% ( 5/7 ) installing python-rsa [ #####################] 100% ( 6/7 ) installing python-s3transfer [ #####################] 100% ( 7/7 ) installing aws-cli [ #####################] 100% :: Running post-transaction hooks... ( 1/1 ) Arming ConditionNeedsUpdate...

Given Arch doesn’t have Hugo 0.56.0 available in the community repository yet I opted to try building the latest version of Hugo from AUR . Sadly the AUR package was let go by its maintainer back in February and now is also behind too:

Building AUR package for Hugo using Deepin Manjaro.

Leaving me the option to modify the After Dark Dockerfile starting with the original and making some light modifications for 0.56.0 as shown here:

Expand to view file diff diff --git a/docker/hugo/Dockerfile b/docker/hugo/Dockerfile index 1d8cc60a..81500838 100644 --- a/docker/hugo/Dockerfile +++ b/docker/hugo/Dockerfile @@ -17,25 +17,25 @@ # along with this program. If not, see <https://www.gnu.org/licenses/>. # -# DOCKER-VERSION 17.12.0-ce, build c97c6d6 +# DOCKER-VERSION 18.09.7-ce, build 2d0083d657 # Pull hugo builder base image -FROM golang:1.10.3-alpine3.7 AS hugobuilder +FROM golang:1.11.4-alpine3.8 AS hugobuilder # Set environment variables for hugo build -ENV HUGO_VERSION=0.44 \ - CGO_ENABLED=0 \ - GOOS=linux +ENV HUGO_VERSION=0.56.0 \ + CGO_ENABLED=1 \ + GOOS=linux \ + GO111MODULE=on \ + BUILD_TAGS="extended" # Build hugo from source using specified version RUN \ - apk add --update --no-cache git musl-dev && \ + apk add --update --no-cache git gcc g++ binutils musl-dev && \ git clone https://github.com/gohugoio/hugo.git $GOPATH/src/github.com/gohugoio/hugo && \ cd ${GOPATH:-$HOME/go}/src/github.com/gohugoio/hugo && \ git checkout v$HUGO_VERSION && \ - go get github.com/golang/dep/cmd/dep && \ - dep ensure -vendor-only && \ - go install -ldflags '-s -w' + go install -ldflags '-s -w -extldflags "-static"' -tags ${BUILD_TAGS} # Move compiled binary into own container FROM scratch

(Or simply download the updated Dockerfile resulting from the patch.)

Providing a clear path back to 0.44 when desirable for the purpose of ensuring backwards compatibility for After Dark’s current minimum supported version of Hugo. The updated file now includes extended builds by default as well.

After a docker build on the updated Dockerfile Docker Machine does its thing, leaving behind an extended 0.56.0 hugo binary inside a scratch container with the new deploy command now available; the build should finish with success:

Removing intermediate container 6e296bc5bfe4 ---> cc23267f44d2 Step 4/7 : FROM scratch ---> Step 5/7 : COPY --from=hugobuilder /go/bin/hugo /hugo ---> f59753f63f73 Step 6/7 : ENTRYPOINT ["/hugo"] ---> Running in 82ccd94d7cb3 Removing intermediate container 82ccd94d7cb3 ---> 4d539183ba54 Step 7/7 : CMD ["--help"] ---> Running in aa9cb9c06b63 Removing intermediate container aa9cb9c06b63 ---> f2b785583ce8 Successfully built f2b785583ce8

Running docker images should include information like:

IMAGE ID CREATED SIZE f2b785583ce8 26 minutes ago 36.6MB

Which tells us our containerized hugo build is a total of 36.6MB . Not bad considering the intermediate containers can get up to 1.5 GiB or more during the build process as it occurs within the intermediate hugobuilder container.

Build Docker Image

Given an IMAGE ID of f2b785583ce8 containing a successful build we can test if things are working as expected using docker run as shown here:

docker run f2b785583ce8 deploy Error: no deployment targets found

Expect an Error for now and if that’s what you see tag the image:

docker tag $( docker images -q | head -n 1 ) gohugoio/hugo:v0.56.0-extended

Which should give you docker images output like:

REPOSITORY TAG IMAGE ID SIZE gohugoio/hugo v0.56.0-extended f2b785583ce8 36.6MB

Enabling use of the -t flag alongside docker run if desired:

docker run -t gohugoio/hugo:v0.56.0-extended deploy

But to be useful we need to copy the binary somewhere on $PATH by:

docker create -it --name temp f2b785583ce8 sh && \ sudo docker cp temp:/hugo /usr/local/bin && \ docker rm -fv temp

Which uses docker create to pull the hugo binary outside Dockerland and into /usr/local/bin on the host. Depending on your system you may wish to copy Hugo to a different directory – perhaps one early on when you echo $PATH .

Once you’ve copied the hugo binary to your host, check the version with:

hugo version

You should see output like this depending on HUGO_VERSION set in the Dockerfile :

Hugo Static Site Generator v0.56.0/extended linux/amd64 BuildDate: unknown

If you don’t have permission to run it chmod +x the file and you should be ready to rock and roll. You’ve now finished building the latest version of Hugo from source using After Dark’s Hugo Dockerfile.

Add Deployment Config

Deployment config is straight-forward, following a pattern like:

[ deployment ] order = [ ".mp4" , ".gif$" , ".png$" , ".jpg$" , ".bpg$" , ".svg$" ] [[ deployment . targets ]] name = "s3-aws" URL = "s3://after-dark.habd.as?region=us-east-1" cloudFrontDistributionID = "E15C0RT21AL7CY" [[ deployment . matchers ]] pattern = "^.+\\.(js|css|svg|ttf|woff|woff2|eot|png|gif|pdf)$" cacheControl = "max-age=630720000, no-transform, public" gzip = true

With a number of targets and matchers possible as described in the docs. Adding a few of these may cause your config.toml to start feeling a bit bulky – a good time to consider refactoring it to into separate files using Configuration Directories resulting in a deployments.toml like:

order = [ ".mp4" , ".gif$" , ".png$" , ".jpg$" , ".bpg$" , ".svg$" ] [[ targets ]] name = "s3-aws" URL = "s3://after-dark.habd.as?region=us-east-1" cloudFrontDistributionID = "E15C0TR21AL7CY" [[ matchers ]] pattern = "^.+\\.(js|css|svg|ttf|woff|woff2|eot|png|gif|pdf)$" cacheControl = "max-age=630720000, no-transform, public" gzip = true

But functionality to use a deployments.toml wasn’t implemented in Hugo v0.56.0 .

Go ahead if you’re following along and add your targets, matchers and whatnot to your site configuration. When you’re finished you’re ready to deploy.

Deploying to AWS

With the latest version of Hugo now available and little site config you’re ready to deploy to AWS. There are only a few more steps:

install the aws cli

install the aws cli upgrade to hugo 0.56.x

upgrade to hugo 0.56.x configure aws cli

aws cli do a hugo deploy --dryRun

do a back-up s3 bucket (optional)

back-up s3 bucket (optional) perform a deployment

Note: If you’re migrating from s3_website you may not yet be familiar with the AWS CLI. Nevertheless, it’s required with Hugo for AWS as noted in the docs and generally a good tool to have handy.

Remember to Install the AWS CLI on the machine doing the deployments:

pip3 install awscli --upgrade --user # recommended in AWS docs sudo pacman -S aws-cli # suggested Manjaro and Arch Linux users

Configure it as suggested by Hugo using Amazon’s configure docs:

aws configure AWS Access Key ID [None]: REDACTED AWS Secret Access Key [None]: REDACTED Default region name [None]: us-east-1 Default output format [None]: json

And test it using the --dryRun flag:

hugo deploy --dryRun

With your config set expectedly you will see output similar to:

Deploying to target "s3-aws" (s3://after-dark.habd.as?region=us-east-1) Identified 512 file(s) to upload, totaling 7.8 MB, and 0 file(s) to delete. [DRY RUN] Would upload: favicon.png (10 kB, Cache-Control: "max-age=630720000, no-transform, public", Content-Encoding: "gzip", Content-Type: "image/png"): size differs [DRY RUN] Would upload: images/addon-high-tea_1440x900-fs8.png (175 kB, Cache-Control: "max-age=630720000, no-transform, public", Content-Encoding: "gzip", Content-Type: "image/png"): size differs [DRY RUN] Would upload: images/addon-high-tea_960x600-fs8.png (60 kB, Cache-Control: "max-age=630720000, no-transform, public", Content-Encoding: "gzip", Content-Type: "image/png"): size differs [DRY RUN] Would upload: images/feature-instant-view-fs8.png (582 kB, Cache-Control: "max-age=630720000, no-transform, public", Content-Encoding: "gzip", Content-Type: "image/png"): size differs

For a prompt with less verbose output run hugo deploy with the --confirm flag:

hugo deploy --confirm Deploying to target "s3-aws" (s3://after-dark.habd.as?region=us-east-1) Identified 512 file(s) to upload, totaling 7.8 MB, and 0 file(s) to delete. Continue? (Y/n)

And when you’re ready hugo && hugo deploy to fire away:

Your browser doesn't support HTML5 video. Download video instead.

That’s all there is to it.

Troubleshooting

Run hugo deploy -h for usage:

Expand to view output of command Deploy your site to a Cloud provider. See https://gohugo.io/hosting-and-deployment/hugo-deploy/ for detailed documentation. Usage: hugo deploy [ flags ] Flags: --confirm ask for confirmation before making changes to the target --dryRun dry run --force force upload of all files -h, --help help for deploy --invalidateCDN invalidate the CDN cache via the cloudFrontDistributionID listed in the deployment target ( default true ) --maxDeletes int maximum # of files to delete, or -1 to disable (default 256) --target string target deployment from deployments section in config file ; defaults to the first one Global Flags: --config string config file ( default is path/config.yaml | json | toml ) --configDir string config dir ( default "config" ) --debug debug output -e, --environment string build environment --ignoreVendor ignores any _vendor directory --log enable Logging --logFile string log File path ( if set, logging enabled automatically ) --quiet build in quiet mode -s, --source string filesystem path to read files relative from --themesDir string filesystem path to themes directory -v, --verbose verbose output --verboseLog verbose logging

To invalidate cache on CloudFront CDN be sure to set a cloudFrontDistributionID value in config.toml and pass the --invalidateCDN flag when running deploy .

When working with Docker it’s usually a good idea to docker image prune every once in awhile to clean-up disk space. If you start running out of disk space and cannot allocate more I’ve detailed what I know of space management with Docker in the High Tea source repo’s README.md .

If you run into other problems:

Summary

In this tutorial I’ve covered the new Hugo Deploy feature, how to build it from source using Docker and why you might want to depending on your current pipeline. And though I didn’t cover all the features available in Go CDK (such as MinIO support) I still hope you found this short guide a gentle introduction to one of Hugo’s latest features for building static websites, media types and APIs.