Table of content

Introduction

Docker v19.03 added a new experimental feature. It is the multi-arch image builder based on the Docker BuildX project.

Also, I started to test Simplexspatial server in my Pine64 cluster, so I need to have the docker image published for two different platforms: amd64 and arm64 .

In this post, I will explain how to create Docker images for different architectures using SBT Native Packager and Docker.

SBT configuration

In the SBT project, I'm using the sbt-native-packager SBT plugin. This allows to package the application in native formats, like msi for Windows, dmg for macOS, deb for Debian distros, rpm for RHEL distro, etc.

In this post, I will talk only about one distribution format: Docker. It will create a Docker image with all stuff necessary to execute the application from any system with Docker installed.

Two simple steps:

Import the sbt plugin. To do it, add the plugin in the project/plugins.sbt : 1

addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.7.0")

Enable Docker and java application packaging in the build.sbt file. Plugings: JavaAppPackaging It will generate a folder structure with the application, dependencies, and a bash script like BashStartScriptPlugin does. I prefer this approach than the painful fat jar.

It will generate a folder structure with the application, dependencies, and a bash script like does. I prefer this approach than the painful fat jar. DockerPlugin enables docker images generation. 1

2

3

4

5

6

7

8

9

10

11

12

13

.enablePlugins(

JavaAppPackaging,

DockerPlugin // Enable the docker plugin

)

.settings(

packageName in Docker := "simplexspatial/simplexspatial",

packageDescription := "The Reactive Geospatial Server",

dockerBaseImage := "adoptopenjdk:11-jre-hotspot", // arm compatible base image

dockerUpdateLatest := true, // docker:publishLocal will replace the latest tagged image.

dockerExposedPorts ++= Seq(2550, 9010, 6080, 7080, 8080),

defaultLinuxInstallLocation in Docker := "/opt/simplexspatial",

dockerExposedVolumes := Seq("/opt/simplexspatial/conf", "/opt/simplexspatial/logs")

)



Generate unique arch Docker image, using sbt-native-packager

To generate the Docker image, it is as simple as to execute sbt docker:publishLocal . Using docker images to list images in your systems, you will find two new images with the name given using the packageName configuration property, and tagged with the version of your project and latest as well.

After building the docker image using sbt plugin, we can check if it is there:

1

2

3

4

5

docker images

REPOSITORY TAG IMAGE ID CREATED SIZE

simplexspatial/simplexspatial 0.0.1-SNAPSHOT 099c0b759140 13 seconds ago 313MB

simplexspatial/simplexspatial latest 099c0b759140 13 seconds ago 313MB

..........



But wait! I want to execute the image in my Pine64 cluster and my Recycled homemade cluster.

So let's check the architecture of the new image:

1

2

docker image inspect 099c0b759140 | grep Architecture

"Architecture": "amd64",



No surprises. amd64 like my laptop!! So It will not run in the Pine64 cluster because nodes are ARMv8 architecture CPUs.

So I'm going to remove the image with sbt docker:clean or docker rmi 099c0b759140 -f and I'll generate it again for amd64 and arm architectures.

Generate multi-arch Docker image, using buildx

So let's create an image for amd64 and another one for armv8 .

From Docker v19.03, there is a new experimental feature based in buildx tool. This tool allows cross platform image creation.

To use this tool, we need to avoid sbt docker:publishLocal and use docker cli directly with the Dockerfile generated by sbt. To do it:

Generate the Dockerfile using SBT. Executing sbt docker:stage will generate the Dockerfile and all files used to generate the Docker image target/docker/stage , but it will not generate the image itself. Enable experimental features in Docker. So It's necessary to add "experimental": true field into the config file /etc/docker/daemon.json and restart the Docker daemon. If the file doesn't exist, create one. 1

2

3

4

5

6

7

8

9

docker version -f '{{.Server.Experimental}}'

false

sudo vi /etc/docker/daemon.json

{

"experimental": true

}

sudo systemctl restart docker

docker version -f '{{.Server.Experimental}}'

true

Enable CLI experimental features (previously we enabled daemon experimental features) through the DOCKER_CLI_EXPERIMENTAL environment variable: 1

2

3

4

5

6

7

8

docker buildx version

docker: 'buildx' is not a docker command.

See 'docker --help'

export DOCKER_CLI_EXPERIMENTAL=enabled

docker buildx version

github.com/docker/buildx v0.3.1-tp-docker 6db68d029599c6710a32aa7adcba8e5a344795a7

Because I'm using Linux, I need to enable binfmt_misc . This gives the capability to the kernel to recognize and run certain types of applications. In our case, it will allow qemu images. To enable binfmt_misc I will run a docker image that is doing the dirty job for us. I'm using the last tag of the image, so before you run it, check for the last image. 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

ls -l /proc/sys/fs/binfmt_misc/

total 0

--w------- 1 root root 0 Apr 14 12:59 register

-rw-r--r-- 1 root root 0 Apr 14 12:59 status

docker run --rm --privileged docker/binfmt:a7996909642ee92942dcd6cff44b9b95f08dad64

Unable to find image 'docker/binfmt:a7996909642ee92942dcd6cff44b9b95f08dad64' locally

a7996909642ee92942dcd6cff44b9b95f08dad64: Pulling from docker/binfmt

5d6ca6c8ba77: Pull complete

b26a8e2c75fc: Pull complete

3436361ddd98: Pull complete

Digest: sha256:758ca0563f371b384cfd67b6590b5be2dc024fef45bc14a050ae104f0caad14e

Status: Downloaded newer image for docker/binfmt:a7996909642ee92942dcd6cff44b9b95f08dad64

ls -l /proc/sys/fs/binfmt_misc/

total 0

-rw-r--r-- 1 root root 0 Apr 14 13:06 qemu-aarch64

-rw-r--r-- 1 root root 0 Apr 14 13:06 qemu-arm

-rw-r--r-- 1 root root 0 Apr 14 13:06 qemu-ppc64le

-rw-r--r-- 1 root root 0 Apr 14 13:06 qemu-riscv64

-rw-r--r-- 1 root root 0 Apr 14 13:06 qemu-s390x

--w------- 1 root root 0 Apr 14 12:59 register

-rw-r--r-- 1 root root 0 Apr 14 12:59 status

grep enable /proc/sys/fs/binfmt_misc/qemu-*

/proc/sys/fs/binfmt_misc/qemu-aarch64:enabled

/proc/sys/fs/binfmt_misc/qemu-arm:enabled

/proc/sys/fs/binfmt_misc/qemu-ppc64le:enabled

/proc/sys/fs/binfmt_misc/qemu-riscv64:enabled

/proc/sys/fs/binfmt_misc/qemu-s390x:enabled

Now, one of the platforms available and enabled is qemu-arm . By default, Buildx is using docker driver to provide the same experience as using the native docker build . It doesn't support cross architecture builds, so it's necessary to use docker-container driver. So I will create a new builder instance and enable it: 1

2

3

4

5

6

7

8

9

10

11

12

13

docker buildx ls

NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS

default * docker

default default running linux/amd64, linux/386

docker buildx create --use --name cross-platform-builder

cross-platform-builde

docker buildx ls

NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS

cross-platform-builder * docker-container

cross-platform-builder0 unix:///var/run/docker.sock inactive

default docker

default default running linux/amd64, linux/386

Now, it is possible to generate images for other platforms using buildx . So let's do it: 1

docker buildx build --platform=linux/arm64,linux/amd64 --push -t simplexspatial/simplexspatial .

Done!!! Afterwards push into the repo, you will find them in Docker Hub repository.

Notes.

How to set a local registry:

Using the Docker registry container:

script 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

docker run -d -p 5000:5000 --name registry registry:2

Unable to find image 'registry:2' locally

2: Pulling from library/registry

486039affc0a: Pull complete

ba51a3b098e6: Pull complete

8bb4c43d6c8e: Pull complete

6f5f453e5f2d: Pull complete

42bc10b72f42: Pull complete

Digest: sha256:7d081088e4bfd632a88e3f3bcd9e007ef44a796fddfe3261407a3f9f04abe1e7

Status: Downloaded newer image for registry:2

4aa12c82df6b1e52ded9560fb8935875a2bbec9920d0e2730b207220602e81a9

docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

4aa12c82df6b registry:2 "/entrypoint.sh /etc…" 6 seconds ago Up 5 seconds 0.0.0.0:5000->5000/tcp registry

90b9a5a4cea5 moby/buildkit:buildx-stable-1 "buildkitd" 2 hours ago Up 2 hours buildx_buildkit_cross-platform-builder0

docker login localhost:5000

Username: angelcc

Password:

WARNING! Your password will be stored unencrypted in /home/angelcc/.docker/config.json.

Configure a credential helper to remove this warning. See

https://docs.docker.com/engine/reference/commandline/login/#credentials-store



Login Succeeded

cat $HOME /.docker/config.json

{

"auths": {

"localhost:5000": {

"auth": "XXXXXXX"

}

},

"HttpHeaders": {

"User-Agent": "Docker-Client/19.03.8 (linux)"

}

}⏎



Local images.

BuildX requires the --output parameter, but there is a bug that is blocking to store it locally, so the easiest option is to use a repository. --push is an alias of --output=type=registry .

Alert!!!! I didn't find the way to deploy into this local repository using buildx build --output . So please, if you know the way, let me know.

SBT + java for ARM64

The default docker image used by SBT is openjdk:8 , but it is not available for arm64 architectures.

, but it is not available for architectures. Keep in mind the memory limitations of arm devices.

Not all openjdk versions are working the same way. I mean, it is easy to get errors related with the openjdk version included in the arm alpine base image. So you will need to play with it until to find a good combination.