I have been toying around a bit with Docker recently. It’s an open source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more. An open source project to pack, ship and run any application as a lightweight container.

If you haven’t tried Docker yet, I invite you to do so. They have a neat getting started page.

There could be a lot of reasons to use it at Nuxeo. Here are a few I have been thinking about:

Why Docker?

Jenkins on demand build slaves

Cloud deployment (obviously)

Remote/unified dev environments

On demand converters for Nuxeo

Hence the Part 1 in the title. I can’t tell you when this will end, but I can tell you there will be more :-)

Anyway, I started doing a Nuxeo image. The Docker way would be to have an image for each process ( apache2 , postgresql , nuxeoctl ) but I wanted to start with an all-inclusive image. I thought it would be simpler. :-)

And it is indeed quite simple once you know your way around the supervision tool. Because when you run a Docker image, you can only run one process. So you have to use tools like Supervisor in order to run more processes. I have chosen Supervisor because it was the most widely used as far as I could see on the web. So my first working image contained everything needed to run Nuxeo and used Supervisor to launch everything. And everything was in a single Dockerfile.

A Dockerfile is like a recipe to build an image. It contains a list of steps that are all versioned. Differences between each step of the image are stored on the filesystem. That means that if you add a new step at the end, for instance, you don’t have to go through all the steps again.

Putting every step in a single file is of course not the best way to do it when you want to do different Nuxeo images. For continuous integration, for example, I will need an image with H2, one with PostgreSQL, one with MySQL etc… So I started splitting my Dockerfile to have something more modular. Here’s the result.

The Base Image

The purpose of this image it to give you an up to date Ubuntu distribution with all the dependencies needed by Nuxeo, a nuxeo user and Supervisor to manage your processes. This will be the basis for the next images to build.

FROM ubuntu:precise MAINTAINER Laurent Doguin <[email protected]> RUN locale-gen --no-purge en_US.UTF-8 ENV LC_ALL en_US.UTF- 8 ENV DEBIAN_FRONTEND noninteractive RUN apt-get install -y python-software-properties wget sudo net-tools RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list RUN apt-add-repository "deb http://apt.nuxeo.org/ precise fasttracks" RUN wget -q -O - http://apt.nuxeo.org/nuxeo.key | apt-key add - RUN apt-get update RUN apt-get upgrade -y RUN apt-get -y install fuse || true RUN rm -rf /var/lib/dpkg/info/fuse.postinst RUN apt-get -y install fuse RUN sudo apt-get install -y acpid openjdk-7-jdk libreoffice imagemagick poppler-utils ffmpeg ffmpeg2theora ufraw libwpd-tools perl locales pwgen dialog supervisor unzip vim RUN mkdir -p /var/ log /supervisor RUN useradd -m -d /home/nuxeo -p nuxeo nuxeo && adduser nuxeo sudo && chsh -s /bin/bash nuxeo ENV NUXEO_USER nuxeo

To build this image, get into the folder containing the Dockerfile and run:

docker build -t nuxeo/nuxeobase .

It’s always good to use the -t option and give a name to the images you build. It’s even more important since it will be used in the next image.

Note that running this image won’t get you anywhere, there is no Nuxeo installed on it.

An All Inclusive Nuxeo Image

This Docker image is based on the work Mathieu did for our VM. I had to adapt one or two things but it’s really close to the original. I mostly added supervisor since I have this one process limitation. Here’s the configuration I used:

[supervisord] nodaemon = true </code> [program:sshd] command =/usr/sbin/sshd -D [program:apache2] command =/bin/bash -c "source /etc/apache2/envvars && /usr/sbin/apache2 -DFOREGROUND" redirect_stderr = true [program:postgresql] user =postgres command =/usr/lib/postgresql/ 9.3 /bin/postgres -D /var/lib/postgresql/ 9.3 /nuxeodb -c config_file=/etc/postgresql/ 9.3 /nuxeodb/postgresql.conf redirect_stderr = true autorestart = true [eventlistener:pgListener] command =python pgListener.py events =PROCESS_STATE_RUNNING numprocs = 1

It will run Apache, PostgreSQL and the ssh daemon. But as you can see there is still no trace of a Nuxeo process. It will actually be launched by the pgListener event listener. It’s a python script that waits for the postgresql process to be running. The code is pretty simple as you can see. You just have to use the Supervisor Python API to read Supervisor events. Once the event saying the postgresql process entered the running state occurs, the firstboot.sh script is launched.

import os import sys from supervisor import childutils def main () : while 1 : headers, payload = childutils.listener.wait() if headers[ 'eventname' ].startswith( 'PROCESS_STATE_RUNNING' ): pheaders, pdata = childutils.eventdata(payload+ 'n' ) if pheaders[ 'processname' ] == "postgresql" : os.system( "sh /root/firstboot.sh" ) break childutils.listener.ok() if __name__ == '__main__' : main()

Now about the firstboot.sh script. It configures the database if it has not been done already and starts Nuxeo.

if [ -f /root/firstboot_done ]; then su $NUXEO_USER -m -c "$NUXEOCTL --quiet restart" exit 0 fi pgpass=$(pwgen -c1) su postgres -c "psql -p 5432 template1 --quiet -t -f-" << EOF > /dev/null CREATE USER nuxeo WITH PASSWORD CREATE LANGUAGE plpgsql; CREATE FUNCTION pg_catalog. text ( integer ) RETURNS text STRICT IMMUTABLE LANGUAGE SQL AS CREATE CAST ( integer AS text ) WITH FUNCTION pg_catalog. text ( integer ) AS IMPLICIT; COMMENT ON FUNCTION pg_catalog. text ( integer ) IS CREATE FUNCTION pg_catalog. text (bigint) RETURNS text STRICT IMMUTABLE LANGUAGE SQL AS CREATE CAST (bigint AS text ) WITH FUNCTION pg_catalog. text (bigint) AS IMPLICIT; COMMENT ON FUNCTION pg_catalog. text (bigint) IS EOF su postgres -c "createdb -p 5432 -O nuxeo -E UTF-8 nuxeo" cat << EOF >> /etc/nuxeo/nuxeo.conf nuxeo.templates=postgresql nuxeo.db.host=localhost nuxeo.db.port= 5432 nuxeo.db.name=nuxeo nuxeo.db.user=nuxeo nuxeo.db.password=$pgpass EOF su $NUXEO_USER -m -c "$NUXEOCTL --quiet restart" touch /root/firstboot_done

Now about the Dockerfile. Notice the first line that says FROM nuxeo/nuxeobase. It’s the name of the image built earlier. Then what happens is we download the latest LTS, set up environment variables, install PostgreSQL, Apache and the OpenSSH server. Once this is done we add different configuration files and scripts. Supervisor.conf and nuxeo.apache2 are respectively the Supervisor configuration and the Apache configuration. I have already told you about pgListener.py and firstboot.sh. What about the two others?

Postinst.sh is dedicated to configuration. We use it to extract Nuxeo from its archive, create the various files and folders needed (like /etc/nuxeo/ , /var/lib/nuxeo , etc..), setup the good permissions, modify nuxeo.conf, drop the main PostgreSQL cluster and create our own nuxeodb cluster and setup Apache.

Entrypoint.sh is run each time you start the Docker container thanks to the ENTRYPOINT command. I use it to re-generate the ssh keys and set a random password if it has not already been done. Then I echo the password and run what’s been given as CMD using exec “[email protected]”

I am not sure if it’s supposed to be used like that, but it allows me to display the password and keep flexibility through CMD, because if you don’t want to run supervisiond directly, you can give /bin/bash at the end of your docker run command.

FROM nuxeo/nuxeobase MAINTAINER Laurent Doguin <[email protected]> RUN wget http://community.nuxeo.com/static/releases/nuxeo-5.8/nuxeo-cap-5.8-tomcat.zip && mv nuxeo-cap-5.8-tomcat.zip nuxeo-distribution.zip ENV NUXEOCTL /var/lib/nuxeo/server/bin/nuxeoctl ENV NUXEO_CONF /etc/nuxeo/nuxeo.conf RUN apt-add-repository "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" RUN wget -q -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - RUN sudo apt-get install -y openssh-server apache2 postgresql-9.3 RUN mkdir -p /var/run/sshd ADD supervisord.conf /etc/supervisor/conf.d/supervisord.conf ADD nuxeo.apache2 /etc/apache2/sites-available/nuxeo ADD postinst.sh /root/postinst.sh ADD firstboot.sh /root/firstboot.sh ADD entrypoint.sh /entrypoint.sh ADD pgListener.py pgListener.py RUN /root/postinst.sh EXPOSE 22 80 ENTRYPOINT [ "/entrypoint.sh" ] CMD [ "/usr/bin/supervisord" ]

You can run it like this:

docker run -P -d nuxeo/nuxeo

Now your container should be running, let’s see if this is true. Typing docker ps should output something like this:

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 02717 b82d503 nuxeo/nuxeo:latest /entrypoint.sh /usr/ 5 seconds ago Up 4 seconds 0.0 .0 .0 : 49153 -> 22 /tcp, 0.0 .0 .0 : 49154 -> 80 /tcp sick_tesla

Thanks to the -P option used in the run command, all the exposed ports are mapped automatically. So if you go to yourDockerHost:49154 with a web browser, you’ll have the running Nuxeo instance.

Now for some reason, like say debugging, you might want to open an ssh session to the server. We already have the port thanks to docker ps . Now we need the password. It’s in the logs thanks to the entrypoint.sh script, so if you type docker logs 02717b82d503 , 02717b82d503 being the container id, you should see something like:

Creating SSH2 RSA key; this may take some time .. . Creating SSH2 DSA key; this may take some time .. . Creating SSH2 ECDSA key; this may take some time .. . start: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused Default password for the root and nuxeo users: EzeegaiN 2014-01-16 16:11:52,410 CRIT Supervisor running as root ( no user in config file) 2014-01-16 16:11:52,410 WARN Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing 2014-01-16 16:11:52,436 INFO RPC interface 'supervisor' initialized 2014-01-16 16:11:52,436 WARN cElementTree not installed, using slower XML parser for XML-RPC 2014-01-16 16:11:52,437 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2014-01-16 16:11:52,437 INFO supervisord started with pid 1 2014-01-16 16:11:53,439 INFO spawned: 'pgListener' with pid 70 2014-01-16 16:11:53,441 INFO spawned: 'sshd' with pid 71 2014-01-16 16:11:53,443 INFO spawned: 'postgresql' with pid 72 2014-01-16 16:11:53,445 INFO spawned: 'apache2' with pid 73 2014-01-16 16:11:54,737 INFO success: pgListener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2014-01-16 16:11:54,738 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2014-01-16 16:11:54,739 INFO success: postgresql entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2014-01-16 16:11:54,739 INFO success: apache2 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2014-01-16 16:12:05,617 INFO exited: pgListener (exit status 0; expected)

The password is on the fifth line. So now you can do something like ssh [email protected] -p 49153 and open a session to your container.

And there you go now you have a fully functional docker container to test the Nuxeo Platform