Jailing GUI Applications

This is a short tutorial on how to run GUI applications jailed. This is done primarily for Firefox but the same principle can be applied for any other application.

There are tutorials on the net that involve connecting to the jail using ssh with X forwarding, for example this fine FreeBSD Forum post. That works too, but is not strictly necessary on the local system, and it is actually much, much slower than using the unix socket directly, even if OpenSSH from ports is used with the None cipher enabled. Alternatively to ssh, and if unix sockets are not wanted, X can be configured to listen on tcp. The config for that is depending on your DE/WM, but in case of i3wm launched with startx , one needs startx -- -listen tcp , and then the DISPLAY env. var (see below) must be changed to either the host IP (mind the firewall rules) or the local jail IP since X will listen on all interfaces, eg. DISPLAY=10.0.0.2:0.0 .

We start by creating a basic jail that depends on your favorite jail abstraction tool. But to unhide all the abstracted details that those tools often do, we're going to do it all manually, and based on ZFS, so feel free to adapt the tasks for whatever jail management tool you're using.

To begin with, we'll dedicate zroot/jails as the base dataset for all jail deployments, and create a basejail in zroot/jails/basejail . The basejail method allows us to simply zfs clone it into target jail dataset, which is the most efficient way to share the base jail among other jails. Updating them all then requires updating the basejail and re-creating jail roots. Here's how.

The commands given are all executed as root, on the host, unless explicitly stated differently.

1 zfs create -o compress =lz4 -o atime =off zroot/jails zfs create -o=lz4 -o=off zroot/jails 2 zfs create zroot/jails/basejail zfs create zroot/jails/basejail 3 bsdinstall jail /zroot/jails/basejail bsdinstall jail /zroot/jails/basejail

Here we used the bsdinstall method for convenience. Otherwise downloading and unpacking base.txz and configuring it should suffice. At this point we'd configure the basejail for pkg, like the location of your Poudriere repo if you have it. No other configuration is required for the base as the jails will basically run a single process.

So we snapshot it and create our Firefox jail filesystem:

1 zfs snapshot zroot/jails/basejail@latest zfs snapshot zroot/jails/basejail@latest 2 zfs create zroot/jails/firefox zfs create zroot/jails/firefox 3 zfs clone zroot/jails/basejail@latest zroot/jails/firefox/root zfs clone zroot/jails/basejail@latest zroot/jails/firefox/root 4 zfs create zroot/jails/firefox/var zfs create zroot/jails/firefox/var 5 zfs create zroot/jails/firefox/tmp zfs create zroot/jails/firefox/tmp 6 zfs create zroot/jails/firefox/home zfs create zroot/jails/firefox/home 7 rsync -a /zroot/jails/firefox/root/var/ /zroot/jails/firefox/var/ rsync -a /zroot/jails/firefox/root/var/ /zroot/jails/firefox/var/ 8 zfs set mountpoint =/zroot/jails/firefox/root/var zroot/jails/firefox/var zfs=/zroot/jails/firefox/root/var zroot/jails/firefox/var 9 zfs set mountpoint =/zroot/jails/firefox/root/tmp zroot/jails/firefox/tmp zfs=/zroot/jails/firefox/root/tmp zroot/jails/firefox/tmp 10 zfs set mountpoint =/zroot/jails/firefox/root/usr/home zroot/jails/firefox/home zfs=/zroot/jails/firefox/root/usr/home zroot/jails/firefox/home

For extra security we want our jail to run with minimum require privilege, so we set some properties on these datasets, which should make obvious why we separated them like this. Of course, these rules are not applicable to every application, as some, unfortunately would like to write or execute to/from paths they shouldn't. For firefox, these suffice, tho'.

1 zfs set setuid =off exec =off zroot/jails/firefox/var zfs=off=off zroot/jails/firefox/var 2 zfs set setuid =off exec =off zroot/jails/firefox/tmp zfs=off=off zroot/jails/firefox/tmp 3 zfs set setuid =off exec =off zroot/jails/firefox/home zfs=off=off zroot/jails/firefox/home

At this point it's worth observing that when base is to be update, all we need to do is update the basejail and create a new snapshot for cloning. With that, and separate var/home/tmp dirs, it's trivial to update the jails' bases, just zfs destroy root dataset and re-clone it from basejail. This will require unmounting and re-mounting the other datasets, but it can all be easily scripted for simple maintenance.

Next, with the filesystem in place, we install the packages. xauth and firefox are the base minimum, while liberation-fonts-ttf is recommended addition for some nice fonts in Firefox.

1 pkg -c /zroot/jails/firefox/root install firefox xauth liberation-fonts-ttf pkg -c /zroot/jails/firefox/root install firefox xauth liberation-fonts-ttf

Next, we need to bootstrap launching firefox. The idea here is that all it takes to "launch firefox" is to launch its jail, and when the browser is closed, the jail should shut down too. This is achieved relatively simply. To make bootstrapping easier, we'll now launch the jail and run the commands in it. But first, we need to set up the jail.conf definition of it:

# /etc/jail.conf allow.nomount; exec.clean; mount.devfs; host.hostname = "$name.your-host-name.lan"; path = "/zroot/jails/${name}/root"; #securelevel = 3; firefox { ip4.addr = "10.0.0.2"; #exec.start = "/bin/sh /home/firefox/run-firefox"; #exec.jail_user = "firefox"; persist; devfs_ruleset = 5; }

At this point, we comment out the exec. directives, and uncomment the persist directive because we want to get inside the jail with no processes running, to bootstrap it. But before we do that, there are two more undefined items here, the devfs ruleset and jail's ip address. So, let's handle those first.

The recommended way to set up the jails' networking is to clone lo0 and give it a dedicated range, then use pf to NAT the traffic out. In short these three things:

1 2 3 4 5 6 7 8 cloned_interfaces = lo1 = lo1 9 ifconfig_lo1_aliases = "10.0.0.1-6/29" 10 11 12 pf_enable = "YES" 13 pf_rules = "/etc/pf.conf"

We enable the network by running service netif cloneup , and the defined lo1 interface will be cloned with that IP range.

Then the pf rules. These are the minimum required to get NAT going, and it doesn't do anything else, so adjust as needed, or if you already have pf in place, the nat line is all that's required.

1 2 3 4 extif = "re0" 5 intif = "lo1" 6 7 set skip on lo skip on lo 8 set state-policy if -bound state-policy-bound 9 10 nat on $extif inet from ( $intif ) to ! ( $intif ) -> ( $extif ) nat oninet from () to ! () -> (

And after that, service pf start should start and enable the firewall. If you're doing this over SSH, this article assumes you know what you're doing, missing the appropriate rules.

Next, the devfs ruleset is required to allow audio devices in the jail, so the applications can play audio. We copy the default ruleset for jails (ruleset number 4) from /etc/defaults/devfs.rules and add audio devices, into /etc/devfs.rules :

# /etc/devfs.rules [devfsrules_desktop_jail=5] add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login add path 'mixer*' unhide add path 'dsp*' unhide

Basically we just added mixer* and dsp* devices in addition to the default jails ruleset. With that file in place, we just have to service devfs restart and the ruleset is in effect.

So now we're ready to start the jail and jexec into it for final setup.

A note on DISPLAY env variable The below approach uses a custom shell script that's started by the jail's exec.start , in which the DISPLAY environment variable is set. This is not strictly needed or the best approach. It suffices to put the in-jail users into a login class, and define an ENV for that class in /etc/login.conf : :setenv=DISPLAY=\c0:\ Don't forget to cap_mkdb /etc/login.conf in the jail. \c is escape sequence for colon.

1 2 jail -c firefox jail -c firefox 3 4 5 jexec -l firefox jexec -l firefox 6 7 8 pw useradd firefox -w random -m pw useradd firefox -w random -m 9 10 11 cat << EOF > /home/firefox/run-firefox cat 12 #!/bin/sh 13 14 export DISPLAY=:0.0 15 /usr/local/bin/firefox > /dev/null & 16 EOF 17 18 19 chown firefox:firefox /home/firefox/run-firefox chown firefox:firefox /home/firefox/run-firefox 20 chmod u+x /home/firefox/run-firefox chmod u+x /home/firefox/run-firefox 21 22 23 mkdir /tmp/.X11-unix mkdir /tmp/.X11-unix 24 chmod 777 /tmp/.X11-unix chmod/tmp/.X11-unix 25 26 27 exit

And that's it. We stop the jail with jail -r firefox , uncomment the exec. bits from jail.conf , comment the persist bit, and the jail is almost ready to run. Finally:

1 2 xhost + xhost + 3 4 5 mount_nullfs /tmp/.X11-unix /zroot/jails/firefox/root/tmp/.X11-unix mount_nullfs /tmp/.X11-unix /zroot/jails/firefox/root/tmp/.X11-unix 6 7 8 zfs set readonly =on zroot/jails/firefox/root zfs=on zroot/jails/firefox/root

Done. We start firefox by starting the jail itself:

1 jail -c firefox jail -c firefox

With the persist item commented out, the jail will shut down automatically when firefox is exited

A few gotchas for maintenance

The root is mounted readonly, so any pkg -j firefox upgrade operations (and similar) will require remounting it with readonly=off first To update/upgrade the base, do all that's required on the zroot/jails/basejail , make a snapshot, zfs destroy zroot/jails/${name}/root (first umount /var , /tmp and /home from it), and re-clone the base into a new root dataset, remount /var , /tmp and /home with zfs from the host. To update basejail: freebsd-update -b /zroot/jails/basejail fetch install

To upgrade basejail: freebsd-update -b /zroot/jails/basejail -r 11.2-RELEASE --currently-running 11.1-RELEASE upgrade , though perhaps just untar the base.txz for each upgrade, thatis create a new basejail.

To unmount datasets under jail's root : mount | grep ' on /zroot/jails/firefox/ | awk '{ print $3 }' | sort -r | xargs umount (replace firefox with whatever jail it is)

To re-mount datasets under jail's root : zfs list -o name | grep -E "^zroot/jails/firefox/" | xargs -n 1 zfs mount (replace firefox with whatever jail it is) The X unit socket will have to be re-mounted after reboot, ZFS datasets are mounted automatically. An exec.prestart could be added to the jail's config ( jail.conf ):

exec.prestart = "mount | grep ' on /zroot/jails/${name}/root/tmp/.X11-unix` || mount_nullfs /tmp/.X11-unix /zroot/jails/${name}/root/tmp/.X11-unix"

Integrating host-side browser launchers with jailed Firefox

Various launchers will want to launch local /usr/local/bin/firefox when an URL is cliecked or otherwise selected to be open in the default browser. To integrate this with jailed Firefox, one needs a wrapper script that will convert local Firefox calls to jexec calls. Three things are needed for this:

Local "firefox" binary that will be used as the wrapper A sudo-enabled NOPASSWD script that's called by the local "firefox" wrapper, and that executes jexec in the jail A sudoer rule allowing the script in #2 to be started with NOPASSWD

As a simple example, assuming local "firefox" will be called with first param being the URL, eg. /usr/local/bin/firefox http://google.com , we then set up (all on the host):

1 2 3 4 if [ -z " $1 " ]; then [ -z]; 5 exit 1 6 fi 7 sudo /home/user/bin/jail-firefox-exec $1 sudo /home/user/bin/jail-firefox-exec

1 2 3 4 if [ -z " $1 " ]; then [ -z]; 5 exit 1 6 fi 7 jexec -U firefox firefox firefox $1 jexec -U firefox firefox firefox

# In sudoers (called by sudo visudo), for example: %wheel ALL=(ALL) NOPASSWD: /home/user/bin/jail-firefox-exec