This is the problem…

This is where the conundrum began. By the nature of things, we don’t know the IP address of the containers beforehand, so there is no way we can use static DNS to route the requests. On the other hand we can’t use self signed certificates in the containers either, since this would require the end user to accept the certificate — a very undesirable side effect. As an additional constraint, the content of the container is visible to our end users, so we cannot put private keys of authority signed certificates in there.

Why is this so damn hard, all we want to do is to establish a secure websocket connection.

…and this is the problem.

We came up with the following solution. We use Nginx instances to proxy the requests and support upgrading to the wss protocol. This solves the first problem, it allows establishing a secure connections to containers without valid certificates. The remaining issue is that we want to essentially connect to random IPs. To solve this, we use the pattern matching of Nginx to parse the IP and forward the request to the correct target.

location /ipv4-regex {

proxy_pass http://$1:port/your/forward/location;

proxy_http_version 1.1;

proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection "Upgrade";

}

Now the only remaining part is to deploy this in a reproducible fashion. This is where Nixos shines. Combined with Nixops, this is almost trivial to do. Here is our deployment script.

{ wsproxy = { pkgs, ... }: {

networking.firewall.allowedTCPPorts = [ 80 443 ];

services.nginx = {

enable = true; recommendedGzipSettings = true;

recommendedOptimisation = true;

recommendedProxySettings = true;

recommendedTlsSettings = true; virtualHosts."your.host" = {

addSSL = true;

enableACME = true;

locations = {

"~ \"^/([\\d]{1,3}\\.[\\d]{1,3}\\.[\\d]{1,3}\\.[\\d]{1,3})$\"" = {

proxyPass = "http://$1:port/your/location";

proxyWebsockets = true;

};

};

};

};

};

}

These fifteen lines completely describe the system. Let me explain what is going on here.

wsproxy is the name of the server we are defining.

is the name of the server we are defining. networking.firewall.allowedTCPPorts opens the ports for HTTP and HTTPS in the the firewall (which by default is enabled on Nixos )

opens the ports for and in the the firewall (which by default is enabled on ) services.nginx.enable = true installs and enables an Nginx instance. The rest is the configuration of Nginx .

installs and enables an instance. The rest is the configuration of . recommended... set a bunch of recommended settings. For example, recommendedTlsSettings translate to the following config

ssl_session_cache shared:SSL:42m; ssl_session_timeout 23m; ssl_ecdh_curve secp384r1; ssl_prefer_server_ciphers on; ssl_stapling on;

ssl_stapling_verify on;

If you are interested in the other settings, take a look at the defining nginx.nix file.

virtualHost."websocket.example.com" defines a new virtual host in the Nginx configuation.

defines a new virtual host in the configuation. addSSL = true; enableACME = true; adds HTTPS to this virtual host and automatically requests and installs a certificate from Let’s Encrypt (how cool is that?). There are also options like forceSSL if you want to redirect all traffic through HTTPS.

adds to this virtual host and automatically requests and installs a certificate from (how cool is that?). There are also options like if you want to redirect all traffic through "~ \"^/([\\d]{1,3}\\.[\\d]{1,3}\\.[\\d]{1,3}\\.[\\d]{1,3})$\"" is a naive regular expression to parse IPv4 addresses. It is actually not the one we are using, but in order to keep it shorter I’ve included this slightly naive version. Note that certain symbols have to be escaped in the string (namely " and \ )

is a naive regular expression to parse IPv4 addresses. It is actually not the one we are using, but in order to keep it shorter I’ve included this slightly naive version. Note that certain symbols have to be escaped in the string (namely and ) proxyPass defines where the connection is proxied to.

defines where the connection is proxied to. proxyWebsockets turns on websocket proxying.

There are many more possible settings, check out the Nixos options (just put “nginx” into the search field).