Story

Fast forward again from my days as an intern to this year and a few things have changed. Almost all of our development at Cascade has transitioned to Node.js, all of our new development uses Docker, and our system is built around distributed microservices. These changes meant that we no longer had a comfortable debugging environment for our applications, which led to heavy use of console.log.

It wasn’t that we liked debugging using console.log but we were developing at such a rapid pace in a fairly new environment that we fell into the same trap that I fell into in my freshman year of college:

Figuring out a debugging environment will take too much time.

After a few months of this, we decided to figure out how to remote debug our Node applications running in Docker containers.

We had a few things going for us: node’s built-in debug flag is extremely useful and our IDE of choice, PHPStorm, has solid remote debug capabilities.

But we still had a number of challenges:

Connecting a remote debugger to a node application inside a container Updating code inside a container without re-building the container for every change Finding and Creating a debugging workflow that felt easy rather than burdensome

Connecting a Remote Debugger to a Node application inside a Container

This was the most straightforward of the challenges we faced. I decided the easiest way to do this was to have the remote debugger think it was connecting directly to the remote machine and ignore the fact that there was a container. This necessitated a small change to our Dockerfile and a change to the command used to run the container.

Dockerfile

FROM node:6 ... EXPOSE <Application-Port>

# Expose node debug port

EXPOSE 5858 ...



Old Docker run command

$ docker run -d -p <application-port>:<application-port> image

New Docker run command

$ docker run -d -p <app-port>:<app-port> -p 5858:5858 image

The changes in the Dockerfile and the run command meant that port 5858 inside the container is mapped to port 5858 on the host machine. This allowed us to specify the host’s public ip address (or DNS entry) as our remote host for debugging in PHPStorm.

Updating Code in a Container without Re-Building the Container Constantly

The changes above only succeeded in providing a way to remote-debug our applications, but did not address the more subtle issue of creating a manageable workflow. We needed a way to easily fit debugging into our development process.

Developing using Docker has several advantages, but one small drawback is a small update to your code requires a full rebuild of the container. This is annoying during development and debugging alike. We first noticed the issue during our review life-cycle, where we would review code change as well as a running copy of the updated code. As review comments and changes were addressed, the container holding the code had to be manually rebuilt each time so the code and running example remained in sync. This was a poor workflow. The solution: Docker volumes and Nodemon.

Nodemon is a tool that watches for file changes and restarts a specified node process when it sees a change. Docker Volumes allows you to map a host directory or file to a location inside a container. The combination of these two tools would allow us to restart the node process inside the running container every time code changes were pushed to the remote host rather than rebuilding the entire container.

The changes required involved edits to both the Dockerfile and the Docker run command:

Dockerfile

FROM node:6 ... RUN npm install -g nodemon ... # Old way

# ENTRYPOINT node <application-entrypoint.js> # New way

CMD ["node", "<application-entrypoint.js>"]

First, we installed Nodemon in the container, and more subtly we switched from using ENTRYPOINT to CMD. The difference between the two is that CMD is a default that can be overridden at Docker run time. This change meant that when we deployed the application there was no change to our build or run process as the default behavior was identical to what it was previously.

Docker Run Command

$ docker run \

-d \

-p 5858:5858 \

... \

-v <location of code on host>:<location of code in container>

image-name \

nodemon --debug <application-entrypoint.js>

The two lines in bold are the changes we made. First, we mounted the code directory on the host to the code directory in the container as a volume. Next, we overrode the default command to use Nodemon and to use the debug flag built into Node.js.

At this point we could:

Set breakpoints and debug our code

Upload code changes to remote host, auto triggering a process restart inside the container

We could have stopped here, but something wasn’t quite perfect yet: The built in debugger in PHPStorm was pretty slow. This wasn’t a big deal, but we decided to spend a few more hours and find a better solution.

Improving Debug Workflow

I could sum up this section with one tool: Visual Studio Code. VSCode has a built in debugger that was easier to set up and much faster than PHPStorm’s. Although VSCode is marketed as a text editor, I find that it strikes an almost perfect balance between useful features and being lightweight and unbiased. The debug workflow was awesome in VSCode, but…

There is one feature that is not included in VSCode that is critical to our daily development, auto FTP. We move files between our local machines and remote development machines so often that the lack of this feature completely eliminated an otherwise perfect tool.

I still go to VSCode if I have a complex bug to hunt down, but I can’t give up my IDE (yet).

To conclude this section, VSCode is awesome and with one more feature, I would gladly make it the only editor on my machine, but that one feature is too mission critical for daily development at Cascade.

Conclusion

We developed our own workflow/methodology to remote debug/develop node applications running in docker containers. This has greatly improved our bug hunting abilities as well as general development/review processes. The process is not perfect, but it is much improved from where we were a few months ago with console.log statements and mounting frustration. We are slowly integrating these new tools and processes into our daily development, and are excited to find other ways to improve our lives as developers in a new and cutting edge ecosystem.

P.S

If you found this post interesting and would like to work with a small team using cutting edge technology to monitor energy consumption, feel free to submit an application or shoot over an e-mail with any questions. Thanks!

—

Nicholas Warlen

Software Engineer @ Cascade Energy Inc.

nick.warlen@cascadeenergy.com

https://github.com/CascadeEnergy

https://github.com/nwarlen