Yesterday, a friend e-mailed me to get my thoughts on managing codebases in the context of modules. In particular, he was asking about when something should be a module and when something should be a versionable module (or a different codebase) and what I thought the best practices were. My response turned out to be long, so I’m publishing it here in hopes that it benefits others.

I focus the discussion on JavaScript and CommonJS modules, but these are only to provide concrete examples — the concepts and concerns should be applicable across many popular languages.

Logical separation of code is very important. I don't know any experienced engineer who wouldn't agree that encapsulation is an important tool in writing software. Encapsulation is often associated with the concept of classes, which encapsulate functions, but it can be generalized to describe functions (as encapsulating logic) and modules (as encapsulating classes).

Most languages provide constructs for encapsulation. In Java you have archives composed of packages composed of classes composed of methods composed of logic. Each level is a different way to encapsulate stuff.

JavaScript has weaker constructs to assist in encapsulation. Many of the constructs and patterns we use in our applications today were invented by the community. Back in 2000, there were functions to encapsulate shared logic and that was enough because there were only a handful of functions necessary for a given client-side “application” (a web page). Having them all share the same global scope wasn't a big problem.

Now that we build applications with thousands of functions, we need to use higher-level abstractions to encapsulate functions. For CommonJS modules (the ones that Node and browserify use), these are files which you follow a certain structure so that you can ‘require()’ them into other CommonJS modules. To draw an analogy, the variables you ‘export’ in CommonJS modules are like ‘static public’ functions in a Java class. So, CommonJS modules allow us to encapsulate functions. Then, with Bower and npm, we can encapsulate modules as versionable artifacts and share them (in Java this would be like a .jar file).

Now that we have the concept of encapsulation, let consider the various levels of it.

No Encapsulation, Just Operations

First let’s consider the basic building block of applications: simple operations.

var squared = 100 * 100;

This isn't really encapsulating anything, but it is our fundamental building block. We're just saying that the variable “squared” should be the result of “100 multiplied by 100". I guess in some sense we could say that squared encapsulates the data “100 * 100", but I won't dwell on that.

You can’t share this method of squaring, because there is no “thing” to share. If you wanted to share this way of squaring things you could copy/paste the same pattern across your application. However, let’s say that later on you realize that you should log an error message if the number being squared would result in a number higher than the largest number your environment can represent (in JavaScript, Number.MAX_VALUE). We would search for all lines of code which look like “<number> * <number>” and add the appropriate error reporting logic to every place we find it. I'm not going to go into why managing duplicated code like this is bad, because I think the reasons are obvious.

Encapsulating Operations with Functions

If we moved the implementation of how numbers are squared to a shared location which could be referenced across our application, then we can make changes to the implementation without impacting all the places we need a number squared.

var squared = square(100);

The benefit here is that we have decoupled the concerns of this line of code from how things get squared. In the first example we had to describe how to square something, in the second example we describe what the end result we want is, but don't dictate how. This means we can now change how we square something without having to change this line of code. The reason for using functions instead of duplicating logic is similar to why you wouldn’t plant wheat whenever you wanted to bread. What you want is bread, you don’t care about how it’s made, and you certainly don't want to make it yourself.

I know this is basic stuff, I promise I'm building up to my point.

In the above example we basically say “I want the result of the function ‘square’ when I give 100 to it”. Since we don't define where to find the ‘square’ function the computer picks the best implied option: it checks the current scope, then the parent of that scope, and so on, all the way to the global scope of the environment. This means that if you want two pieces of code to share the same ‘square’ implementation, they need to have a scope “ancestor” in common. The closer to global scope something is, the easier it is to share an implementation with other code because all code shared the global environment scope.

So, with just functions, we're left with a strong incentive to share code in the global scope. Unfortunately, it’s usually not just your code using the global scope, it’s everyone’s. Without cooperation, you run the risk of having others code overwrite your ‘square’ implementation with theirs which may do something unexpected. Global scopes require great collaboration, which is not the case in web applications which often uses code written by hundreds of people across different projects.

Modules as Addresses

If we were explicit about which ‘square’ function we needed, there would no longer be a need for an implied search, no ambiguity, and no motivation to put things in a global scope. This is part of the reason why the AMD and CommonJS specifications were created.

var math = require(‘./math.js’);

var x = math.square(100);

With the CommonJS module pattern, as in the example above, you can have a “math.js” module which exports a “square” function. Since you define a physical location in the filesystem (‘./math.js’) there is no ambiguity of where to look for the ‘square’ function.

The module acts as a scope for the functions (and other things) it exports and can be injected to any other scope. This frees us from sharing things via a global scope. When you use a function from a module that you've loaded you're explicitly saying “I want to use this function which lives at this specific location in the file system”.

A Time and A Place: Version Control

With the modules concept we have an unambiguous way to address the location of the code we want to reuse. There is a missing piece here: time. Code tends to change as time progresses — new features are added, old ones removes, APIs change, expectations change, and so on…

Addressing a module is explicit in location but not in time. When “require()”-ing a module in CommonJS, we effectively say “load the module as it is now”, where “now” is the time when the module is imported during runtime (or during build time if using something like Browserify).

You cannot refer to local modules within one codebase across time (you can’t say “load ./math.js as it was on January 14, 2015"). This implicit assumption is OK when modules are used to break apart a service which will eventually run together and are managed by the same team of engineers. The modules within a codebase are bound together in time — they have to move through time in lockstep.

Let’s walk through an example of when this becomes problematic. Let’s say you have a code base (a git repository), which has three modules: Module A, Module B, and Module C. Both Module A and Module B depend on Module C. If Module C needs to be updated in a backwards-incompatible way to add a feature for Module B, then Module A must also be updated (otherwise it will break due to the backwards-incompatible change).

In an ideal world we don't have to do work we don't need to. So if we could lock Module A’s dependency on Module C to a specific state of Module C in time, then Module B and Module C can evolve without needing to modify Module A.

Versionable modules solve this issue by allowing you to address a module not by place but also by time (the “version”). If Module A, Module B, and Module C referred to each other by name + version, we could say Module A needs Module C at version 1, and Module B needs Module C and version 2. You can even continue making backwards incompatible changes to Module C and introduce version 3 without having any adverse impacts on Module A or B. Versionable modules decouple dependencies in time.

Tools like Bower and npm allow for this pattern of development. You “publish” a version of a CommonJS module on npm so that others can install it. The version is an immutable state of the module in time, which guarantees that if it works today, it will work the same tomorrow.

Practicality

So far we've maintained a fairly idealistic view of the world. In my experience as you move to the higher abstractions the costs of managing the code increase significantly.

Writing operations “x = 100 * 100" is the simplest thing you can do (and you must do it to write software).

Writing functions is also super simple because it’s a language feature in JavaScript (just add “function(){}” around some operations).

Writing modules is easy if the environment supports it — it’s trivial with Node.js applications because the Node environment understands CommonJS out of the box, but browsers don't. To use CommonJS in browser applications you have to use Browserify or a similar tool to build your code before running it on a browser. Also, in CommonJS, modules are files, so if you're not particularly good with using an IDE, you have to juggle jumping from file to file to get your work done.

Writing versionable modules is not easy. It’s orders of magnitude more difficult to write and maintain a versionable module than just write operations or functions. You usually need to set up a code repository, learn various specifications (e.g., semver, package.json, bower.json), maybe register with a repository, maybe run a publish job whenever you're ready to release a version. If you need to develop two modules in parallel you have to learn and manage package linking. These costs are non-trivial, which is a strong case for why not all modules should be versionable if you have time constraints (which we all do).

The Conclusion

All of these abstractions come at a cost. I think it’s generally accepted that bundling operations as functions is worth the overhead (especially compared to the alternative of copy/pasting). Modules are cheap and useful so long as your environment supports it or you have a sufficiently large browser application where there may be ambiguity in addressing functions. Versionable modules are not cheap, but useful for managing dependencies between teams of engineers because it decouples their development in time.

When considering whether a module should be versionable, consider your teams experience and comfort level with working with these systems. Then consider how likely you are to independently develop these versioned modules (because that’s a primary benefit of having them versioned).

In general, I think it’s preferable to make modules versionable, I’m pro-many-small-modules. It encourages good practices and makes it difficult to maintain bad practices (you're less likely to code against the implementation of a remote module then you are to a sibling module or function). However, I don't recommend them for everyone because I think they're difficult to create and maintain efficiently. I've had lots of free time to build up my tooling and knowledge to manage them, that’s not true for everyone, especially for teams with many junior members. There are significant pitfalls to managing multiple versionable modules within one engineering team (e.g., synchronizing who is working on which version and linking them together). So, like most things in life your decision should depend on the specific situation you're in.

If you have any questions or comments, please send them to ozanturgut@gmail.com or tweet to @ozanturgut. My many small modules live on GitHub.