Pragmatic reasons for choosing Common Lisp.

It seems like every other Common Lisp post on Hacker News devolves into a debate on why anybody would want to use the language in these modern times.

I switched to web development quite recently after many years of using C on embedded processors. The first project I had in mind was a hybrid web / client-side application. It was obvious that C was not the best choice for the project.

I set out to find a high-level language with some specific application deliverability properties. Ruby, Python, Scheme and Common Lisp were the languages I investigated in most detail. From these only Lisp met my requirements.

After three years of working in Common Lisp I still don’t use many of its renowned features. Things like complex macros, DSLs and CLOS. Even without using that power development progressed many times faster than it would using C.

Given that many of Common Lisp’s defining features and advantages are also available in other languages, who would choose Lisp over any of the more mainstream options? Someone who needs to write code that is portable across operating systems and competing implementations in a high-level, compiled language that generates standalone executable binaries with execution speed comparable to C.

Some background

The project I wanted to do is a web service that involves encryption and security amongst other things. It consists of both server-side and client-side applications. The server-side runs on Linux while the client-side runs as a standalone executable on Linux, Windows or macOS. The project included plans for client-side applications on Android and iPhone.

I would be the sole programmer for an indeterminate time.

Requirements

I evaluated the different language options according to this list of requirements:

It must be a high-level language. Development must progress quickly and the application does not require bit-level control of the data representation or memory layout. The language must have automatic memory management. Parts of the application involve security. Automatic memory management avoids all the vulnerabilities associated with manual memory management like buffer overflows and invalid pointers. The ability to create native GUI applications is useful but not required. The client-side program is a language agnostic SDK. It is intended to be used as a tool by other applications. The client-side program must be a single, standalone executable binary or shared library. Multi-file distributions will complicate the installation procedure and increase support issues. It must be a compiled language. Prefer compilation to native code over byte code. Native code executes faster than byte code. It also provides some protection against reverse engineering. It must be a cross-platform language. The same code must compile on all platforms with the minimum amount of conditional directives and the compiler must be of equal quality on all the target platforms. There are three client OSes and only one programmer. It would be useful if the language can be used for Android and iPhone development. If the language can compile to the mobile operating systems a lot of the desktop client’s code can be re-used. The language must be mature and stable. There is enough uncertainty in the project, the extra uncertainty caused by using a bleeding edge language is not worth the improbable gains. A mature web development framework must be available. The project contains a significant part web development. The library ecosystem must be healthy. My focus is on the application, not on creating libraries.

Language evaluation

I selected Ruby, Python, Scheme and Common Lisp as the high-level languages to evaluate.

For Ruby and Python I only considered the official implementations. The evaluation results for both languages will be different if the alternative implementations are also considered but that will introduce many language specific issues that will need to be considered. It is a good idea to consider the other implementations once you have decided on the language, or when your requirements differ from those I listed, for example if you are looking for a high-level language that can run on a Java Virtual Machine.

Scheme refers to at least two language standards, each with multiple incompatible implementations. For this evaluation I looked at the tendency amongst a few of the well known implementations.

Common Lisp has a broad language specification and all complying implementations are compatible with each other. For the topics which are not covered by the specification I once again looked at the tendency.

Ruby Python Scheme Common Lisp High-level Yes Yes Yes Yes Managed memory Yes Yes Yes Yes Native GUI apps Difficult Easy Varies Difficult Executable binaries Source code bundle Involved process Yes Yes Compiled code Byte code Byte code Native or Byte code Native or Byte code Cross-platform Almost Yes Yes Yes Mobile options Yes Yes Yes Yes Mature language Moving target Moving target Yes Yes Web Dev Frameworks Yes Yes Yes Yes Library ecosystem Huge Huge Medium Medium

Notes:

Language selection

I disqualified Ruby and Python because of two issues. First is that building standalone executables is not a viable option. Second is that perfectly working code will eventually need major modifications just because the language standard has changed.

That left me to consider Scheme and Common Lisp. According to the tabled results Scheme is the better option because it has better GUI support than Lisp and they match everywhere else.

Scheme follows a minimalist design philosophy where the language specification only specifies a small core. Everything outside the core is left up to the implementor. This resulted in many incompatible implementations. It is non-trivial to switch a project between implementations because of these incompatibilities.

Common Lisp’s design philosophy is expansive and leaves very little for implementors to decide. This resulted in many implementations that are fully compatible with each other. Source code can generally be compiled without modification on any of the conforming implementations.

At the time when I had to make the decision I had no experience in either Scheme or Lisp. Common Lisp’s portability was a safety net. If I selected the wrong implementation it would be trivial to switch because the existing code could be used unmodified.

I decided to use Common Lisp and I am glad I did.