A Unix Shell in Ruby

Published on February 16, 2012 by Jesse Storimer

This is the first article in a series where I'll implement a Unix shell in pure Ruby code.

Why a shell?

A shell is the quintessential example of a Unix program. It hits all of the interesting points that a Unix system is capable of.

It has to read input from STDIN, spawn processes requested from users, send signals to processes based on input, connect processes together with pipes, and provide a scripting environment for programmers to get work done.

What's a shell?

It's possible that you're using a shell all the time without knowing where the shell begins and where it ends. Or the fact that there are many different shells available to you. The default shell on most Unix systems is called bash . bash was written in the eighties as a replacement for its predecessor: the Bourne shell. Hence, bash stands for 'Bourne Again SHell'.

bash is a program like any other. It gets treated the same way as would grep , cat , or tail . So what makes a shell different? When you launch a shell it enters an endless loop of reading input and processing it. It reads the commands you type and launches programs based on that.

You can see this firsthand if you open a command line and try typing bash . Chances are you are already in an instance of a bash shell, but since it's a program like any other you can launch it again. Now you're in a shell in a shell. Inception!? Go ahead and exit the 'subshell' to get back to your original bash session.

Prior Art

There are many shells available for Unix systems. Besides bash , the most popular alternative is zsh . There are lots of good reasons to give zsh a try, enumerated here.

The Plan

We'll start out building a shell that resembles bash for familiarity. Along the way we might choose to make design decisions that deviate from what bash does, but that's the fun of writing your own! You can do it your way.

shirt

Our shell will be called shirt . This stands for 'SHell in Ruby? Totally!'. Here's our first implementation:

#!/usr/bin/env ruby $stdin . each_line do | line | pid = fork { exec line } Process . wait pid end

Stick that in a file, make it executable with chmod +x shirt and then run it using ./shirt . You'll be presented with a blank line, but try typing in a shell command like ls or ruby -v . This shell is still missing key features but the absolute basics are working.

So what's going on here?

Taking the Hard Way

The truth is that we could have implemented this in a much simpler way. We could replace the whole fork , exec , and wait bit with one call to system() . We're goint to avoid this method for two reasons:

We're here to learn, so we're going to use the lowest level building blocks available to us to get an idea of what's really possible here. This implementation is closer to bash or other system shells. This is more flexible. We won't get to it much in this article but doing it this way will actually provide us with more flexibility later when we want to implement things like pipes.

I'll explain things one piece at a time.

Getting Input

$stdin . each_line do | line |

This line will put the program in an endless loop. It will read from standard input until it gets a newline and then pass that line of text into the block.

If this is your first introduction to $stdin it's fairly simple. Every process has a $stdin file hooked up by default. This is where processes look to read input. This input might be coming from a keyboard or from a pipe, either can be read just the same. In this case it causes the program to block until it gets an entire line of input from the user.

Executing a Command

pid = fork { exec line }

There are two important concepts here: fork and exec .

Calling fork creates an new process that's an exact copy of the current process. It gets a copy of everything in memory and all open file descriptors. Certain housekeeping data is copied over like process group id, session group id, etc., and the new process gets its own process id, but for all intents and purposes it's an exact copy. So fork gets us a new process where we will execute the command that came in from $stdin .

Notice that we pass a block to fork and store its return value. The block is passed to the newly created process. It runs that block and then exits. In the parent process the process id (pid) of the newly created process is returned from fork . More on that pid in the next section.

Calling exec transforms the current process into another one specified by the file passed as input. An example should clear this up, calling exec('ls') will transform the current Ruby process into an ls process. The transformation happens immediately so any code that you have after the exec will never be reached. exec is a no-looking-back kind of thing.

Try out exec('ls') in irb if you never have before. Notice that you get the output of ls and then you're back to the terminal. The irb process is gone, replaced entirely by ls . So when ls exited you're back to the shell.

The pattern we're using here, fork and exec is a common way to spawn commands. It's the method used by Ruby constructs like Kernel#system and Kernel#` . If we were to skip the fork and just call exec then the shirt process itself would be transformed and it would no longer be able to respond to input. Calling fork creates a new child process to be transformed into the necessary command.

Waiting Patiently

Process . wait pid

The last piece of the puzzle is Process.wait . Calling fork will return immediately. It creates a child process then returns context to the parent process. If we didn't wait for the child process to finish its work then we'd end up spawning multiple processes in parallel. In that case their input and output would be interleaved all over each other and it would be unclear which output is coming from which command. This keeps things usable and gives us a synchronous shell.

Process.wait is a blocking call that won't return until the process represented by the passed-in pid has exited. Using this ensures that our shell will only run one command at a time.

Some Chrome

Before we end this first article let's give our shell a bit of chrome so we can recognize the prompt.

#!/usr/bin/env ruby $stdout . print '-> ' $stdin . each_line do | line | pid = fork { exec line } Process . wait pid $stdout . print '-> ' end

Now when you run the shell you get a simple prompt before each command so you can tell which lines are output and which lines are input.

Just the Start

There's still lots of ways to break the shell. Did you try using cd ? How about going back in history?

There's still lots of missing features, and since the shell is actually running inside another instance of a shell it's getting some features for free.

Next up, in part 2 we launch shirt outside the context of another shell and implement some built-in commands.

Grab the source on Github.

For a more thorough, Ruby-centric introduction to Unix programming basics, check out Working With Unix Processes.