blog | oilshell.org

Four More Posts in "Shell: The Good Parts"

(Last updated 2020-03-08)

This is an outline of the most important series in this blog. As I wrote on Hacker News, I started this project nearly 4 years ago with a plan:

Explain the problems with shell, motivating the need for a new one. I tagged these posts #shell-the-bad-parts and #why-a-new-shell. Write a new but compatible shell called Oil. Explain the good parts of shell with #shell-the-good-parts. Circulate these posts and let people try the good parts without the bad parts.

Unfortunately there are only two posts in that series, written over 3 years ago. Step 2 is taking longer than expected!

So while I'm chipping away at Oil's implementation, here are the main ideas. These ideas should be useful for experienced shell users, and I hope to explain them more carefully to non-shell users later in 2020.

Thanks to reader enriquto for reminding me to write this.

Each section below describes an article I'd like to write. The heading is the proposed title, and the body has the main ideas and important URLs.

Semi-automation With run.sh Scripts

I used to call these scripts do.sh , but that name shares a prefix with the common doc/ directory, creating a speed bump for autocompletion.

2/26 update: I'm not the only one who advocates this pattern:

The $0 -Dispatch Pattern Solves Three Important Problems

Related points:

Benchmarks are naturally shell scripts because you're gluing together heterogeneous programs. For example, I compare Oil vs. other shells, or an old version vs. a new version of Oil.

If you're not using more than one test framework, you're probably not testing enough. Oil uses many kinds of tests, and the results are published with each release.

Web Sites Are Naturally Made With Shell Scripts

An HTTP daemon is such a simple thing that a simple shell script will often suffice. — Tim Berners-Lee, inventor of the World Wide Web

http://info.cern.ch/hypertext/WWW/Provider/Overview.html: This page shows that the early web was "just" a shim over heterogeneous collections of files.

http://info.cern.ch/hypertext/WWW/Provider/ShellScript.html: HTTP 1.0 was trivial to process with /bin/sh . Berners-Lee parses the GET verb and a URL with the shell's read builtin, and cat s the file back to the network. (This practice ignores escaping, which I wrote about in 2017.)

The Unix philosophy of combining heterogeneous tools remains useful and powerful today. Building sites with tools rather than libraries leads to efficient, flexible, and loosely coupled code.

I built this website with classic Unix tools and my own tools written in Python. The main() is always in shell, not Python.

Some direct analogies between HTML and shell:

HTML is to CSS/JavaScript/WebAssembly as Shell is to sed/awk/make.

HTML files and shell scripts both talk about hierarchical paths. The Web adds the crucial concept of a hyperlink .

. Data URLs are like here docs.

The Alternative Shell Challenge

I've noticed that some alternative shells on the wiki don't have all the good parts of shell. This is often because they don't directly control the file descriptor table of a process, which is done with low-level system calls like dup2() and fcntl() .

That is, they aren't Unix shells in the classic sense.

A couple years ago, I wrote a challenge for alternative shells (and Perl). How do you express this simple program?

f () { echo --- ls / echo --- } f > out.txt f | wc -l

It's very related to the Git Log in HTML post, which adds proper escaping.

print-html-listing () { local dir = " $1 " echo '<html>' echo ' <body>' ls " $dir " | escape-html echo ' </body>' echo '</html>' } print-html-listing > listing.txt

I recently learned that Elvish does well on this challenge with a "virtual" file descriptor table. However, it doesn't have the POSIX semantics of "copying" the shell's memory with fork() .

I'd like to do a survey of other shells. Let me know if you'd like to help test an alternative shell or scripting language.

What's Next?

This post is two levels deep on the "blog stack". After releasing Oil 0.8.pre1, I drafted an announcement:

Oil 0.8.pre1: Contributions and Metrics

But it needed context, so I started writing:

February Brain Dump: with links to Hacker News comments and Zulip threads.

This post summarized just one comment, mentioned in the first paragraph. The next two posts will pop the stack, summarizing more comments and describing the latest release.