blog | oilshell.org

Pratt Parsing Index and Updates

(Last updated 2020-04-18)

I've noticed that my November posts on expression parsing with Pratt's algorithm are popular and still drawing readers.

So here's an index of them, including yesterday's update, with related links.

Original Series

(1) Pratt Parsing and Precedence Climbing Are the Same Algorithm

After parsing arithmetic in OSH, I noticed that the precedence climbing algorithm is a special case of the earlier Pratt parsing algorithm. But see the update below.

Parsing Expressions by Recursive Descent (1999 an on) by Theodore Norvell: This article coined the term precedence climbing, citing a 1986 article by Clarke.

(2) Review of Pratt/TDOP Parsing Tutorials

I reviewed four articles, in order to motivate a different code structure.

(3) Pratt Parsing Without Prototypal Inheritance, Global Variables, Virtual Dispatch, or Java

Crockford deserves credit for reviving Pratt's algorithm, but I think his idiosyncratic style leaked into almost every future exposition of it.

In this article, I explain the code in my pratt-parsing-demo repo.

I prefer to use a pair of lookup tables for precedence, rather than token objects with dynamic dispatch. I prefer to enclose the entire parser in an class, with the current token as a member variable, rather than as a global variable.

Why? I tend to use classes somewhat formally, either for dependency inversion or for maintaining invariants on state. If everything's a class, then nothing is.

(4) Pratt Parsers Can Be Statically Typed

A minor update to correct a misleading notion.

2017 Updates

(5) Precedence Climbing is Widely Used

In November, I recommended dropping the "precedence climbing" name to avoid confusion. After looking at real code, I've changed my view.

From Precedence Climbing to Pratt Parsing (2016). I contacted Theodore Norvell after publishing my articles, and he agreed with my claims. Later he published this follow-up article, which treats Pratt parsing as an extension of precedence climbing.

(6) Code for the Shunting Yard Algorithm, and More

An encyclopedic update from Jean-Marc Bourguet.

2019 Update

The post How Desmos Uses Pratt Parsers may be useful if you prefer to read code in TypeScript. See desmosinc/pratt-parser-blog-code.

They also compare Pratt Parsing to using Jison, a LR(1) parser generator like Yacc or GNU Bison.

2020

Simple But Powerful Pratt Parsing. An introduction to the algorithm, with code in Rust. (reddit comments)

The post says that the algorithm is both recursive and iterative, which is a good point.

Here's a related observation that may also help. You can divide recursive algorithms into two categories:

Algorithms where the recursive function is also pure. For example, counting the nodes in a tree is a one-line function that has no variables:

function count(node) { return 1 + count(node.left) + count(node.right) }