It's the sequel to … a programming language Buy the first part for $29 Writing A Compiler In Go is the sequel to Writing An Interpreter In Go. It starts right where the first one stopped, with a fully-working, fully-tested Monkey interpreter in hand, connecting both books seamlessly, ready to build a compiler and a virtual machine for Monkey. In this book, we use the codebase (included in the book!) from the first part and extend it. We take the lexer, the parser, the AST, the REPL and the object system and use them to build a new, faster implementation of Monkey, right next to the tree-walking evaluator we built in the first book. The approach is unchanged, too. Working, tested code is the focus, we build everything from scratch, do baby steps, write tests firsts, use no 3rd-party-libraries and see and understand how all the pieces fit together. It's a continuation in prose and in code. Do you need to read the first part before this one? If you're okay with treating the code from the first book as a black box, then no. But that's not what these books are about; they're about opening up black boxes, looking inside and shining a light. You'll have the best understanding of where we're going in this book, if you know where we started.

Learn how to write a compiler and a virtual machine Our main goal in in this book is to evolve Monkey. We change its architecture and turn it into a bytecode compiler and virtual machine. We'll take the lexer, the parser, the AST and the object system we wrote in the first book and use them to build our own Monkey compiler and virtual machine … from scratch! We'll build them side-by-side so that we'll always have a running system we can steadily evolve. Read a sample What we end up with is not only much closer to the programming languages we use every day, giving us a better understanding of how they work, but also 3x faster. And that's without explicitly aiming for performance. Here's what we'll do: We define our own bytecode instructions , specifying their operands and their encoding. Along the way, we also build a mini-disassembler for them.

, specifying their operands and their encoding. Along the way, we also build a for them. We write a compiler that takes in a Monkey AST and turns it into bytecode by emitting instructions

that takes in a Monkey AST and turns it into bytecode by emitting instructions At the same time we build a stack-based virtual machine that executes the bytecode in its main loop View table of contents We'll learn a lot about computers, how they work, what machine code and opcodes are, what the stack is and how to work with stack pointers and frame pointers, what it means to define a calling convention, and much more. We also build a symbol table and a constant pool

and a do stack arithmetic

generate jump instructions

build frames into our VM to execute functions with local bindings and arguments !

into our VM to execute ! add built-in functions to the VM

to the VM get real closures working in the virtual machine and learn why closure-compilation is so tricky

The Monkey Programming Language The implementation of Monkey we build in this book is markedly different from the tree-walking interpreter we built in Writing An Interpreter In Go, but Monkey stays the same. At the end, Monkey will still look and work like this: // Integers & arithmetic expressions let version = 1 + ( 50 / 2 ) - ( 8 * 3 ); // Strings let name = "The Monkey programming language" ; // Booleans let isMonkeyFastNow = true ; // Arrays & Hashes let people = [{ "name" : "Anna" , "age" : 24 }, { "name" : "Bob" , "age" : 99 }]; // Functions let getName = fn ( person ) { person [ "name" ]; }; getName ( people [ 0 ]); // => "Anna" getName ( people [ 1 ]); // => "Bob" And it will still support recursive functions, conditionals, implicit and explicit returning of values: let fibonacci = fn ( x ) { if ( x == 0 ) { 0 } else { if ( x == 1 ) { return 1 ; } else { fibonacci ( x - 1 ) + fibonacci ( x - 2 ); } } }; The crown-jewel we'll build into our new Monkey implementation are closures: // `newAdder` returns a closure that makes use of the free variables `a` and `b`: let newAdder = fn ( a , b ) { fn ( c ) { a + b + c }; }; // This constructs a new `adder` function: let adder = newAdder ( 1 , 2 ); adder ( 8 ); // => 11 Yes, we'll compile all of that to bytecode and execute it in a stack-based virtual machine. That not only works, but is also tremendous fun to build.

What readers are saying about this book: "This is a very polished pair of books that together give a steady path to follow for learning some of the real techniques that are used to implement programming languages. They're both well above average for their depth, technical clarity, and accessibility. I've been recommending them to everyone I work with who wants to get involved in compilers." Chris Seaton (@ChrisGSeaton)

Researcher at Shopify, PhD in Ruby, Founder of Truffle Ruby "Thorsten has a wonderful gift — as you read his books, you'll feel as though you've discovered compilers, all on your own! As a self-taught engineer without a CS degree, I found myself led on a journey of learning, driven by the nuggets of 'why' that Thorsten shares alongside code. I've recommended these books to many, and I won't hesitate to continue. Compilers, interpreters and programming languages may seem esoteric, but if you look closely, they're everywhere. The Monkey language is my go-to project whenever I learn a new programming language. There's no better way to take a new language through its paces!" Lauren Tan (@sugarpirate_)

Software Engineer "Longing for implementing a compiler, but lacking the guidance? Then this book is for you. Thorsten offers invaluable mentorship for tackling the multiple challenges that arise when writing a single-pass compiler and a virtual machine to execute the resulting code, in a pragmatic and test-driven approach. After going over this book, you'll certainly feel inspired to go further, such as extending the compiler with code transformations of your own." Ricardo Nabinger Sanchez (@rnsanchez)

Performance Engineer at Taghos Tecnologia "Making is understanding. The book carefully teaches us how VM-backed interpreters work, while building our own compiler and virtual machine. Written in a TDD style, a step-by-step manner and with a humane tone. I've enjoyed the wonderful journey!" Yoji Shidara (@darashi)

Programmer, English/Japanese Translator of Writing An Interpreter In Go "I think that it's one of the best self-published books on the market I've ever read, not only in Go area, but whole technical market." Bart Smykla (@bartsmykla)

Open Source engineer at VMWare "Finally, a compiler book that you don't have to fight each and every page to get through! Other books burden the reader with so many details and subtleties that you'll never make progress. But if you want to make an actual compiler, read this. I'll definitely be recommending this to my students." Austin Henley (@austinzhenley)

Assistant Professor, Department of Electrical Engineering & Computer Science, University of Tennessee "If you are scared of Dragons but want to understand how compilers work, this book is for you. Not only does it make a daunting subject approachable but it also make it fun and enjoyable." Franck JEANNIN (@frje)

Founder DataNuage "The Monkey books are everything I've ever wanted in programming books, but could never find until now. They are a real labor of love and it shows on every page. The books are fun to read and showcase the Go programming language, but what makes them extraordinary is the fact that they're completely focused on giving the reader an accessible, conceptual understanding of the implementation of interpreters and compilers. It's hard to overstate just how rare yet extremely valuable that focus on understanding is in an educational book." Brandon Ramos

Product Owner/Developer "Just awesome books. If we had books like this on other subjects (search engines, DB systems), I think developers would have less fear of contributing to large open source projects. - Thank you @thorstenball. This is already paying off in my DB internals course." Evance Soumaoro (@evanxg852000)

Software Engineer | System Programming Enthusiast