Thank you for your comment

 Why language-oriented programming? Why Racket?

 This is a sequel to my earlier piece Why Racket? Why Lisp?, which I wrote a year or so after I discov­ered Racket. As someone new to Lisp languages, I had trudged through bogs of Lisp flat­tery, never sure what to make of the often hand­wavy claims. For instance, that Lisp even­tu­ally induces “profound enlight­en­ment”. Sure—what­ever you say, bro.

 I had a simpler ques­tion: what’s in it for me, now? My earlier piece tried to answer that ques­tion. I summa­rized why someone who hadn’t looked into a Lisp language—or Racket in partic­ular—might want to.

 I included a list of the nine language features that were most valu­able to me as a Racket noob. Feature #5 was “create new program­ming languages”. This tech­nique also goes by the name language-oriented program­ming, or LOP.

 In the years since, language-oriented program­ming has risen to become my favorite part of Racket. Along the way, I converted my enthu­siasm into action. In addi­tion to making languages, I wrote this online book—Beau­tiful Racket—that teaches LOP as a tech­nique, and Racket as a tool.

 In my own work, one example is Pollen, a text-based language I wrote to make my online books Prac­tical Typog­raphy and Beau­tiful Racket possible. + Pollen is based on Racket’s docu­men­ta­tion language Scribble, which handles most of the heavy lifting. In Pollen, the para­graph above is programmed like so:

 #lang pollen



I included a list of the ◊link["why-racket-why-lisp.html#so-really-whats-in-it-for-me-now"]{nine language features} that were most valuable to me as a Racket noob. Feature #5 was "create new programming languages". This technique also goes by the name ◊em{language-oriented programming}, or ◊em{LOP}. 1 2 3 #lang pollen I included a list of the ◊link["why-racket-why-lisp.html#so-really-whats-in-it-for-me-now"]{nine language features} that were most valuable to me as a Racket noob. Feature #5 was "create new programming languages". This technique also goes by the name ◊em{language-oriented programming}, or ◊em{LOP}.

 Another example is brag, a parser gener­ator (in the lex/yacc style) that takes a BNF grammar as its source code. A simple example for the bf language:

 #lang brag



bf-program : (bf-op | bf-loop)*

bf-op : ">" | "<" | "+" | "-" | "." | ","

bf-loop : "[" (bf-op | bf-loop)* "]" 1 2 3 4 5 #lang brag bf-program : (bf-op | bf-loop)* bf-op : ">" | "<" | "+" | "-" | "." | "," bf-loop : "[" (bf-op | bf-loop)* "]"

 Both these languages are imple­mented in Racket, and can be run with the usual Racket inter­preter, or inside the Racket IDE (called Dr­Racket).

 And yet. Though this book has started thou­sands of people on their LOP journey, I some­times fear that I’ve fallen into the same quick­sand as the Lisp advo­cates I once crit­i­cized.

 If LOP is so great, then you shouldn’t need to spend a few days working through the tuto­rials in this book. Right? I should be able to explain it concisely, with minimum hand­waving. I should be able to answer two simple ques­tions:

 What prob­lems are best suited to language-oriented program­ming?  Why is Racket best suited for the task of making languages?

 The second ques­tion is easy. The first ques­tion isn’t. I’ve been asked it many times. I’ve often resorted to the answer made famous by Justice Potter Stewart: you’ll know it when you see it. That answer is good enough for those already LOP-curious. But not for those still on the side­lines who want more of a prac­tical justi­fi­ca­tion.

 So here’s a better attempt. Bear in mind that I’m not a computer-science professor. I’m not going to be talking about program­ming languages the way they might. Rather, I use Racket and DSLs for prac­tical purposes—my daily work depends on them. My goal is to frame the issue in a way that will be useful for other prac­tical users. (If I haven’t, you’re welcome to click in the left margin and send me a comment.)

 Language-oriented program­ming is really an inter­face-design tech­nique. It’s unbeat­able for tasks that demand minimum nota­tion while preserving maximum preci­sion. Minimum nota­tion = the only nota­tion is what you permit. There is nothing extra­neous. Maximum preci­sion = the meaning of that nota­tion is exactly what you say. There is no scaf­folding or boil­er­plate. LOP gets to the point like nothing else.  (The impa­tient can jump ahead to these cate­gories of tasks that are likely to benefit from LOP.)  Racket is ideal for LOP because of its macro system. Macros are indis­pens­able for making languages because they make compiler-style code trans­for­ma­tions easy. Racket’s macro system is better than any other.

 About half the readers of this piece are now departing to post anony­mous internet comments disputing the above claims. Before you leave, please know: I’m hedged either way. LOP and Racket have been an incred­ible force multi­plier on my program­ming produc­tivity. I’m happy to share this knowl­edge with you, so that you too may benefit. But I’m equally happy for these tools to remain my secret weapons, so I can keep producing work that is more ambi­tious, more impres­sive, and more prof­itable than the other 99.9%.

 The choice, however, is yours.

 I finally started to unravel the Big Ques­tions by consid­ering a metaque­s­tion: why does it seem hard to explain the bene­fits of language-oriented program­ming?

 Perhaps because when we speak of program­ming languages—or just languages, short­hand that I’ll use inter­change­ably—the term is freighted with expec­ta­tions about what a language is and what it does. As long as we’re standing inside that box, it’s more diffi­cult to see the value of language-oriented program­ming.

 But if we zoom out and look at languages as part of the larger cate­gory of human–computer inter­faces, then it becomes easier to see the special bene­fits of LOP.

 So let’s do that.

 First, some termi­nology. Language-oriented program­ming (aka LOP) is the idea of solving a program­ming problem by making a new program­ming language, and then writing a program with the new language. Often, these “little languages” are known as domain-specific languages or DSLs.

 As the name implies, a domain-specific language is one tailored to the needs of a partic­ular set of prob­lems. For instance, Post­Script, SQL, make, regular expres­sions, .htaccess, and HTML qualify as domain-specific languages. They don’t try to do every­thing. Rather, they focus on doing one thing well.

 At the other end of the language spec­trum are what we’ll call general-purpose languages. Out here we find the usual suspects—C, Pascal, Perl, Java, Python, Ruby, Racket, etc. Why aren’t these languages DSLs? Because they hold them­selves out as being suit­able for a wide range of computing tasks.

 In prac­tice, general-purpose languages often have certain jobs that they do better than others, depending on the condi­tions of their birth. For instance, C excels at systems program­ming. Perl excels as a sysadmin scripting language. Python excels as a language for begin­ners. Racket excels at LOP. In each case, because that’s what the language was designed to do.

 The distinc­tion between domain-specific and general-purpose languages is perme­able. For instance, Ruby started as a general-purpose language, but became popular largely as a web-appli­ca­tion DSL through its asso­ci­a­tion with Ruby on Rails. Going the other direc­tion, Java­Script was orig­i­nally a DSL for web-browser scripting. Like a mutating virus, it has since grown far beyond.

 If all these things on the spec­trum between domain-specific languages and general-purpose languages deserve to be called languages, then what are the defining features of a language?

 I know what you’re thinking: “Well, that’s where you’re wrong. HTML isn’t a language. It’s just markup. It can’t express algo­rithms.” Or maybe: “Regular expres­sions aren’t a language. They can’t stand on their own. They’re just syntax within another language.”

 I once felt that way too. But the closer I looked, the more these distinc­tions seemed vaporous. Thus, my first major claim (of three): a program­ming language is inher­ently a medium of exchange—a nota­tion system mutu­ally intel­li­gible to humans and computers.

 “Nota­tion” connotes that a language has syntax; “intel­li­gible” connotes that a language has meaning (or seman­tics, to use the fancier word) attached to the syntax. This defi­n­i­tion covers all general-purpose program­ming languages. And all DSLs. (But not every data stream—about which more later.)

 (By the way, even though “program­ming” and “language” are words idiomat­i­cally used together, these languages are not used strictly by humans to program computers. Some­times they’re used by computers to commu­ni­cate with us; + For instance: S-expres­sions. some­times with each other. + For instance: XML, JSON, HTTP. Defi­n­i­tion­ally, it seems wrong to exclude these possi­bil­i­ties. But in prac­tice, yes—what we’re usually doing with a program­ming language is, you know, program­ming.)

 Consider HTML. It’s a way of telling a computer—in partic­ular, a web browser—how to draw a web page. It’s a nota­tion system (= angle brackets, tags, attrib­utes, and so on) intel­li­gible to the human and computer (= the charset meta defines the char­acter encoding, p tag contains a para­graph, and so on).

 Here’s a little HTML page:

 <!DOCTYPE html>

<html>

<head>

<meta charset="UTF-8">

<title>My web page</title>

</head>

<body>

<p>Hello <strong>world</strong></p>

</body>

<html> 1 2 3 4 5 6 7 8 9 10 <!DOCTYPE html> <html> <head> <meta charset= "UTF-8" > <title> My web page </title> </head> <body> <p> Hello <strong> world </strong></p> </body> <html>

 Suppose you dispute that HTML is a program­ming language. Fine. We’ll output our page with Python. That’s a real program­ming language, right?

 print "<!DOCTYPE html>"

print "<html>"

print "<head>"

print "<meta charset=\"UTF-8\">"

print "<title>My web page</title>"

print "</head>"

print "<body>"

print "<p>Hello <strong>world</strong></p>"

print "</body>"

print "<html>" 1 2 3 4 5 6 7 8 9 10 print "<!DOCTYPE html>" print "<html>" print "<head>" print "<meta charset= \" UTF-8 \" >" print "<title>My web page</title>" print "</head>" print "<body>" print "<p>Hello <strong>world</strong></p>" print "</body>" print "<html>"

 If Python is a program­ming language and HTML is not, then it must be true that this Python sample is a program, and the HTML sample is not.

 Plainly, this distinc­tion is tortured. Here, the Pythoniza­tion adds nothing but complexity and boil­er­plate. More piquantly, the only inter­esting semantic content in the Python program—in terms of control­ling the web browser—is what’s embedded in the HTML. + Arguably, HTML tags—like DOCTYPE and meta and strong—are nothing more than func­tions that take argu­ments. My DSL Pollen builds on this corre­spon­dence. Consis­tency compels us to conclude that HTML, though simpler and less flex­ible than Python, is never­the­less a program­ming language too.

 Our HTML example was contrived. But this pattern—a DSL nested within another language—is perva­sive. Languages used in this way are called embedded languages. They repre­sent the most common form of language-oriented program­ming. As a programmer, you’ve relied on LOP for years, even if you didn’t know its name.

 For instance, regular expres­sions. + Other exam­ples: printf format­ting strings, CLDR date/time patterns, SQL. We may not think of regular-expres­sion nota­tion as an inde­pen­dent language. But every programmer knows what this means:

 ^fo+(bar)*$ 1 ^fo+(bar)*$

 More­over, you can prob­ably type this regular expres­sion into your favorite program­ming language and it will just work. This consis­tent behavior is only possible because regular-expres­sion nota­tion is an embedded language, exter­nally defined (by POSIX).

 As with HTML, we could write the equiv­a­lent string-matching compu­ta­tion in the nota­tion of the host language. For instance, Racket supports Scheme Regular Expres­sions (SREs), which are regular expres­sions that use S-expres­sion nota­tion. The above pattern would be written like so:

 (seq bos "f" (+ "o") (* (submatch "bar")) eos) 1 ( seq bos "f" ( + "o" ) ( * ( submatch "bar" )) eos )

 But Racket program­mers rarely use SREs. They’re too long and too hard to remember.

 Another ubiq­ui­tous example of an embedded DSL: math expres­sions. Every programmer knows what this means:

 (1 + 2) * (3 / 4) - 5 1 (1 + 2) * (3 / 4) - 5

 On their own, math expres­sions don’t make inter­esting programs. We need to combine them with other language constructs. But as with regular expres­sions, that’s an ergonomic and prac­tical consid­er­a­tion. Math expres­sions have their own nota­tion and meaning, intel­li­gible by both humans and computers, and thus qualify as a sepa­rate embedded language.

 Indeed I am. I’m claiming that HTML (and regular expres­sions and math expres­sions) qualify as rudi­men­tary program­ming languages. This neces­sarily implies that writing HTML (or regular expres­sions or math expres­sions) qual­i­fies as rudi­men­tary program­ming.

 Please—don’t panic. We can agree that someone who billed them­selves as a “programmer” on Linked­In while only knowing HTML and arith­metic would be consid­ered delu­sional. + And next week, will prob­ably have a $180K soft­ware-engi­neering job in Menlo Park. But that spills into a sepa­rate issue about what the desig­na­tion “programmer” typi­cally denotes in the job market. That’s not our concern here.

 If this defi­n­i­tion of program­ming languages still rankles, maybe it’s because you think a real program­ming language needs to be capable of expressing all possible algo­rithms—that is, it must be Turing complete.

 I see why that might be part of the intu­ition. Every general-purpose program­ming language is Turing complete.

 The problem? Turing complete­ness is a low bar. It’s a tech­nical measure­ment that doesn’t tell us anything inter­esting about the language in the real world. + “Beware of the Turing tarpit in which every­thing is possible but nothing of interest is easy.“—Alan Perlis For instance, regular expres­sions aren’t Turing complete, but they’re valu­able because they express a lot of compu­ta­tion with minimal nota­tion. HTML is also not Turing complete, but it’s a useful way to control a browser. By contrast, the bf language is Turing complete, but even the most banal tasks require acres of impen­e­trable code.

 Does my defi­n­i­tion of a program­ming language include every­thing? No.

 Binary data formats don’t qualify as languages. For instance, a jpeg file. Though a computer can under­stand it, a human cannot. Or a PDF: if you crack one open, you’ll find some parts that seem human read­able. But that’s inci­dental to how PDF works. There’s no sense in which a human is intended to notate ideas using PDF constructs.

 Plain text files are not languages. Suppose we have a file containing Homer’s Iliad. We humans can read and under­stand this file. Though a computer can triv­ially process the file—say, by printing its contents—the text within the file is unin­tel­li­gible to the computer.

 Graph­ical user inter­faces are not languages. Yes, they’re nota­tion systems (that rely on text and image). But they’re only intel­li­gible to humans. Computers draw GUIs, but they’re not intel­li­gible to the computer. + But see Racket’s 2d language, an embedded language that makes ASCII-art boxes intel­li­gible.

 Above, I described a program­ming language as a “medium of exchange” between humans and computers. In that way, languages fit among the broader cate­gory of things we call inter­faces.

 This brings me to my second major claim (of three): that language-oriented program­ming is funda­men­tally an inter­face-design tech­nique. If you like thinking about inter­faces, you’ll love LOP. If you don’t, you may still love LOP, because it enables certain inter­faces that are other­wise unat­tain­able.

 One of my favorite exam­ples of language as inter­face is brag, a parser-gener­ator language made with Racket. If you’ve ever used the lex/yacc tool­chain, you know the goal is often to generate a parser from a BNF grammar. For instance, the BNF grammar for the bf language looks like this:

 bf-program : (bf-op | bf-loop)*

bf-op : ">" | "<" | "+" | "-" | "." | ","

bf-loop : "[" (bf-op | bf-loop)* "]" 1 2 3 bf-program : (bf-op | bf-loop)* bf-op : ">" | "<" | "+" | "-" | "." | "," bf-loop : "[" (bf-op | bf-loop)* "]"

 To make a parser in a general-purpose language, we’d have to trans­late this grammar into a pile of native code. Even with the help of a parser gener­ator, it’s a tedious job. And point­less—haven’t we already notated the grammar? Why do it again?

 With brag, however, our wish comes true. To make a parser, we simply add #lang brag to the file, which magi­cally converts our BNF grammar into brag source code:

 #lang brag

bf-program : (bf-op | bf-loop)*

bf-op : ">" | "<" | "+" | "-" | "." | ","

bf-loop : "[" (bf-op | bf-loop)* "]" 1 2 3 4 #lang brag bf-program : (bf-op | bf-loop)* bf-op : ">" | "<" | "+" | "-" | "." | "," bf-loop : "[" (bf-op | bf-loop)* "]"

 That’s it! When compiled, this file will export a func­tion called parse that imple­ments this BNF grammar.

 This is one of my favorite exam­ples of LOP because it’s unas­sail­ably supe­rior to the long-winded alter­na­tive. More­over, with a general-purpose language, this kind of inter­face is essen­tially impos­sible.

 But the language-oriented programmer does it all the time.

 This brings me to my third and final major claim, which is that among inter­faces, languages have unique strengths. The cate­gories below are not exhaus­tive or exclu­sive, of course. But I’ve found that LOP has a lot to offer in these situ­a­tions:

 When you want to create an inter­face usable by less skilled program­mers, or nonpro­gram­mers, or lazy program­mers (don’t under­es­ti­mate the size of that last cate­gory).  For instance, Racket has an elab­o­rate web-appli­ca­tion library. But it’s also possible to spin up a simple web server quickly with the web-server/insta language:  #lang web-server/insta

(define (start request)

(response/xexpr

'(html (body "Hello LOP World")))) 1 2 3 4 #lang web-server/insta ( define ( start request ) ( response/xexpr ' ( html ( body "Hello LOP World" ))))  Matthew Flatt’s article Creating Languages in Racket demon­strates a language that gener­ates playable text adven­tures. As with brag, it looks more like a spec­i­fi­ca­tion than a program, but it works:  #lang txtadv



===VERBS===



north, n

"go north"



south, s

"go south"



get _, grab _, take _

"get"



===THINGS===



---cactus---

get

"Ouch!"





===PLACES===



---desert---

"You're in a desert. There is nothing for miles around."

[cactus, key]



north

meadow



south

desert 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 #lang txtadv ===VERBS=== north, n "go north" south, s "go south" get _, grab _, take _ "get" ===THINGS=== ---cactus--- get "Ouch!" ===PLACES=== ---desert--- "You're in a desert. There is nothing for miles around." [cactus, key] north meadow south desert  When you want to simplify the nota­tion. Regular expres­sions are one example. Another example is my DSL Pollen, a language for making online books (including this one). Pollen is like Racket, except that you start in plain-text mode, and use a special char­acter to denote Racket commands, which are eval­u­ated and spliced into the content. + Pollen is based on Racket’s docu­men­ta­tion language Scribble, which handles most of the heavy lifting. So the first part of this para­graph is programmed like so:  #lang pollen



When you want to simplify the notation. Regular expressions are one example. Another example is my DSL ◊link["https://pollenpub.com"]{Pollen}, a language for making online books (including this one). 1 2 3 #lang pollen When you want to simplify the notation. Regular expressions are one example. Another example is my DSL ◊link["https://pollenpub.com"]{Pollen}, a language for making online books (including this one).  Pollen takes care of inserting all the neces­sary tags and converting it to infal­lible HTML. I get the bene­fits of manual markup (I still have total control of what ends up in the page) but none of the costs (I cannot, say, acci­den­tally omit a closing tag).  Another example of simpli­fied nota­tion is lindenmayer, a language for gener­ating and drawing frac­tals called Linden­mayer systems, like this one:   In ordi­nary Racket, a Linden­mayer program might look like this:  #lang racket/base

(require lindenmayer/simple/compile)

(define (finish val) (newline))

(define (A value) (display 'A))

(define (B value) (display 'B))

(lindenmayer-system

(void)

finish

3

(A)

(A -> A B)

(B -> A)) 1 2 3 4 5 6 7 8 9 10 11 12 #lang racket/base ( require lindenmayer/simple/compile ) ( define ( finish val ) ( newline )) ( define ( A value ) ( display ' A )) ( define ( B value ) ( display ' B )) ( lindenmayer-system ( void ) finish 3 ( A ) ( A -> A B ) ( B -> A ))  But one can use the simpli­fied nota­tion merely by changing the #lang desig­na­tion at the top of the file:  #lang lindenmayer/simple

## axiom ##

A

## rules ##

A -> AB

B -> A

## variables ##

n=3 1 2 3 4 5 6 7 8 #lang lindenmayer/simple ## axiom ## A ## rules ## A -> AB B -> A ## variables ## n=3  The language presumes you already know some­thing about Linden­mayer systems. But the simpli­fied nota­tion makes it easy to trans­late what you know into a program that does what you want.  When you want to build on existing nota­tion. Above, we saw brag, a DSL that uses a BNF grammar as source code.  #lang brag

bf-program : (bf-op | bf-loop)*

bf-op : ">" | "<" | "+" | "-" | "." | ","

bf-loop : "[" (bf-op | bf-loop)* "]" 1 2 3 4 #lang brag bf-program : (bf-op | bf-loop)* bf-op : ">" | "<" | "+" | "-" | "." | "," bf-loop : "[" (bf-op | bf-loop)* "]"  Another example—early on, people who tried Pollen said “that’s cool dude, but I prefer Mark­down.” Wish granted. pollen/markdown is a Pollen dialect that offers the seman­tics of Pollen but accepts ordi­nary Mark­down nota­tion:  #lang pollen/markdown



When you want to simplify the notation. Regular expressions are one example.

My DSL [Pollen]("https://pollenpub.com") is a language for making online books. 1 2 3 4 #lang pollen/markdown When you want to simplify the notation. Regular expressions are one example. My DSL [Pollen]("https://pollenpub.com") is a language for making online books.  The nicest part? It took me about one hour to create this dialect by combining a Mark­down parser with my existing code.  When you want to create an inter­me­diate target for other languages. JSON, YAML, S-expres­sions, and XML are all DSLs that define data formats meant to be machine-writable and -read­able.  In Beau­tiful Racket, one tuto­rial language is called jsonic. It lets us embed Racket expres­sions in JSON, thereby making JSON program­mable. So a source file like this:  #lang jsonic

// a line comment

[

@$ 'null $@,

@$ (* 6 7) $@,

@$ (= 2 (+ 1 1)) $@,

@$ (list "array" "of" "strings") $@,

@$ (hash 'key-1 'null

'key-2 (even? 3)

'key-3 (hash 'subkey 21)) $@

] 1 2 3 4 5 6 7 8 9 10 11 #lang jsonic // a line comment [ @$ 'null $@, @$ (* 6 7) $@, @$ (= 2 (+ 1 1)) $@, @$ (list "array" "of" "strings") $@, @$ (hash 'key-1 'null 'key-2 (even? 3) 'key-3 (hash 'subkey 21)) $@ ]  Compiles to an ordi­nary JSON result:  [

null,

42,

true,

["array","of","strings"],

{"key-1":null,"key-3":{"subkey":21},"key-2":false}

] 1 2 3 4 5 6 7 [ null, 42, true, ["array","of","strings"], {"key-1":null,"key-3":{"subkey":21},"key-2":false} ]  When the bulk of the program is config­u­ra­tional. Dotfiles, for instance, can be char­ac­ter­ized as DSLs. A more sophis­ti­cated example in Racket is Riposte by Jesse Alama, a language for testing JSON-based HTTP APIs:  #lang riposte



$productId := 41966

$qty := 5

$campaignId := 1



$payload := {

"product_id": $productId,

"campaign_id": $campaignId,

"qty": $qty

}



POST $payload cart/{uuid}/items responds with 200



$itemId := /items/0/cart_item_id



GET cart responds with 200 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 #lang riposte $productId := 41966 $qty := 5 $campaignId := 1 $payload := { "product_id": $productId, "campaign_id": $campaignId, "qty": $qty } POST $payload cart/{uuid}/items responds with 200 $itemId := /items/0/cart_item_id GET cart responds with 200  As a minia­ture scripting language, Riposte is a lot smarter than the average dotfile. It hides all the inter­me­diate code required for HTTP trans­ac­tions, and lets the language user focus on writing tests. It’s still house­keeping. But at least you can focus on the house­keeping that you care about.

 A common critique of LOP is “why make a domain-specific language? Isn’t that more work than writing a native library?”

 Not if you have the right tool. Racket is unusual: it’s been designed from the ground up to support LOP. Thus, imple­menting a DSL in Racket is faster, cheaper, and easier than the alter­na­tives. For instance, in the first tuto­rial in this book, I show how you can make a language in one hour—even if you’ve never used Racket.

 Under the hood, every DSL in Racket is actu­ally a source-to-source compiler that converts the nota­tion and seman­tics of the DSL into the equiv­a­lent Racket program. For this reason, a Racket DSL isn’t going to run as fast as one written in hand-carved C. But it also makes all of Racket’s tooling and libraries acces­sible in every Racket DSL. So what you lose in perfor­mance, you get back many times over in conve­nience. And when DSLs are conve­nient and cheap, they become a real­istic option for a much wider range of prob­lems.

 Thus, to answer the critique—no, a DSL is not neces­sarily more work than a native library. More­over, as we’ve already seen, as an inter­face, a language can do things that a native library cannot.

 Because Racket DSLs compile to Racket, a language-oriented programmer using Racket needs to write some syntax trans­formers that convert the DSL nota­tion into native Racket. These syntax trans­formers are known as macros. Indeed, macros can be char­ac­ter­ized as exten­sions to the Racket compiler.

 Racket’s macro system is vast, elegant, and unde­ni­ably its crown jewel. But a lot of this book is about the joy of Racket macros. That mate­rial is ready when you are. For now, the two standout features:

 Racket has a special­ized data struc­ture called a syntax object that is the medium of exchange among macros. Unlike a string, which can only contain raw code, a Racket syntax object pack­ages the code in a way that preserves its hier­achical struc­ture, plus meta­data like lexical context and source loca­tions, and arbi­trary fields called syntax prop­er­ties. This meta­data stays attached to the code during its various trans­for­ma­tions. (See syntax objects for the details.)  Racket macros are hygienic, which means that by default, the code produced by a macro retains the lexical context from where the macro was defined. In prac­tice, this elim­i­nates a huge amount of the house­keeping that would ordi­narily be required to make DSLs work. (See hygiene for the details.)

 Is it possible to imple­ment a DSL in, say, Python? Sure. In fact, I wrote my first DSL in Python—one that I still use to help with my type-design work. Yikes. Once was enough. Ever since, I’ve used Racket.

 At this point, you may be having one of two reac­tions:

 “LOP seems inter­esting, but I don’t know what I’d do with it.” Sure—I wasn’t neces­sarily expecting to imme­di­ately recruit you into the LOP army. Rather, my goal has been to torque your thinking in a produc­tive way. You’ve now learned about a previ­ously unfa­miliar tool. Today, you might not see a use for it. But someday, you’ll confront a problem that exceeds the limits of your current favorite language. On that day, LOP will have its opening.  “OK, you’ve convinced me, but there’s no way I can get LOP or Racket into my work­place.” Jesse Alama’s story of how he intro­duced his DSL Riposte is a great example of winning over colleagues with LOP (emphasis mine below):  [One can try] to get permis­sion upfront to do some­thing in Racket. That’s less likely to succeed, I think, than just making some­thing great and explaining its bene­fits ... In my case, that meant talking with co-workers about their work and asking: “How can we model the proposed change in the API and be sure that we’ve really succeeded?” The implied answer being: “Write a Riposte script.” That’s the moment where it becomes clear that [the DSL] I made has real bene­fits. I don’t even “push” Racket. I just intro­duce the DSL and show them how it helps them.

 At the end of Why Racket? Why Lisp?, I said that a Lisp language “offers you the chance to dis­cover your poten­tial as a pro­gram­mer and a thinker, and thereby raise your expec­ta­tions for what you can accom­plish”.

 LOP offers a similar oppor­tu­nity: to raise our expec­ta­tions for what program­ming languages can do for us. Languages are not black boxes. They are inter­faces that we can design. In so doing, we open up new possi­bil­i­ties for what we can accom­plish with programs.

 If you can find a better program­ming tech­nique, use it. Now that I have LOP & Racket, I’m never going back.