include StarbucksDSL

order = latte venti, half_caf, non_fat, no_foam, no_whip

print order.prepare



Whoa! This looks like a long post with a lot of code snippets. Am I going to have to do a lot of hard thinking, or can I just relax and enjoy a good rambling essay?This is a bit long, probably (like all my posts) 200% longer than necessary. If you just want to see a neat DSL that implements Haskell and Python’s List Comprehensions written in Ruby, just scroll to the bottom If I do bother to read it all, will I learn some neat hacks?Yes, but you could learn them just as well by reading the source code directly.So the benefit of reading the whole thing is...?The List Comprehensions DSL is the. The source code is the. But the essay is theReading the whole thing will take you through some of the pitfalls of writing DSLs and explain why I chose my particular workarounds.Furthermore, there are a lot of corners in Ruby where you can easily assume that things work one way, but really they don’t . If you actually try the snippets on your computer, you’ll have a much better chance of remembering where the pitfalls are. That’s why I tried to give a working example for every point, rather than just explaining things in words.Of course, if you have no interest in writing your own Domain Specific Languages in Ruby just yet... this isn’t meant as a popular essay, rather it’s meant as an experience report for fellow practitioners. And honestly, there’s a world market for maybe five tools for writing DSLs in Ruby.But since you’re here, the essay starts below!Ruby is often touted as a good language for writing Domain-Specific Languages (“DSLs”). There are a few arguments in favour of writing a DSL as part of an application.The first argument that comes to mind is that if the application’s domain experts have a specific natural language or jargon of their own, writing a DSL makes it easy for programmers and domain experts to collaborate. While it is rare to find substantial applications entirely written by non-programmers at this time in any language, it is quite feasible for non-programmers to write or validate portions of an application representing its “business rules” or domain logic, while programmers maintain its infrastructure.

—Building Domain Specific Languages in Ruby

has_and_belongs_to_many :Bar

validates_presence_of :blitz

some_bars = Bar.find_by_tavern_license(license_number)







The Ruby Way is the perfect second Ruby book for serious programmers. The Ruby Way contains more than four hundred examples explaining how to do everything from distribute Ruby with Rinda to dynamic programming techniques just like these.



is the perfect second Ruby book for serious programmers. The Ruby Way contains more than four hundred examples explaining how to do everything from distribute Ruby with Rinda to dynamic programming techniques just like these.



Lispers are among the best grads of the Sweep-It-Under-Someone-Else’s-Carpet School of Simulated Simplicity.



Another argument in favour of a DSL is that even when non-programmers are not involved directly in coding an application, the programmers themselves often have a jargon of their own to describe entities, algorithms and data structures in the application. Having portions of the application written in a language closely resembling the programmer’s own jargon makes it easy for them to read each other’s work and understand its intent.Successful examples of DSLs embedded within existing languages and frameworks include Ruby on Rails ’ ActiveRecord, where statements such as:Are self-documenting to anyone familiar with relational models.The final argument I’ll repeat here is that a DSL is a very effective way to separate the what from the how of an algorithm. Separation of concerns is a desirable property of good programs , and DSLs provide this separation very clearly. In the ActiveRecord examples above, the exact mechanisms of relating tables, validating records, and performing searches is “abstracted away” from the code where the programmer declares how she would like the results used.DSLs can be hacked together quickly in Ruby (whether they can be made sufficiently robust for your production needs may require considerably more care). Hacking a DSL together with little effort is a benefit, especially when prototyping: sometimes the best way to design a DSL is to try to use it, so you can discover what you need to express.Some developers have raised the concern that extensive use of “magic” features leads to code that cannot be understood or maintained.My own feeling is that DSLs lead to code that is easier to understand, not more difficult to understand. This leaves an argument about maintenance. Some techniques for meta-programming, such as extending core classes like Array, have what you might call “non-local effects.”For example, two different pieces of code might try to extend the same core class, interfering with each other. Each works in isolation and passes all of its unit tests. But when plugged into a larger application that uses them together, they break.

—Larry Wall

Kernel

instance_eval



I’m trying to get the Zen of building DSLs using Ruby. After reading a dozen or so pieces referenced by my favourite search engine, I have a feeling I’m still not quite getting it.



Another problem occurs with extending theclass or creating “top level” methods to be used as verbs in a DSL. You end up with name space crowding: you must be very careful that you do not redefine en existing method.To fix this problem, the code that implements the DSL needs to be contained so that it does not interfere with other code. We can still implement verbs as methods, but we must implement those methods in separate objects, classes, or modules.An established technique for implementing methods in objects is to define the methods and then execute a block of code usingso that it has access to the object’s methods.

—Don Box

def bjarne

'Barney'

end



dsl = Object.new

def dsl.phred

'Fred'

end



plus = ' plus '



print dsl.instance_eval {

phred + plus + bjarne

}

##### "Fred plus Barney"



dsl

plus

bjarne

bjarne

Kernel

bjarne

bjarne

bjarne

phred

phred

phred

bjarne

MyDsl = Object.new



def MyDsl.phred

'Fred'

end



class ClientCode



def bjarne

'Barney'

end



def friends

plus = ' plus '

MyDsl.instance_eval { phred + plus + bjarne }

end



end



ClientCode.new.friends

##### -:15:in `friends': undefined local variable or method `bjarne' for # (NameError) from -:15:in `friends' from -:20



ClientCode

bjarne

phred

bjarne

instance_eval

some_object.a_method

bjarne

self.bjarne

bjarne

Kernel

def foo

'top level foo'

end



def bar

'top level bar'

end



class Test

def bar

'instance method bar'

end

def test

p foo

p bar

end

end



Test.new.test

##### "top level foo" "instance method bar"



()

self

self

instance_eval

self

def foo

'top level foo'

end



def bar

'top level bar'

end



class Test

def bar

'instance method bar'

end

def blitz

'current object blitz'

end

def test

p foo

p bar

o = Object.new

def o.blitz

'redefined self blitz'

end

p o.instance_eval { blitz }

p o.instance_eval { 'bar within o gives: ' + bar }

end

end



Test.new.test

##### "top level foo" "instance method bar" "redefined self blitz" "bar within o gives: top level bar"



instance_eval





The Seasoned Schemer is devoted to the myriad uses of first class functions. This book is approachable and a delight to read, but the ideas are provocative and when you close the back cover you will be able to compose programs from functions in powerful new ways.



is devoted to the myriad uses of first class functions. This book is approachable and a delight to read, but the ideas are provocative and when you close the back cover you will be able to compose programs from functions in powerful new ways.

instance_eval

self



Those who do not learn from the History of Lisp are doomed to repeat it.



DomainSpecificLanguage

require 'dsl'



class MyDSL < DomainSpecificLanguage



def bjarne

'Barney'

end



end



class TheGreat



def phred

'Fredrick'

end



def test

plus = ' plus '

MyDSL.eval { p phred + plus + bjarne }

end



end



TheGreat.new.test

##### "Fredrick plus Barney"



kernel

with

with

eval



with MyDSL do

p phred + plus + bjarne

end



eval

class Censor < DomainSpecificLanguage

attr_reader :ok_on_tv



def initialize (given_binding)

super(given_binding)

@ok_on_tv = true

end



def say something

something.split.each do |word|

@ok_on_tv = false if ['feces', 'urine', 'love', 'pudendum', 'fellator', 'oedipus', 'mammaries'].include?(word)

end

end



end



class GeorgeCarlin

def test

Censor.eval {

say "People much wiser than I have said, I'd rather have my son watch a film with two people making love than two people trying to kill one other."

say "And I of course agree. I wish I know who said it first, and I agree with that."

ok_on_tv

}

end

end



p GeorgeCarlin.new.test

##### "false"



eval

instance_exec

with Let do

let :x => 0, :y => 1 do

assert_equal(1, x + y)

let :x => 2 do

assert_equal(3, x + y)

end

assert_equal(0, x)

end

end



with

Let

let

let

Let

x

let

x

y

x

y

x

let

Let

S = [ x | x<-[0..], x^2>3 ]

[x, y, x * y]

x

1..12

y

1..12



require 'comprehension'



class MultiplicationTable

def twelve_by_twelve

with Comprehension::DSL do

list { [x, y, x * y] }.given(:x => 1..12, :y => 1..12)

end

end

end

p MultiplicationTable.new.twelve_by_twelve

##### [[1, 1, 1], [1, 2, 2], [2, 1, 2], [1, 3, 3], [2, 2, 4] ...



list {

[x, y, x * y]

}.given(:x => 1..12,

:y => 1..12)

with Comprehension::DSL do...

end



class MultiplicationTable

def twelve_by_twelve

with Comprehension::DSL do

list { "#{x} times #{y} is #{x * y}" }.given(:x => 1..12, :y => 1..12)

end

end

end

p MultiplicationTable.new.twelve_by_twelve

##### ["1 times 1 is 1", "1 times 2 is 2", "2 times 1 is 2", "1 times 3 is 3", "2 times 2 is 4", ...





class MultiplicationTable

def twelve_by_twelve_odds

with Comprehension::DSL do

list { "#{x} times #{y} is #{x * y}" }.given(:x => 1..12, :y => 1..12) { (x % 2 == 1) && (y % 2 == 1) }

end

end

end

p MultiplicationTable.new.twelve_by_twelve_odds

##### ... 3 times 5 is 15", "5 times 3 is 15", "7 times 1 is 7", "1 times 9 is 9", ...





class MultiplicationTable

def odds_times_evens

with Comprehension::DSL do

list { "#{x} times #{y} is #{x * y}" }.given(

:x => list { x }.given(:x => 1..12) { x % 2 == 0 } ,

:y => list { x }.given(:x => 1..12) { x % 2 == 1 } )

end

end

end

p MultiplicationTable.new.odds_times_evens

##### ... "2 times 11 is 22", "4 times 9 is 36", "6 times 7 is 42", ...



Let

Let

dsl.rb

dsl.rb

dsl.rb

comprehension.rb

comprehension.rb

I generally call “Bullshit!” on any line of reasoning that sets up a straw man argument just to knock it down. So read on with skepticism!

Alan Kay has said that he regrets popularizing the notion of “Object-Oriented” programming, and that he should have called it “Message-Oriented” programming.

You know, code expresses an idea better than words express an idea… when the idea is about coding. Please try this example in irb. Don’t just skim the text and nod: there’s a powerful learning mechanism at work when you physically do things as you’re learning, even if it’s just copying, pasting, comparing the result in one window to the text in another, and so on:What does this show? Well, we have created a way to use a method defined in ourobject, a local variable, and a top-level method. We can imagine scaling this up to defining a rich DSL in our DSL object and being able to mix verbs from the DSL with instance variables and other methods as we please.Touching back on the subject of containment, we have definedin. Nowis essentially global. If we already definedsomewhere else, we just clobbered it. And if we later run a piece of code that defines, we’ll clobber our own version.is different. It’s defined inside of an object, and it doesn’t conflict with any otherwe define elsewhere.Great! So… Can we cite a few examples of this technique in action (such as Jamis’ post where he callsandexamples of Sandboxing and Top-level methods) and end the post here?No. The code above looks fine. But there is a hidden problem with this sandboxing technique: WTF ?! This looks just like our top-level example, but we’ve placed our code inside of amethod. Andis a method in ClientCode: this way we can continue to separate concerns, keepinginside our DSL andinside of the class where we are using the DSL. But it doesn’t work.As you know, everything in Ruby is either a variable or a method (how it figures out the difference is a major irritation). When you invoke a method, you are actually sending a message to a receiver.Sometimes you name the receiver (), and there is no ambiguity.But when you just name the method (like), Ruby tries to find the method for itself. It does so by looking to see whether it is an instance method, in which case it behaves like. If not, it looks to see whetheris top-level, in which case it calls that method in the. See for yourself:See? It looks for instance methods and then for top-level methods if it can’t find anything. (Again, we are hand-waving over the pesky problem with instance variables in the case where we don’t use). What’s the problem? Well, I actually mis-described what happens. Here it is again, with more precision:It looks for methods defined in the object, and then for top-level methods if it can’t find anything. Of course,is the current object. Unless it isn’t: That’s whatdoes: it evaluates a block but it changesto point to its receiver instead of the object where the code is executing. Everything else stays the same. One more example to show the mechanism:Now we see: when we use, we route around our current object and all of our methods are ignored within the block. Ruby really only has two levels of scope: whatever belongs to self and whatever belongs to Kernel.This state of affairs is unsatisfactory: we would like to introduce a DSL in such a way that we retain access to all of our methods without kludges (like storing the current object in an instance variable).You can think of the current scope as being nested inside of the top-level scope.doesn’t change the scope for things like local variables, it just pointselsewhere.What we want is a new scope for our DSL nested inside of the current scope. So when we search for a method, we should check the DSL. If we don’t find it there, check the current object’s scope. If we don’t find it there, check the top-level.Oops. John McCarthy called from 1960. He wants Lisp’s dynamic scoping back. Yes, our new feature is almost fifty years old. This is why either a through grounding in CS theory or a hobbyist’s interest in the history of programming are important for programming: much of what we want to do has already been done before, and sometimes in unexpected contexts. Who would have thought that a technique for helping programmers collaborate with Bond Traders has roots in Lisp 1.5? Here’s an implementation of a nested scope construct that does exactly what we want. You declare a new class that extends, and then you can use methods from your DSL, from your current object, and from the top-level (if you so choose). For example:This does exactly what we want with methods.There's also a single extension to, the methodreplaces themethod so you can also say:Themethod creates a new instance of your DSL class, so you can track state within an evaluation. For example:The first obvious drawback of this approach is that the blocks we pass tocannot take parameters. For this reason, rumour has it that a method calledwill be added to Ruby in 1.9. (There are some implementations available that work in Ruby 1.8 if you would like to experiment.)The second is that you don’t get anything like nested local variables, a ‘la Pascal, Scheme, or any other language with block structure. Block structure is very powerful: You can use a variable within a particular scope and nowhere else. Here’s a trivial example:We're using thesyntax. In theDSL, there’s a new method calledcreates a new DSL within. You can see that re-declaringdoes not clobber the value in the outer scope. That is because whenwrote a new DSL, it addedandas methods.So really, that block of code says “Write a new DSL whereandare methods returning zero and one. Execute some code in that new DSL. That code will createDSL whereis a method returning two.”Becausedefines methods and not local variables, bad things happen when you try to override real local variables. It’s best to usefor some things and local variables for others, but not mix the two.Like what, you ask? List Comprehension is syntactic sugar that lets you build collections using set-like notation. For example,is a list comprehension in Haskell.Here is a List Comprehensions DSL in Ruby . Let’s say we’re building up a multiplication table. We want tuples of the formgivenis in the rangeandis in the range. Let’s write that:(In everyday use, you don’t need a class and a method for each comprehension: the important bit is. I just wrote it this way so you could see that comprehensions work fine inside of methods. You can also use more than one comprehension inside of a singleblock: see the unit tests for examples.)The expression in the block doesn’t have to be a tuple:And you can stick a “where” block on the end:Would you like to nest them? Your expression is the interpreter’s command:What is the relationship to? Well,builds the scopes needed for evaluating the where clause and the block defining the elements of the list. Yes, we’ve built a DSL on top of a DSL on top of a DSL. Does this seem like weird trickery? I don’t know why. Do you have any idea how many levels of abstraction are responsible for you reading this essay right now?This is what we humans do: we build tools on top of tools. Your browser runs on an OS, possibly in a VM, perhaps in a hypervisor, on top of a BIOS, and on and on. This is the normal state of affairs, not an exception.It is possible to build DSLs in Ruby to facilitate cross-functional teamwork and separation of concerns. Care must be taken to avoid polluting the top-level name space, but it is possible to work within sandboxes and still have access to the current object’s context.Oh yes, and programming is fun as alwaysUpdate: The copy ofhas been updated to the latest version. I had committed a rather typical manual synchronization error: I copied the latest file to the wrong directory when I first posted this. Thanks, Justin!: Open DomainSpecificLanguage and Let . Save the text only (not the HTML) as. Open Comprehension . Save the text only as anything you like, as long as it is in the same directory as: I use. Run

Labels: lispy, popular, ruby