Chalain

Feb. 18th, 2008 03:06 pm Don't Push the River There is a Buddhist saying: "Don't Push the River." Sometimes, when you are repeatedly frustrated, that is life's wonderful way of helping you stop pushing the river, to stop trying too hard, to stop fighting the current and instead flow with it.



Ruby has an interactive program called IRB that gets a lot of usage; for example, Rails uses it as its command-line interface. One of its features is that after every command, it returns the result of the last evaluated statement. This is in keeping with Ruby's return semantics; if you leave an argument at the end of a function it will be returned automatically. This is strongly in keeping with Ruby's functional programming leanings.



The problem with this, however, is that in Ruby, everything returns stuff. If you print a string with puts , the return code from puts is printed. ( puts always returns nil.) If you iterate over a collection with each , the collection will be returned.



This has been especially griefsome to me because a lot of the time I use IRB to "pretty print" some collection of data. Since I'm an old C programmer who apparently can't change his shorts (or his longs), I do it the way I would do it in C: loop over the collection, printing each element.

>> Airport.find(:all).each do |a| ?> puts "#{a.abbrev}: #{a.name}" >> end DFW: Dallas Fort-Worth ATL: Atlanta Hartsfield-Jackson => [#<Airport:0x21f131c @attributes={"name"=>"Dallas Fort-Worth", "abbrev"=>"DFW", "id"=>"1"}, #<Airport:0x21f12f4 @attributes={"name"=>"Atlanta Hartsfield-Jackson", "abbrev"=>"ATL", "id"=>"2"}] You can see the result: the list is printed, and then the collected hash is displayed. This is tolerable when I'm printing 2 airports, but when I'm printing 200 of them, the collection at the end is trash that scrolls off the screen.



Well, I'm nothing if not a gross hacker, so I have gotten in the habit of adding an extra line of code to the end of the loop: nil . This causes IRB to evaluate nil as the last expression, and it returns that after the pretty display:

>> Airport.find(:all).each do|a| ?> puts "#{a.abbrev}: #{a.name}" >> end; nil DFW: Dallas Fort-Worth ATL: Atlanta Hartsfield-Jackson => nil Nice! ...in a sort of lame, hackish way.



You see, since I read Paul Graham's post about making new things I have started reconsidering all the little things that hurt in Ruby, with an eye towards writing some cheap, simple fixes. It shouldn't be too hard to tweak IRB to stop returning the last expression.



But then I realized that this would change the way IRB works fundamentally. Sometimes you want this behavior. In fact, most of the time you do. Okay, maybe I'll just make it a mode you can toggle, like expr_on|expr_off .



And then I realized: I am pushing the river here. I am trying to alter a fundamental Ruby behavior--functional orientation--to match my procedural thinking. How hard would it be to just change my thinking?



Hmm. In functional programming, we say what we want rather than how we want to get it. Okay, so instead of starting with a loop, perhaps I should start with puts . It just so happens that puts will automatically print a collection one item per line. So all I need to do is transform my collection of airports into a collection of pretty strings. For that I'd want map or collect instead of each :

>> puts Airport.find(:all).collect {|a| "#{a.abbrev}: #{a.name}" } DFW: Dallas Fort-Worth ATL: Atlanta Hartsfield-Jackson => nil Nice! And proper this time. I went ahead and inlined the block because I was concerned about the result of the transform rather than the side effects of the loop¹. I am very pleased about this result: I had a way I wanted to do it, and Ruby was resisting me, and I finally realized that I was in the wrong.



I like Ruby for two reasons. First, it was the first language I learned in over a decade that actually forced me to learn a new language². Second, there are moments, often daily, when I become consciously aware of experiencing pleasure while using the language. This was an example of both reasons.



(Edit: Thanks, Reg! Linked.)



² Many of you might argue that C++, Visual Basic, Java, PHP, perl, C#, and Python are wildly different languages. You are wrong. They're all lexically-scoped, block-oriented languages. That means you can think in C++ and write in any of those languages and get by just fine. Sure, there may be features missing in a language (like pointers in VB) and you may leave pieces of the language unexplored (like generators in Python), but in all of those languages you can get by just fine learning the syntax and a handful of idioms and then continuing to think and write in C++ grammar. This is not a good thing. Take it from me: If you only know procedural programming, you should learn a functional language this year. ¹ Shout-out directly to Reg (you know who you are): I learned this from your blog , but after 10 minutes of searching for it I could not find the post it is from. Would you e-mail me the url? I'd like to link it here for the benefit of others.² Many of you might argue that C++, Visual Basic, Java, PHP, perl, C#, and Python are wildly different languages. You are wrong. They're all lexically-scoped, block-oriented languages. That means you can think in C++ and write in any of those languages and get by just fine. Sure, there may be features missing in a language (like pointers in VB) and you may leave pieces of the language unexplored (like generators in Python), but in all of those languages you can get by just fine learning the syntax and a handful of idioms and then continuing to think and write in C++ grammar. This is not a good thing. Take it from me: If you only know procedural programming, you should learn a functional language this year. Current Mood: contemplative

Current Music: Glidepath (132bpm) - DJ Steveboy

25 comments - Leave a comment

From: nightsinger Date: February 18th, 2008 11:35 pm (UTC) (Link) in all of those languages you can get by just fine learning the syntax and a handful of idioms and then continuing to think and write in C++ grammar



I totally noticed that as well. As a novice, I find it quite useful functionality... but you do have an excellent point. I'll make an effort to pay attention to those differences in new languages I learn from here on out. :) Reply ) ( Thread

From: chalain Date: February 19th, 2008 12:28 am (UTC) (Link)



I highly recommend the In the long term it is easy to shortchange yourself by learning new syntaxes without ever learning a new language. In the short term, it makes you highly employable if you can pick up a new language and run with it. :-)recommend the SICP Lectures Reply ) ( Parent ) ( Thread

From: kazriko Date: February 19th, 2008 12:37 am (UTC) (Link) The thing about python is... that while it's possible to write while thinking C++, it's entirely the wrong way to approach python. It's also possible to think Lisp and write python (you just have to be a bit more explicit than Lisp,) and be closer to right though still a ways from the right way of coding it.



I think this was a factor in your dismissal of Python, and many times when you posted things in IRC that you had written in python, they seemed entirely wrong and too C++. You were working entirely too hard and not looking at what tools were available to you before you reinvented the wheel. (A disturbing lack of List Comprehensions where they were both easier to read and faster to execute was one thing I remember.)



Personally, your complaint on python being too close to C++ is my same complaint about Ruby. It's putting a Perl-esque syntax on top of Lisp, when Common Lisp is a far more elegant language to work with than Perl. It's comfort food/trojan horse for those who don't want to abandon their old syntaxes rather than something revolutionary. The difference between the python/C++ to Ruby/Lisp comparison is that Python is an actual and real improvement in coding time to C++ and a good prototyping tool (though perhaps not an improvement over lisp,) where I'm not sure Ruby improves Lisp's already excellent coding speed at all. (It also doesn't really do anything to make Lisp more comprehensible to the general public, only to those from a narrow subset that know the syntaxes that it is using...)



A trend I've seen in all the time I've been programming is the fact that there always seems to be one bandwagon trendy language that large groups are touting as the be-all-end-all. As far as that goes, I'm slightly annoyed by those who tout Ruby as a superbly great language for every conceivable problem. (Example: Ruby on Rails and its using Ruby to do every stage of the process, from database access to logic to presentation.) Ruby has succeeded C++, Perl, Java, and Python as the trendy language de jour. It's useful in some fields just as Perl in inordinately useful in some tasks, and Java is useful in some tasks and fields still. None of them will ever be the language to do every task.



It will eventually be supplanted by another trendy language de jour and all the trendy language hoppers will abandon it, leaving a core of people who actually understand it and find it useful in their field.



I do need to get back into Lisp, it has been 5-6 years since I used it seriously. Erlang might be a better path now.



Edited at 2008-02-19 12:40 am (UTC) Reply ) ( Thread

From: ext_15571 Date: February 19th, 2008 12:47 am (UTC) Did you mean (Link) Programming Conventions as Signals (http://weblog.raganwald.com/2007/11/programming-conventions-as-signals.html)? Reply ) ( Thread

From: (Anonymous) Date: February 19th, 2008 12:48 am (UTC) Python is not a C language (Link) > in all of those languages you can get by just fine learning the syntax and a handful of idioms and then continuing to think and write in C++ grammar



You seem to be arguing that it's possible to write "C++ in Python" but not in Ruby. I think your zeal for your new friend Ruby is blinding you a bit here... As you demonstrate in this very post, it's quite possible to write C++ in Ruby. That kind of code in Python is just as un-idiomatic as non-block-oriented Ruby.

Reply ) ( Thread

From: (Anonymous) Date: February 19th, 2008 12:52 am (UTC) Re: Python is not a C language (Link) P.S. Ruby resembles Smalltalk much more closely than it resembles Lisp. A "functional language" is about more than just first class functions (which Python also has, incidently). Reply ) ( Parent ) ( Thread

From: chalain Date: February 19th, 2008 05:23 am (UTC) Re: Python is not a C language (Link) I certainly agree when you write C++ code in language XYZ, you generally end up with lousy XYZ code. The frequency and spectacularness with which my Ruby code stopped working when I tried to code in C++, however, was an order of magnitude higher than with other languages.



As far as Python not being a C language... let's just agree that you're wrong. Oh sure, it's got continuations and closures and list comprehensions. But the language works just fine without them. If you sit down and write straight-up C++ architecture in Python, it will work. Un-idiomatic? Yes. Wrong and icky? Yes. Absolutely anything in the language itself to suggest that you're doing it wrong? No.



Ruby, on the other hand, does not work just fine without its non-C bits. When I first got into it, I was furious and considered it a defect that my C-style code broke in weird ways. And then I realized that I was doing it wrong, and there WAS a right way to do it.



Don't get me wrong: Python is a wonderful language, and the way I used it was inelegant and brutal. I'm not criticizing the language on those terms at all. But it's lexically scoped and block oriented, and I shipped close to 100,000 lines of Python code over 3 years with a syntax reference and headful of C++ architecture. As Kazriko has already pointed out, it was dreadful code. But neither the language nor the Python community at the time gave me any feedback that suggested there was anything wrong with it. Reply ) ( Parent ) ( Thread

From: (Anonymous) Date: February 19th, 2008 11:03 am (UTC) Re: Python is not a C language (Link) > But it's lexically scoped



I don't understand this objection at all. Virtually every modern language, including Ruby, is lexically scoped.



> and block oriented



I'm not sure what this is supposed to mean, either, since again, Ruby is also block oriented.



> it was dreadful code, but neither the language nor the Python community at the time gave me any feedback that suggested there was anything wrong with it



How was the community supposed to give you feedback? If you post dreadful code to comp.language.python, for instance, you will certainly get suggestions for improvement.



My experience was that the language itself quickly pushed me towards using it more idiomatically.



Is it not possible that you are simply a more mature programmer today than you were when you were shipping 100kloc of lousy Python? Reply ) ( Parent ) ( Thread

From: chalain Date: February 19th, 2008 06:37 pm (UTC) Re: Python is not a C language (Link) Thank you for your comment. Your experience with the Python community was an order of magnitude better than mine--so much so that my experience with that community seems offensive to you. I did not mean to indict the Python community for my experiences in it; I apologize for giving offense.



That said, I'd like to have this conversation, but not in this tone. I can't tell if you're angry at me for liking Ruby, or for not liking Python enough, but neither of these is the conversation I want to be having. We're done here. Reply ) ( Parent ) ( Thread

From: (Anonymous) Date: February 20th, 2008 04:46 am (UTC) Re: Python is not a C language (Link) > I can't tell if you're angry at me



Ah, the drawbacks of online media. I'm not angry with you. Confused? A little. But not angry. :) Reply ) ( Parent ) ( Thread

From: chalain Date: February 20th, 2008 05:07 am (UTC) Re: Python is not a C language (Link) Fair enough, and thanks. Reply ) ( Parent ) ( Thread

From: giles_bowkett Date: February 19th, 2008 01:43 am (UTC) with my gem you get this for free (Link)



http://utilitybelt.rubyforge.org/

If you install my gem utility_belt and then require it in your .irbrc, you get a pair of commands which turn off IRB return values and turn them back on. The commands are quiet and verbose, but they have one-character aliases, namely q and v. Reply ) ( Thread

From: chalain Date: February 19th, 2008 05:33 am (UTC) Re: with my gem you get this for free (Link) Don't push the river, Giles!



The cost of getting this feature is that I would have to install your Utility Belt gem. Further evidence that whenever somebody says "we get this for free" they really mean "we are lying to ourselves about the terrible cost in human lives". :-)



If you look closely at my IRB posts, you'll notice the standard IRB syntax is gone; this is because I haves me some utility belt action already. I was unaware of the q,v features. Nice!



(For anyone else: if you're using Ruby, utility_belt is not optional. It rocks.) Reply ) ( Parent ) ( Thread

From: giles_bowkett Date: February 19th, 2008 06:46 am (UTC) Re: with my gem you get this for free (Link) (For anyone else: if you're using Ruby, utility_belt is not optional. It rocks.)



wow. thanks!



I admit I may be overlooking the terrible cost in human lives. I may have pushed the river as well. Just some missionary zeal for self-promotion. The downside with a Swiss Army Knife like utility_belt is the useful but overlooked features. :-) Reply ) ( Parent ) ( Thread

From: chalain Date: February 19th, 2008 07:03 am (UTC) Re: with my gem you get this for free (Link) I've been using and loving utility_belt for some weeks now. My personal favorite feature is the _ variable; it was one of the best features of Python's interactive mode.



I confess I was a little embarrassed to realize that I had never read the instructions for your plugin. You just plug it in and IRB stops hurting. Syntax coloring, better indentation, the _ variable... it never occurred to me that there might be more... esoteric... features in the utility_belt.



Now, if I could just figure out what the little rounded hook thing next to the corkscrew is for.... Reply ) ( Parent ) ( Thread

From: giles_bowkett Date: February 19th, 2008 02:02 pm (UTC) Re: with my gem you get this for free (Link) it never occurred to me that there might be more... esoteric... features in the utility_belt



wait til I get the Lisp interpreter up and running. :-)



Reply ) ( Parent ) ( Thread

From: samwibatt Date: February 19th, 2008 02:27 am (UTC) (Link) in all of those languages you can get by just fine learning the syntax and a handful of idioms and then continuing to think and write in C++ grammar...



Guilty.



The very phrase "functional language" makes me recoil, through no real fault of the languages'. I took a class in about 1993 in which the (newly-hired) professor insisted that everything we write be in a language which was so aggressively functional it didn't even support global variables, nor was it object-oriented. That was such a nightmare, given the kinds of assignments he had us do, that I still balk at the very idea of functional languages, but intellectually I understand their charm.



I'm not entirely procedural-boy; I did do a ton of programming in Prolog 20 years ago. FWIW. Reply ) ( Thread

From: chalain Date: February 19th, 2008 05:51 am (UTC) (Link) Banning global variables seems stupid and religious. I suspect it made for an awful language with limited utility; something that make hard things impossible and easy things painful.



Now, banning state altogether... that's the kind of crazy that makes me shiver. I mean, it's serious crazy--we're talking the acid-dropping, bat-shit, voices-tell-me-to-eat-people kind of crazy here--but it's not the stupid kind of crazy, and you almost want to listen to them when you hear it.



They talk about it in the SICP lectures, and explain why it's bad. And there's this part of me that realizes that I would probably enjoy the color of the sky in their world.



If only I could breathe the ammonia atmosphere over there on Planet Crazy, of course. Reply ) ( Parent ) ( Thread

From: jerith Date: February 19th, 2008 03:01 pm (UTC) (Link) When I learned Erlang, I thought the two biggest issues would be the syntactic weirdness and the variables that don't. These were the two things I got used to quickest. Reply ) ( Parent ) ( Thread

From: kazriko Date: February 19th, 2008 10:47 pm (UTC) (Link) Banning state is something that is a trade off. Not having state gives you some very interesting options for execution, keeping all of that "state" as parameters and return values that are flying around. That potentially gives you the ability to really heavily parallelize the whole thing, but also makes coding for it much harder. You don't have to synchronize the state between multiple processors, as the only state is stored in these little function calls that are only on one processor at a time.



There's other benefits and detriments to it as well... I'm sure they discussed those in SICP, even though I haven't watched that yet.



It'll be more interesting when we have 64-256 cores and you're trying to spread your processing out amongst them. Reply ) ( Parent ) ( Thread

From: chalain Date: February 19th, 2008 11:21 pm (UTC) (Link) I'm excited for the growing trend towards parallelization in software thinking. A recent project I was on had a lot of "single-minded naivete" in its thinking. It was obvious that the programmers wrote their code thinking that they were operating in an infinitely long critical section.



IIRC, the SICP lectures get into state about halfway to 3/4 of the way through. And it really is a beautiful thing when you can eliminate it in a section of code. For a given series of inputs, the output of the system is a known constant... I'm not sure if I'm explaining it well. They give an ATM example, where a user has $100. She wants to withdraw $20, then deposit $50. Taken as a series of events, the state of the system is completely constant: $100 plus two transactions of -$20 and +$30, then $80 plus one transaction of +$30, then $110 plus no transactions. All three of those balance + transaction statements were effectively identical. The state of the account balance is immaterial because after these events it is a constant f(x) = x+10.



Suddenly you start seeing your GUI as a constant thing: This program plus that mouse click equals this program. It all falls apart, of course, when multiple event streams must be considered, which is where state creeps in: If another user withdraws $90 before the first transaction, the first withdrawal will fail, so now you have an output state dependent on operating conditions. It all comes apart like a stained-glass window being hit with a rock.



Which was probably Hal's point in the lecture. Sometimes we need state. When we can eliminate it, we can do lots of wonderful things (reversible computing, anyone?) but when you can't, functional programming grinds down a bit. Sure, you can still create a function for your bank balance. But in order to remain constant, it must take your bank balance as an input. :-)



For me the current take-home lesson is that it doesn't have to be all-or-nothing. There are places in my programs where I would normally write stateful code, but after looking at it I realize that that one section could be reduced to a function. The result has been better code almost universally: it certainly is more testable, more decoupled, and more reusable. Reply ) ( Parent ) ( Thread

From: kazriko Date: February 20th, 2008 05:40 am (UTC) (Link) You don't necessarily need to pass the bank balance in to get the bank balance out, but you probably would need to pass in the synchronized way of getting transaction history, such as an external store of state data. (Perhaps a relational database...) For the purposes of parallel programming, that would just be a critical section that would need to be done sequenced with other critical sections accessing the same data.



In the real world, what happens is that both of those withdrawals go into the database, then a program runs at midnight and assesses a $39 charge to that account for overdraft. ;)

Reply ) ( Parent ) ( Thread

From: zenspider Date: February 21st, 2008 09:31 am (UTC) (Link) I agree that the "; nil" approach is hacky, but IMO switching to the map form is as well... sometimes it simply doesn't translate or is more obtuse to do so. You can certainly go that route or, you can learn the tools you use to be more effective:



>> IRB.conf[:MAIN_CONTEXT].echo = false

>> (1..100).to_a.map { |n| n * n }

>>



you can wrap that up pretty easy in your ~/.irbrc:



def toggle_output

IRB.conf[:MAIN_CONTEXT].echo = ! IRB.conf[:MAIN_CONTEXT].echo

end



Edited at 2008-02-21 09:31 am (UTC) Reply ) ( Thread