p5 on the JVM

From: Ben Evans

Date: August 21, 2009 00:23

Subject: p5 on the JVM

Message ID: 38a53eb30908210023w3743b901ydc364017286ab89a@mail.gmail.com

August 21, 2009 00:23p5 on the JVM

Hi, As quite a few of you are aware, a few of us have been working on approaches to putting perl5 on top of the JVM. We've decided to call it quits, but I didn't want to just disappear without explaining a little. First off, it was a very enjoyable exercise (and very informative for those of us working on it - just sorry we couldn't provide more useful artifacts for people not on the project). Secondly, we would strongly recommend that anyone who feels tempted to give this another go read some of our meagre outputs, and perhaps contact one of us. Thirdly, in many ways, I was surprised by how much could be done. Code generation is eminently possible, and while far from perfect, the JVM really is changing to fit the needs of more dynamic languages, (eg the forthcoming version 3 of the JVM spec is now totally decoupled from the Java language - the only linkage they are supposed to share is the .class format) and a lot of smart people are working hard to make that happen. The problem with Perl is parsing. In particular: Nullability - http://www.perlmonks.org/?node_id=663393 - I'm not sure I've fully grasped the full details of Jeffrey's argument as it related to Halting yet, but the basic point about nullability as discussed by him (and originally, I believe, Randal) stands as a big parsing problem. This combination of the language features - "does not require function prototypes" and "does not require brackets round function-call arguments" is not a happy one, from the point of view of static analysis. In my view, Perl further complicates this with the form of LIST which does not (always) require brackets round it, and possibly the comma operator's behaviour, but that's somewhat off topic. Lack of clean separation between parsing phases. This is more a problem for the automated parser-generator tools, but it does hugely increase the amount of work required to build an alternative implementation of Perl 5. If we can't really meaningfully use the existing (and extremely powerful) tools, then we're back to hand porting the parser components from C to Java (or bytecode). That's a huge barrier to entry. It's not insurmountable, the Ruby people did it, but it was a lot of work for them, and their community was, shall we say, somewhat more amenable to there being multiple implementations. Existing syntax missteps - Indirect object syntax, lack of clarity of BLOCK vs SUBREF vs closure and grep/map syntax all deserve a mention here. Now, I realise that all of these are too deeply ingrained to be fixed, so I'm not proposing that we do. I'm just pointing out how these language features, some of which have much broader use cases than the problem they were intended to solve, interact to basically make non-heuristic parsing and all forms of static analysis (that I can think of) basically impossible. So, those are all the reasons we stopped - too much would have to change in order to make a different implementation possible, and we felt that there would be too much ill-feeling generated if we were to seriously propose it. I guess the next question is, does it matter? To me, I think it does. You may, of course, disagree, but here's why I think so. My interests have, for quite a while now, been in languages which are safe enough for day to day use by developers of average ability, but with features that should provoke curiousity and which contain a very large amount of expressive power if one knows how to use it. For many years, that language was Perl for me, and I find it sad that just as a new generation of developers discover functional aspects, and as improvements in chip design force us to reconsider parallelism abstractions - and I firmly believe that the thread paradigm is fundamentally *not* safe enough for the avergae developer to be using on a day-day basis - that Perl's relevance in that arena should be fading, due to being trapped in its' implementation. So there we are. Thanks to everyone who helped out - James Laver, Ovid, Shevek and co. Ben



