Oct 14, 2011

this is the third entry in an n-part series explaining the compilation techniques of Clojure.1

There was an interesting discussion about invokedynamic on the Clojure mailing list focused on the need for and potential benefits of invokedynamic. Granted this topic is often quite technical, so I suppose that it’s understandable that confusion and disagreement would arise. However, the general tone of the thread and the ensuing Internet discussion was that Clojure didn’t need invokedynamic. This is technically true, but there is a distinct difference between need and benefit. This post will hopefully clarify the current state of affairs regarding just how invokedynamic can benefit, and also hurt, Clojure on the JVM.

Why Clojure might not need invokedynamic

Tal Liron provided a very nice summary of the reasons that invokedynamic might be unnecessary for Clojure. I highly suggest you read his assessment as I will only summarize some high points here and in the next section, offering corrections where his thesis differs from the realities of the invokedynamic story for Clojure.

Java Interop

Let’s take a look at the first potential use of invokedynamic for Clojure, interop calls. As you may know, Clojure allows one to call Java class methods directly as below:

(defn at [s] (.charAt s 1)) (at "abc" 1) ;=> \b

However, the call above is problematic in that the compiler cannot resolve the class of the charAt method so an expensive route through reflection must take place at runtime. Unfortunately for us, though some common paths through reflection are indeed optimized by the JVM HotSpot, we wouldn’t want to hang our hat on that fact. Would it be possible therefore to use invokedynamic instead of the reflective call? Probably. However, it’s worth noting that through compile-time type inferencing, most interop calls are emitted in the most efficient ways possible. In fact, most of the interop calls in Clojure are compiled in the same way that Java itself is compiled. For those times as above when the Clojure compiler in unable to infer the correct target type, then Clojure provides type hinting to help the compiler along:

(defn at [^String s] (.charAt s 1))

Like many Lisps, Clojure is built around the principle that one should write the code correctly first and then only add adornments later when speed is needed. That is not to say that there is no place for a scenario whereby invokedynamic is used instead of reflection in the non-inferred case, but the presence of type hinting makes that scenario less than urgent and definitely not compelling enough to consider it a worthwhile venture on its own. I would imagine that time would be better spent implementing more comprehensive type inferencing. This is a clear win that would benefit many codebases quickly, with none of the downsides listed in the last section of this post.

Multimethods

Clojure has a multimethod function type that provides runtime dynamic dispatch of a named function to a set of unique implementations based on the result of a separate dispatch function. A simple example is below:

(defmulti say-count count) (defmethod say-count 0 [_] "Zero") (defmethod say-count 1 [_] "One") (defmethod say-count :default [_] "A Bunch") (say-count [1 2 3]) ;=> A Bunch

As you might see, the multimethod say-count is defined to dispatch based on the result of the interceding function count . When count returns 0 the specific method associated with that value is invoked. For every call to a multimethod2 its dispatch method is called. This is a potential bottleneck, but what you give up in speed you gain in flexibility. But does it need to be this way? Can invokedynamic help alliviate this tradeoff? The answer as it turns out is similar to interop’s answer. How invokedynamic can help multimethods escapes me, although they do have a PIC-like doo-dad (technical term) under the hood — so the jury is still out here.

How invokedynamic can help Clojure

a nice source of core knowledge on this section is this treatment of dynamic invocation on the JVM by John Rose

Before any discussion about how invokedynamic can help begins, it’s critical to understand what it provides. In a nutshell, invokedynamic provides the raw material for building efficient polymorphic inline caches that are subject to finer grained HotSpot optimizations. At the moment Clojure and JRuby (and others for sure) build those PICs from “something else” (technical term). However, as I will discuss regarding Vars next, invokedynamic’s benefits are not limited to the case where one might find a PIC.

Much of the following is covered by the eminent Paul Stadig in the original thread, but I’ll provide an overview below.

def

At the core of Clojure’s dynamic heart is the Var. At it’s most boiled (i.e. before taking into account dynamic or thread-local bindings), a Var is a single point of mutability that holds a value or function. Clojure as a Lisp is predicated on the ability to change a function definition at any time. In production this explicit function redefinition is not used very much, but at the REPL, Clojure’s interactive console, this capability is used to great effect in incrementally building a solution and in-place sanity checking (i.e. REPL-based testing).

To provide this level of flexibility Clojure establishes a level of indirection.3 Specifically, all function lookups through a Var occur, at the lowest level, through an atomic volatile.4 This happens every time that a function bound using the def / defn special forms is called. This indirection is not amenable to HotSpot optimizations.

This is a great place for invokedynamic, especially during production scenarios where the root value of Vars remain stable. That is, invokedynamic would eliminate the volatile lookup on every call and likely lead to HotSpot inlining. From another perspective, the JVM provides a way to change out class implementations on the fly via something called safepoints. This implementation swapping is analogous, if not a mirror of, the swapping of Var root bindings. Safepoints are intuitively viewed as stable execution points where interrupts can be utilized to save JVM state, thus allowing magic (technical term) to happen safely and atomically. At the moment, invokedynamic is the only path to safepoints for the JVM language implementor. It would be awesome (technical term) to have a direct path to safepoints, but I digress.

protocols

Clojure’s protocols are polymorphic on the type of the first argument. The protocol functions are call-site cached (with no per-call lookup cost if the target class remains stable). In other words, the implementation of Clojure’s protocols are built on polymorphic inline caches. This is a clear win in using invokedynamic. This fact was, IMO, the biggest omission from the apparent motivation behind the Clojure mailing list thread.5

How invokedynamic can hurt Clojure

In all of the invokedynamic discussions that I have read there is little to no attention paid to the potential downsides. This is symptomatic of programming discussions in general that seem to always focus on the gain of some technology, and rarely, if ever on the costs. As expected, the discussions around invokedynamic tend toward a benefit-benefit analysis, but there are some serious questions that work to bring pause to its adoption in Clojure.

Issue #0 – Splitting the compiler

The Clojure developers work very hard to ensure that the compilation target operates on any Java5, Java6, and Java7 compatible JVM. However, to utilize invokedynamic would call for one of the following two scenarios:

Break compatibility with Java5 and Java6 and target Java7 only The adoption curve for Java7 is shaped like a gigantic question mark at the moment Split the compiler codebase into two branches targeting the invokedynamic and non-invokedynamic cases

Option #1 is probably out of the question, but like anything there may be overwhelming advantages to doing so that I’m just not seeing (although I doubt it). JRuby could possibly take this approach since in general Ruby devs are pretty adventurous with versioning,6 but I suspect that they will take option #2 instead. Option #2 is the practical choice, but it’s still not one to take lightly. Any compiler is complex and to maintain and enhance one is a tremendous effort. The effort to maintain two disparate branches is not linear in its complexity, but more likely geometric.7

Issue #1 – Speed

Charles Nutter, my personal invokedynamic guru, recently ran some speed tests for constant lookup in JRuby using various mixtures. For some measurement of speediness, the results were as follows:

Plain old JRuby: 2

JRuby with current invokedynamic: 37

JRuby with a dev build of HotSpot: 0.5

So it definitely seems that the future looks bright. However, clearly the current implementation falls very short of the speed ideal set by JRuby and Clojure’s current “manual” PIC implementations. Will this enormous speed gap persist? If so, then for how long? What are the downsides of the faster development HotSpot? These are all legitimate questions that have no definitive answers. Granted if the dev HotSpot gets rolled in and lives up to its potential, then this issue probably goes away. However, the future might look bright, but a problem with the future is that it comes when it’s ready and never before, no matter how much we want it to or might need it. A bright future is not motivating enough to split the compiler into separate codebases.

Issue #2 – What is the cost?

As discussed there is a (potential) speed and maintenance cost to using invokedynamic in its current manifestation. However, there is potentially also a size cost as well. That is, in Mr. Nutter’s aforementioned post it is unclear what supporting code is required to support the observed speed improvements. When asked, Mr. Nutter seemed to dance around the issue. I am not trying to imply any impropriety in any way. Instead, the likely reason is that for JRuby the speed gain is the motivating factor for including invokedynamic and a tradeoff in code size is deemed acceptable. It could also mean that the size is not problematic, but from the perspective of Clojure adoption, it would factor into a final decision. Aside from crass code size considerations, there are also unknown complexities that might arise from the invokedynamic interface itself, but I must be honest that the subtleties surrounding them tend to fly over my head.8

Issue #3 – Java does not consume invokedynamic

In the original thread on the Clojure group, Tal Liron aptly states:

one of the challenges of using invokedynamic is that the Java language does not support it

The final, and in my opinion the most insidious, factor is that at the moment Java itself is in no way a consumer of invokedynamic. That is not to say that there are no potential benefits to be gained, only that the primary audience for the feature are external languages. Without pulling out the tin-foil hat, it is fair to posit that if given limited resources, the choice to improve Java or improve invokedynamic Oracle will almost always spell doom for the latter. As it stands, invokedynamic (like anything else I suppose) is subject to marketing pressures in addition to technical details. Therefore, Oracle’s approach is likely to follow a standard cost-benefit analysis; just as other marketing strategies. This is in no way meant to take away from the Herculean effort spent by those internal to Sun/Oracle in the design and development of invokedynamic. However, if it appears that the inclusion and maintenance of invokedynamic is a net loss from a management perspective, then what does that mean for languages like JRuby and Clojure? No one knows of course, but it’s a valid question and a valid concern.

Issue #4 – Time

Time spent on invokedynamic for the Clojure/core team is time not spent on something else (e.g. fork-join). However, in **no way* should that dissuade some brilliant Clojure community member from taking on invokedynamic in the ways mentioned (and those not mentioned) above. Clojure will become stronger based not solely on the efforts of Clojure/core (nor should it), but on the effort of the community at large. Do you have great ideas for invokedynamic? We would love to see them.

Finis

For those of you still awake, I wish that I could send you a cookie. You deserve it. For any new and exciting technology it’s natural for a level of excitement to follow. However, this post is meant to provide a perspective that mixes the good with the bad, or more appropriately the benefit with the cost. I hope that this post provides the canonical perspective on invokedynamic from a Clojure perspective. If you see something that doesn’t seems kosher then please comment below.

thanks to Rich Hickey for reading a draft of this post

:F