Notes about Valhalla from a non-Java perspective

Hi, it seems that no one from the Scala team found enough time/interest to participate on valhalla-dev (Martin mentioned that most of the Scala team is on holidays/in a move/in recovery) yet ... I would have preferred to not get involved (bad impressions from other OpenJDK lists), but I'll just post my personal notes (which I sent to scala-internals a few weeks earlier) here directly before they fall completely out of date. Please note that it's written from a Scala development perspective, so "we" == "arbitrary group of Scala devs/users". Here are my (slightly improved) notes: ============== * On the general approach of using class file attributes with tuples of (bytecode index -> type) It's kind of funny, because that's exactly the approach I took almost 18 months earlier when I was thinking about this topic. I considered that to be quite a hack at that time and thought "if the JVM ever gets this, they will surely come up with a more principled way of doing this". * vcmp This feels a bit ad-hoc currently. I think it would be considerably more useful if they tried to come up with a design which would work across all types, and not yet-another special case. Scala's == implementation for instance is around 100 lines of code with dozens of branches to work around Java's/the JVM's idea of equality. It would be nice that if they were adding another comparison operator that they wouldn't repeat the mistakes of equals & friends. They could be on the right way, but hard to tell without looking at it more closely. My suggestion would be to have a notion of "primitive" equality which is defined as "do the most basic comparison available": Compare the bits of the underlying value, which means ... o the bits of value types, taking types into account o the bits of the reference for reference types o wrappers are unboxed o Double.NaN is equal to itself, 0.0 and -0.0 are not equal This would be pretty much be in line with our earlier debate about supporting eq on value types. If this could be encoded as a single vcmp operation, it would be a huge win. Additionally, one could also consider a corresponding operation for "semantic" equality: o Use "equals" implementation for value types o Use Java-style "==" for primitives o Wrappers are unboxed o Use "equals" implementation for references o Double.NaN is not equal to itself, 0.0 and -0.0 are equal o (Optionally) Compare arrays by comparing the element That would vastly simplify our == implementation, but that's not really the point ... it would be possible to do that, but I think priority should be to prevent vcmp from being either artificially limited to value types or ending up incoherent like the rest of equality stuff in Java. In the end, I think how to deal with primitive wrappers is still an uncharted territory. Retrofitting those wrapper classes as value boxes very likely won't work, so maybe there is more specification required on how "T=Integer" for "any T" is treated (or int in need of a box in specialized code). * No reification for reference types (Reified in the sense of "the type is available at runtime", not "it gets a specialized class".) I'm split on this. On the one side, this could give us the escape hatch for types not expressible with the new Generics, but on the other hand it would really suck because it would mean we couldn't just drop ClassTags altogether, but would need to drag them around for every type even if only references would actually need them. Additionally, the split between value types/reference types is very likely not similar to the split we would need to erase Scala's types to the JVM's level of expressiveness. It feels like they are again trying to cut corners here, trading implementation ease for additional language complexity. (As far as I remember, there were some concerns about what happens with Java's .class/getClass() when reference types are reified ... but imho, Java problems should stay Java problems and shouldn't make the JVM approach worse than necessary.) Also note, how this fits with my work on making classOf[T] ClassTag-aware. Additionally, we have already an escape hatch with the existing erased Generics, so having yet-another different style of Generics doesn't feel like the right way ... which brings us to the next point: * Reified/Erased Generics interop This seems to be a really dark corner. The draft is pretty silent on this. It currently looks like you can't have a type hierarchy were A is erased at the top but reified in a a subclass (or the other way around) ... I think this will make it very hard to use erased Generics as an escape hatch. Combined with the earlier point, I'd prefer better reified/erased interop and having reified reference types with reified Generics. Otherwise, it feels like tons of people will try to wrap random reference types in value types to get around that limitation. * No variance for value types This is a big conceptual problem with heterogeneous translation and my personal conclusion 1.5 years ago was the same: It can't really be done without a huge explosion in runtime complexity. * Poor man's typeclasses There was a short mention of "conditional methods", e.g. enabling methods only for a subset of some generic type. That smells a lot like a crippled version of typeclasses. Might be useful to watch what happens in that space. * Members on any T There seems to be a debate about whether/how "any T" should expose methods by itself. While this seems to be nicely in line with what a lot of people in Scala would like to do, I think Java will not be able to properly add constraints to T to describe required methods. They currently only have upper bounds to express those things, but even in Scala where bounds are less horrible than in Java, people have pretty much abandoned bounds based on subtyping in favor of context bounds. So even if Any/T wouldn't get any members I think we should be aware that Java's options for adding constraints are very, very primitive and probably not what we would want to use in Scala. Let's take that into account before arguing for or against this ... My conclusion regarding Scala: * Exposing two kinds of Generics to the user is highly undesirable from a language design POV. F# did the same with a much better starting point than Java (well-designed runtime support for reified Generics) and even for them I think the increase of complexity just wasn't worth the benefit. Java might not have a choice here, but I think we should make sure to not leak these "implementation details" to the user. We will have to support both kinds of Generics for interop anyway. * We should think about how we would like to see T/Any and eq/== evolve in the long-term and communicate that clearly to the Valhalla people. It's probably much more painful if they e.g. decide to have no members on any T, but decide on an incompatible constraint mechanism (which would very likely be reified in the bytecode). That way, keeping status quo and implementing member-free Any/T as a scalac fiction might work better. * We should start preparing for the variance/value type breakage. People who need variance can migrate to boxed representations easily (replace Vector[Int] with Vector[Integer] for instance), but trying to keep things as-is would mean just not supporting JVM value types, and I think that would be a terrible decision. From a type system POV, instantiating a type parameter with a value type could be seen as collapsing all the bounds of that type parameter to make it invariant * We should really think about having a scalac prototype which tries to emit the new class file format and leverage the new semantics. There is already an ASM fork out there with preliminary support, but I don't know how stable/complete that code is. Experience has shown (JRuby) that this is the most effective way to actually influence the design. That's just some short overview, I can expand on these topics as necessary, but I expect that people have read the draft proposal and the complete mailing list already so that everyone is on roughly the same level. ============== I hope this is helpful. Thanks, Simon