[jruby] TruffleRuby Status, start of 2017

Hello all, This year has been a real turning point for JRuby+Truffle as we continue to progress on completeness and begin to run Rails while maintaining our often order-of-magnitude performance lead over other implementations of Ruby. This email summarises what we have done, gives some exciting news about some upcoming things, and talks about what we have decided to change. JRuby+Truffle now passes 100% of the Ruby language specs - every last one of them, and more than any other alternative implementation of Ruby. We pass 96% of the core library specs, again more than any other alternative implementation. This year we have also started to run Rails. We pass 100% of the Active Support and Active Model tests, and 98% of Action Pack. Active Record, Action View and Railties basically work. We run a simple Rails blog application we have written ourselves, including installing it using bundler and running the database setup and migration. Alongside completeness we’ve maintained our performance lead on existing benchmarks and experimented with running some new benchmarks. The new optcarrot benchmark https://github.com/mame/optcarrot runs around 9x faster on JRuby+Truffle than it does on MRI (https://eregon.me/blog/2016/11/28/optcarrot.html). Until recently, a modified JVM was needed to run the Graal compiler needed by JRuby+Truffle. That’s no longer the case as the interface that Graal uses, JVMCI, is now merged into JDK 9 and is usable in the early access builds (https://github.com/jruby/jruby/wiki/Using-Graal-in-JDK-9-EA-Builds). We have now solved almost all of the hurdles originally identified by Charlie Nutter for new implementations of Ruby (http://blog.headius.com/2012/10/so-you-want-to-optimize-ruby.html), but a couple of big issues remain - support for C extensions, and interpreter startup time and baseline footprint. We’re now solving those as well. We have already shown at RubyConf how we’re solving the C extension problem with our LLVM bitcode interpreter, Sulong (https://www.youtube.com/watch?v=YLtjkP9bD_U). We are working on using this to run key C extensions such as OpenSSL and Nokogiri, and we hope that this work will enable a large number of gems to be testable that weren’t before. We’re going to start testing these gems ourselves, and aim to present a compatibility report for increasingly large proportions of the gems available on RubyGems. We have talked about how we are solving the startup time and baseline footprint problem a couple of times in the past. We are using the SubstrateVM, an ahead of time compiler for Java programs, to compile JRuby+Truffle to a single statically-linked binary, that doesn’t even have a dependency on a JVM. When compiled with the SubstrateVM we run hello-world in order of 100 ms. We hope to get this down around 10 ms, which is how fast MRI runs it. So far we have only been able to show you these numbers, but in early 2017 we hope to release the SubstrateVM to let people actually run this themselves. We think SubstrateVM could become a common way to run JRuby+Truffle, rather than using a JVM. Both of the really important pieces of the upcoming year’s work - C extensions and the SubstrateVM - require closer integration with the rest of the GraalVM ecosystem. In order to enable that and to make development easier and faster we’ve decided that we will be best off with our own repository, so we can be free to remove classic code that we aren’t using, reduce dependencies and reorganise, simplify and focus the code base. From our work preparing for the SubstrateVM there are actually already no dependencies left at all between Truffle and JRuby, and we’re more like two separate modules living in different branches of the same repository. We think it makes more sense to have them in two repositories. We also think it’s a good time to simplify the name of our project to just TruffleRuby. Our distribution won’t include JRuby any more. With the SubstrateVM, you won’t even need a JVM to run TruffleRuby and we are focusing on C extensions rather than Java extensions as a way to extend the interpreter, so TruffleRuby is not just for people running a JVM. The JRuby team provide lots of great functionality that we don’t yet, such as support for more platforms and architectures, Android, non-Oracle JDKs, JDK 7, and today you still want JRuby for all development and deployment. We’re really grateful for the code and help from the JRuby team. There is still a huge volume of code from both JRuby and Rubinius in TruffleRuby and the JRuby core team would be right to describe themselves as major contributors to TruffleRuby. The new home of the project is be https://github.com/graalvm/truffleruby In summary, this has been the year where we’ve broken through to starting to run Rails, and we’ve solved major criticisms of the project, such as requiring a custom JVM and slow startup time. Solving further issues such as C extensions require more integration into the GraalVM system and a simpler code base, and for that reason we are forking to develop in a new repository. Chris