Concerns with JPMS spec and Jigsaw implementation

Apologies that this has turned into rather a long note. I wanted to capture in a single message the details that led me to suggest a "no" vote for proceeding at this time. I've tried to structure the note to outline the community issues that we should aim to resolve in order to reach closer consensus, and the outstanding technical issues I think remain important to draw to a conclusion in time for Proposed Final Draft. == Community Issues == (1) Importantly, several expert group (EG) members have said we are not ready for moving to Proposed Final Draft. While I understand the desire to keep the process moving along, I believe there is value in listening to the collective wisdom of the EG and respecting their opinion. It should not come as a shock that executive committee representatives talk to the EG members and support their position as to whether the specification is ready to proceed. We should only move to Proposed Final Draft when the EG reaches broad consensus that we are ready to do so, and that decision should be based upon the merit of the specification. Multiple EG members have said that we are not ready yet. e.g. David's note http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000626.html Robert's note http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000631.html Neil's note http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000633.html Tim's note http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000644.html Mark's rebuttal http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000635.html Mark's decision to proceed despite lack of consensus http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000650.html (2) The EG have offered their insight on decisions that JPMS may regret. While I understand that JPMS is not intended to be a replacement for the OSGi and JBoss module systems in place today, there is an onus on the EG to ensure that JPMS doesn't cause any 'harm' to the platform now, and in the foreseeable future. There are examples where EG members have tried to offer experience and suggestions that have not been accepted, and there are examples where EG members have arguably overreached the goals of JPMS. A number of the JSR 376 design decisions have been made in the face of experienced module system designers participating in the expert group. The EG are not just there to provide credibility for the process. e.g. Brian's call for naming conventions "The build tools can help this migration from the old world to the new one, but you must let us help." http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000677.html Tom's review of implementation effectiveness http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000623.html David's suggestion to use class loaders for separation/isolation http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000608.html David's objection to prohibiting cycles http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-February/000600.html (3) Reluctance to accept changes that support alternative implementations. The JPMS design was originally exclusively for the modularization of the platform itself. Subsequently APIs emerged that form part of the SE platform to allow applications to interact with the JPMS. While it was generally agreed that the JPMS APIs would not provide a meta-module definition for different module system implementations, requests for enhancements that allow for better support of existing, successful module systems have been refused without reasonable justification. e.g. Remi's note on adding APIs (even if not to support existing systems) http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-March/000655.html (4) Closing out issues despite EG objections. While not all objections will be fully resolved to everyone's satisfaction, I'd expect to see consensus on how to move forward, e.g. by agreement that ideas are out of scope, should be deferred, are too complex to implement, etc. especially where the EG objection is that such decisions are likely to be detrimental to the Java community. e.g. David's objection to decisions on issue resolution http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000679.html Remi's call for finding "common ground" http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-April/000668.html == Specific technical issues with the spec == I'll reiterate that I'm not expecting this JSR to design a replacement for existing module systems that already do a fine job addressing their goals. The technical issues I call out below come down to ensuring that the JPMS design 'does no harm', i.e. it should not cause undue work, surprise, problems, and confusion to developers; and ultimately potential issues for our customers. While they have all been discussed on these lists, I don't think we have reached broad consensus on their resolution, and we should try harder to find a mutual understanding. *Significant* Insufficient isolation - non-exported (private) packages in a module should be strictly an implementation detail. The fact that JPMS defaults to a single class loader per layer means private packages can cause surprising conflicts when being loaded at the same time. Automatic module names - modules' dependencies are expressed as module names, so implementers providing a standard API must all use the same module name. However, automatic module names are based upon jar file names which are implementation details. We need to decide if module names are shorthand for a set of APIs, or a specific implementation. Access restrictions break libraries doing reflection - a module must know a priori if it will be open to reflection, and know the reflector's module name. This will break dependency injection etc. *Lesser technical issues / Nice to haves* Just for the record, these items which have been discussed by the EG could have been addressed by JPMS, but I think we can proceed knowing these limitations. I'll just outline them for the record. - Lack of support for runtime cycles - Lack of support for module versioning - Non-textual module descriptor - Transitive requirements are explicitly named. That is, where a module requires another module transitively, it forms part of module definition and downstream changes to the transitively required module can cause issues. Addressing insufficient isolation can ameliorate this problem. - API not isomorphic with module description - you cannot do things with the JPMS API that can be done via binary module descriptor. e.g. the module name, package name, version string can be different in the binary description. - Lack of support for split packages - JPMS module resolution is not pluggable by alternative module system implementations. This results in awkward support to implement alternative module system rules. Layers are created and resolved according to strict JPMS rules. A layer Controller is available to create dynamic connections between runtime modules which may break JPMS rules in order to support other module system rules. In my opinion, the EC voting "yes" to proceed to Proposed Final Draft would be appropriate when the EG are ready to call for the public review because an understanding (not necessarily universal agreement) has been reached across the team. Thanks for reading! Tim Remi Forax <forax at univ-mlv.fr> wrote on 01/05/2017 23:12:20: > while i'm also disappointed by the outcome of some discussions. > I would like to know precisely what are the issues you would like us > to have a closer consensus. > > I do not like the blog post of Scott Stark mostly because a lot of > references are from 2015 so a lot things are outdated in that post. > So apart from the automatic modules, what are points you want to > discuss more ? > > regards, > Rémi > > ----- Mail original ----- > > De: "Tim Ellison" <Tim_Ellison at uk.ibm.com> > > À: "jpms-spec-experts" <jpms-spec-experts at openjdk.java.net> > > Cc: jpms-spec-comments at openjdk.java.net > > Envoyé: Vendredi 28 Avril 2017 17:48:43 > > Objet: Re: Concerns with JPMS spec and Jigsaw implementation > > > mark.reinhold at oracle.com wrote on 27/04/2017 21:02:09: > >> 2017/4/14 20:18:03 -0700, Scott Stark <sstark at redhat.com>: > >> > Via our participation in JSR 376/JPMS we have raised a number of > > concerns > >> > regarding the implementation decision in Jigsaw as well as the scope > > and > >> > consensus of the expert group efforts. We, along with other EC > > members, and > >> > EG members have compiled a document that details these concerns. These > >> > concerns are outlined in the following blog: > >> > https://developer.jboss.org/blogs/scott.stark/2017/04/14/critical- > >> > > deficiencies-in-jigsawjsr-376-java-platform-module-system-ec-member-concerns > >> > >> This document does not raise any substantive technical issues that have > > not > >> already been discussed in the JSR 376 EG. > > > > I believe that document demonstrates there is still work required to bring > > the > > community closer to an agreement on the proposed standard. > > > > Even where the technical issues have already been discussed in the EG, it > > is > > incumbent upon this group to make best efforts to understand and resolve > > any > > differences. If people feel that their argument has not been considered, > > then > > it is not surprising that frustration ensues. > > > > I am personally disappointed that the move to call the Public Review > > ballot > > was made despite explicit objection from a number of EG members. > > > >> > As it stands, Red Hat will not vote for the approval of this public > > review > >> > draft of JPMS as it is not in the best interest of the Java community. > >> > >> Acknowledged. > > > > IBM is also voting "no" which reflects our position that the JSR is not > > ready at > > this time to move beyond the Public Review stage and proceed to Proposed > > Final > > Draft. > > > > The JSR 376 Expert Group and the public have raised a number of reasonable > > issues > > and concerns with the current public review draft of the specification > > that > > warrant further discussion and resolution. We advocate work continuing > > amongst > > all members of the Expert Group to address the issues documented on the > > mailing > > lists. > > > > IBM would like to see closer consensus amongst the entire Expert Group > > before > > this specification proceeds to the next step. > > > > Regards, > > Tim > > > > > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU