Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

(Thanks for all the feedback on the initial proposal [1]. Here's a revised version, which incorporates some of the suggestions received and includes a bit more advice. An implementation is already available for testing in the Jigsaw EA builds [2]. Further comments welcome!) Over time, as we've gotten closer and closer to the JDK 9 GA date, more and more developers have begun paying attention to the actual changes in this release. The strong encapsulation of JDK-internal APIs has, in particular, triggered many worried expressions of concern that code that works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of this change was given at run time in JDK 8. To help the entire ecosystem migrate to the modular Java platform at a more relaxed pace I hereby propose to allow illegal-access operations to internal APIs from code on the class path by default in JDK 9, and to disallow them in a future release. This will enable smoother application migration in the near term, yet still enable and motivate the maintainers of libraries and frameworks that use JDK-internal APIs to fix their code to use proper exported APIs. New command-line option: `--illegal-access` ------------------------------------------- The recently-introduced `--permit-illegal-access` option [3] will be replaced by a more-general option, `--illegal-access`. This option takes a single keyword parameter to specify a mode of operation, as follows: `--illegal-access=permit` This mode opens each package in each module in the run-time image to code in all unnamed modules, i.e., code on the class path, if that package existed in JDK 8. This enables both static access, i.e., by compiled bytecode, and deep reflective access, via the platform's various reflection APIs. The first reflective-access operation to any such package causes a warning to be issued, but no warnings are issued after that point. This single warning describes how to enable further warnings. This mode will be the default for JDK 9. It will be removed in a future release. `--illegal-access=warn` This mode is identical to `permit` except that a warning message is issued for each illegal reflective-access operation. This is roughly equivalent to the current `--permit-illegal-access` option. `--illegal-access=debug` This mode is identical to `warn` except both a warning message and a stack trace are issued for each illegal reflective-access operation. This is roughly equivalent to combining `--permit-illegal-access` with `-Dsun.reflect.debugModuleAccessChecks`. `--illegal-access=deny` This mode disables all illegal-access operations except for those enabled by other command-line options, e.g., `--add-opens`. This mode will become the default in a future release. When `deny` becomes the default mode then `permit` will likely remain supported for at least one release, so that developers can continue to migrate their code. The `permit`, `warn`, and `debug` modes will, over time, be removed, as will the `--illegal-access` option itself. (For launch-script compatibility the unsupported modes will most likely just be ignored, after issuing a warning to that effect.) How to prepare for the future ----------------------------- The default mode, `--illegal-access=permit`, is intended to make you aware when you have code on the class path that reflectively accesses some JDK-internal API at least once. To learn about all such accesses you can use the `warn` or `debug` modes. For each library or framework on the class path that requires illegal access you have two options: - If the component's maintainers have already released a new, fixed version that no longer uses JDK-internal APIs then you can consider upgrading to that version. - If the component still needs to be fixed then we encourage you to contact its maintainers and ask them to replace their use of JDK-internal APIs with proper exported APIs [4]. If you must continue to use a component that requires illegal access then you can eliminate the warning messages by using one or more `--add-opens` options to open just those internal packages to which access is required. To verify that your application is ready for the future, run it with `--illegal-access=deny` along with any necessary `--add-opens` options. Any remaining illegal-access errors will most likely be due to static references from compiled code to JDK-internal APIs. You can identify those by running the `jdeps` tool with the `--jdk-internals` option. (JDK 9 does not issue warnings for illegal static-access operations because that would require deep JVM changes and degrade performance.) Warning messages ---------------- The warning message issued when an illegal reflective-access operation is detected has the following form: WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM where: - $PERPETRATOR is the fully-qualified name of the type containing the code that invoked the reflective operation in question plus the code source (i.e., JAR-file path), if available, and - $VICTIM is a string that describes the member being accessed, including the fully-qualified name of the enclosing type In JDK 9's default mode, `--illegal-access=permit`, at most one of these warning messages will be issued, accompanied by additional instructive text. Here is an example, from running Jython on the current Jigsaw EA build [2]: $ java -jar jython-standalone-2.7.0.jar WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD() WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal Type "help", "copyright", "credits" or "license" for more information. >>> ^D $ If `--illegal-access=warn` is used then only warnings are displayed, with no instructive text. The run-time system makes a best-effort attempt to suppress duplicate warnings for the same $PERPETRATOR and $VICTIM. Here is an example, again running Jython: $ java --illegal-access=warn -jar jython-standalone-2.7.0.jar WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD() WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding() Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal Type "help", "copyright", "credits" or "license" for more information. >>> ^D $ Notes ----- - There is no `--illegal-access` mode that suppresses all warnings. This is intentional: It ensures that developers know that all illegal-access operations will be denied by default in a future release, at which time code that generates warnings today will fail. Warnings can be suppressed completely via one or more `--add-opens` options. - The first proposal [1] opened every package in every explicit module, rather than just the packages in modules in the run-time image, to every unnamed module. Peter Levart pointed out [5] that this could tempt developers to use internal APIs that are new in JDK 9 (e.g., `jdk.internal.misc.Unsafe`) and thus make the eventual transition from JDK 9 no less painful than that from JDK 8. This proposal thus only opens internal packages that existed in JDK 8. - This proposal will require adjustments to JEP 260, "Encapsulate Most Internal APIs" [6]. APIs that are internal to the JDK will still be strongly encapsulated from the standpoint of code in modules, whether those modules are automatic or explicit, but they will not appear to be encapsulated at run time from the standpoint of code on the class path. - This change will not magically solve every JDK 9 adoption problem. The concrete types of the built-in class loaders are still different, `rt.jar` is still gone, the layout of a system image is still not the same, and the version string still has a new format. [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html [2] http://jdk.java.net/jigsaw/ [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html [4] This will usually but not always be possible, since there are still a few critical internal APIs without exported replacements [6]. [5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html [6] http://openjdk.java.net/jeps/260