Two months ago we looked at the state of portable authentication for GlassFish, Payara, JBoss/WildFly, WebLogic and Liberty in Java EE 7. With the exception of WebLogic 12.2.1, most servers performed pretty well, but there were still a number of bugs present.

Since then both Payara and WildFly have seen bug fixes that again reduce the number of bugs present where it concerns portable Java EE authentication. Do note that both updated servers have not had an official (supported) release yet, but pre-release resp. rc/cr builds containing those fixes can be downloaded from the vendors.

In anticipation of the final version of those Java EE 7 servers we already took a look at how they improved. The results are shown in the table below. For reference we show several older versions as well. For Payara we took the GlassFish release upon which Payara based its additional fixes, while for WildFly it's a selection of older builds. (no less than 29 builds were released for WildFly 8,9,10/EAP 7 alpha,beta) .

Running the Java EE 7 samples JASPIC tests Module Test Payara 4.1.1.161-pre GlassFish 4.1.1 WildFly 10rc5 WildFly 10rc4 WildFly 9.0.1 WildFly 8.0.0 async-authentication testBasicAsync Passed Passed Passed Passed Passed Failed basic-authentication testProtectedPageNotLoggedin Passed Passed Passed Passed Passed Passed basic-authentication testProtectedPageLoggedin Passed Passed Passed Passed Passed Passed basic-authentication testPublicPageLoggedin Passed Passed Passed Passed Passed Passed basic-authentication testPublicPageNotLoggedin Passed Passed Passed Passed Passed Passed basic-authentication testPublicAccessIsStateless Passed Passed Passed Passed Passed Passed basic-authentication testProtectedAccessIsStateless Passed Passed Passed Passed Passed Passed basic-authentication testProtectedAccessIsStateless2 Passed Passed Passed Passed Passed Passed basic-authentication testProtectedThenPublicAccessIsStateless Passed Passed Passed Passed Passed Passed custom-principal testProtectedPageLoggedin Passed Failure Passed Passed Passed Passed custom-principal testPublicPageLoggedin Passed Failure Passed Passed Passed Passed custom-principal testPublicAccessIsStateless Passed Passed Passed Passed Passed Passed custom-principal testProtectedAccessIsStateless Passed Passed Passed Passed Passed Passed custom-principal testProtectedAccessIsStateless2 Passed Passed Passed Passed Passed Passed custom-principal testProtectedThenPublicAccessIsStateless Passed Passed Passed Passed Passed Passed dispatching testBasicForwardViaProtectedResource Passed Passed Passed Passed Passed Passed dispatching testBasicForwardViaPublicResource Passed Passed Passed Passed Passed Passed dispatching testBasicIncludeViaPublicResource Passed Passed Passed Passed Passed Failure dispatching-jsf-cdi testCDIForwardViaProtectedResource Passed Passed Passed Passed Passed Passed dispatching-jsf-cdi testCDIForwardViaPublicResource Passed Passed Passed Passed Passed Passed dispatching-jsf-cdi testCDIIncludeViaPublicResource Passed Passed Passed Passed Passed Failure dispatching-jsf-cdi testJSFwithCDIForwardViaPublicResource Passed Passed Passed Passed Passed Passed dispatching-jsf-cdi testJSFwithCDIForwardViaProtectedResource Passed Passed Passed Passed Passed Passed dispatching-jsf-cdi testJSFwithCDIIncludeViaPublicResource Failure Failure Failure Failure Failure Failure dispatching-jsf-cdi testJSFForwardViaPublicResource Passed Passed Passed Passed Passed Passed dispatching-jsf-cdi testJSFForwardViaProtectedResource Passed Passed Passed Passed Passed Passed dispatching-jsf-cdi testJSFIncludeViaPublicResource Failure Failure Failure Failure Failure Failure ejb-propagation publicServletCallingProtectedEJB Passed Passed Passed Passed Passed Failure ejb-propagation protectedServletCallingProtectedEJB Passed Passed Passed Passed Passed Failure ejb-propagation publicServletCallingPublicEJBThenLogout Passed Passed Passed Passed Passed Failure ejb-propagation protectedServletCallingPublicEJB Passed Passed Passed Passed Passed Passed invoke-ejb-cdi protectedInvokeCDIFromSecureResponse Passed Passed Passed Passed Failure Failure invoke-ejb-cdi protectedInvokeCDIFromCleanSubject Passed Passed Passed Passed Passed Passed invoke-ejb-cdi protectedInvokeCDIFromValidateRequest Passed Passed Passed Passed Passed Passed invoke-ejb-cdi publicInvokeCDIFromSecureResponse Passed Passed Passed Passed Failure Failure invoke-ejb-cdi publicInvokeCDIFromValidateRequest Passed Passed Passed Passed Passed Passed invoke-ejb-cdi publicInvokeCDIFromCleanSubject Passed Passed Passed Passed Passed Passed invoke-ejb-cdi protectedInvokeEJBFromSecureResponse Passed Failure Passed Passed Failure Passed invoke-ejb-cdi protectedInvokeEJBFromCleanSubject Passed Passed Passed Passed Passed Passed invoke-ejb-cdi protectedInvokeEJBFromValidateRequest Passed Failure Passed Passed Passed Passed invoke-ejb-cdi publicInvokeEJBFromSecureResponse Passed Failure Passed Passed Failure Passed invoke-ejb-cdi publicInvokeEJBFromValidateRequest Passed Failure Passed Passed Passed Passed invoke-ejb-cdi publicInvokeEJBFromCleanSubject Passed Passed Passed Passed Passed Passed jacc-propagation callingJACCWhenAuthenticated Passed Passed Failure Failure Failure Failure jacc-propagation callingJACCWhenAuthenticated Passed Passed Failure Failure Failure Failure jacc-propagation callingJACCWhenNotAuthenticated Passed Passed Passed Passed Passed Passed lifecycle testBasicSAMMethodsCalled Passed Passed Passed Passed Failure Passed lifecycle testLogout Passed Passed Passed Passed Passed Passed register-session testJoinSessionIsOptional Passed Passed Passed Passed Passed Passed register-session testRemembersSession Passed Passed Passed Passed Passed Passed status-codes test404inResponse Passed Passed Passed Failure Failure Passed status-codes test404inResponse Passed Passed Passed Failure Failure Passed wrapping testResponseWrapping Passed Passed Passed Passed Passed Passed wrapping testRequestWrapping Passed Passed Passed Passed Passed Passed

Not shown in the table, but the absolute greatest improvement since JBoss switched to its new JASPIC implementation all the way back in WildFly 8.0.0.Alpha1 is the fact that JASPIC now finally works without the need of modifying WildFly by putting a dummy fragment in its standalone.xml file. It's not 100% perfect yet as the application archive (.war) still needs what is effectively a marker file to activate JASPIC, but this is much, much preferred over having to modify a server in order to activate a standard Java EE API that should just be there. Kudos to the JBoss team and a special thanks to Jason Greene for finally making this happen!

As can be seen, WildFly has seen many improvements over the years. Along the way a few regressions were introduced, but they were fixed again and now WildFly10rc5 is almost perfect with respect to the known bugs. Role propagation to JACC however still doesn't work. Although the usage of custom JACC providers is not that high, the test in question here uses the default provider for a rather useful query; "Can the authenticated user access a given resource?", e.g. "Can Pete access http://example.com/assets/someresource?".

The top performer as of now is Payarra, which passes all tests except for one of minor importance where a JSF based resource is included by an authentication module. As mentioned in the previous report this likely has to be fixed on the JSF side of things.

If all goes well we'll see a new beta of Liberty 9 this month which should also contain a number of fixes. The most problematic server at this moment is still WebLogic, which introduced a major regression between 12.1.3 and 12.2.1. Hopefully WebLogic will fix this regression soon. We'll repeat this test again when either of those publish their latest version.

Arjan Tijms