In the beginning and middle of this year we looked at how well modern Java EE servers supported portable authentication (JASPIC) in Java EE. As the end of 2016 approaches we take a third look to see how things are progressing.

Since our last time new versions of all servers have been released. Payara went from 163-beta to 164, WildFly went from 10.0 to 10.1, Liberty beta went from 2016-5 to 2016-11, WebLogic went from 12.2.1 to 12.2.1.2 and TomEE went from 7.0 to 7.0.2. We also added a new server, namely Tomcat. Tomcat was indirectly already tested via TomEE, but given the importance of standalone Tomcat we decided to put this one in explicitly. Do note that Tomcat is not a full or web profile Java EE server, so the integration tests for technologies it doesn't support (like JSF, CDI, etc) are simply omitted.

Tests were added for request.authenticate, an injected CDI request, the servlet path after a forward, isMandatory in a SAM, and finally for a SAM request to be seen by a Filter. For the first time there was also a test removed, namely including a JSF based resource from a SAM. This fails on many servers but the failure appeared to be a general JSF failure unrelated to JASPIC and/or its integration with the Java EE environment.

The results of running the latest series of JASPIC tests are shown below:

Running the Java EE 7 samples JASPIC tests Module Test Payara 164 WildFly 10.1 Liberty 2016-11 Weblogic 12.2.1.2 TomEE 7.0.2 Tomcat 8.5.6 async-authentication testBasicAsync Passed Passed Passed Passed Passed - basic-authentication testProtectedPageNotLoggedin Passed Passed Passed Passed Passed Passed basic-authentication testProtectedPageLoggedin Passed Passed Passed Passed Passed Passed basic-authentication testPublicPageLoggedin Passed Passed Passed Passed Passed Passed basic-authentication testPublicPageNotLoggedin Passed Passed Passed Passed Passed Passed basic-authentication testPublicAccessIsStateless Passed Passed Passed Passed Passed Passed basic-authentication testProtectedAccessIsStateless Passed Passed Passed Passed Passed Passed basic-authentication testProtectedAccessIsStateless2 Passed Passed Passed Passed Passed Passed basic-authentication testProtectedThenPublicAccessIsStateless Passed Passed Passed Passed Passed Passed custom-principal testProtectedPageLoggedin Passed Passed Passed Failure Passed Passed custom-principal testPublicPageLoggedin Passed Passed Passed Failure Passed Passed custom-principal testPublicAccessIsStateless Passed Passed Passed Passed Passed Passed custom-principal testProtectedAccessIsStateless Passed Passed Passed Passed Passed Passed custom-principal testProtectedAccessIsStateless2 Passed Passed Passed Passed Passed Passed custom-principal testProtectedThenPublicAccessIsStateless Passed Passed Passed Passed Passed Passed dispatching testBasicForwardViaProtectedResource Passed Passed Passed Passed Passed Passed dispatching testBasicForwardViaPublicResource Passed Passed Passed Passed Passed Passed dispatching testBasicIncludeViaPublicResource Passed Passed Passed Passed Passed Passed dispatching-jsf-cdi testCDIForwardWithRequestProtected Passed Passed Passed Passed Passed - dispatching-jsf-cdi testCDIForwardWithRequestInjectPublic Passed Passed Passed Passed Failure - dispatching-jsf-cdi testCDIForwardViaProtectedResource Passed Passed Passed Passed Passed - dispatching-jsf-cdi testCDIForwardViaPublicResource Passed Passed Passed Passed Passed - dispatching-jsf-cdi testCDIForwardWithRequestPublic Passed Passed Passed Passed Passed - dispatching-jsf-cdi testCDIForwardWithRequestInjectProtected Passed Passed Passed Passed Failure - dispatching-jsf-cdi testCDIIncludeViaPublicResource Passed Passed Passed Passed Passed - dispatching-jsf-cdi testJSFwithCDIForwardViaPublicResource Passed Passed Passed Passed Passed - dispatching-jsf-cdi testJSFwithCDIForwardViaProtectedResource Passed Passed Passed Passed Passed - dispatching-jsf-cdi testJSFForwardViaPublicResource Passed Passed Passed Passed Passed - dispatching-jsf-cdi testJSFForwardViaProtectedResource Passed Passed Passed Passed Passed - ejb-propagation publicServletCallingProtectedEJB Passed Passed Passed Failure Passed - ejb-propagation protectedServletCallingProtectedEJB Passed Passed Passed Failure Passed - ejb-propagation publicServletCallingPublicEJBThenLogout Passed Passed Passed Passed Passed - ejb-propagation protectedServletCallingPublicEJB Passed Passed Passed Passed Passed - ejb-register-session testRemembersSession Passed Passed Passed Failure Passed - ejb-register-session testRemembersSession Passed Passed Passed Failure Passed - invoke-ejb-cdi protectedInvokeCDIFromSecureResponse Passed Passed Failure Failure Passed - invoke-ejb-cdi protectedInvokeCDIFromCleanSubject Passed Passed Passed Passed Passed - invoke-ejb-cdi protectedInvokeCDIFromValidateRequest Passed Passed Failure Failure Passed - invoke-ejb-cdi publicInvokeCDIUseInjectedRequestFromValidateRequest Passed Passed Failure Failure Passed - invoke-ejb-cdi publicInvokeCDIFromSecureResponse Passed Passed Failure Failure Passed - invoke-ejb-cdi publicInvokeCDIFromValidateRequest Passed Passed Failure Failure Passed - invoke-ejb-cdi publicInvokeCDIUseInjectedRequestFromCleanSubject Passed Passed Passed Passed Passed - invoke-ejb-cdi publicInvokeCDIUseInjectedRequestFromSecureResponse Passed Passed Failure Failure Passed - invoke-ejb-cdi publicInvokeCDIFromCleanSubject Passed Passed Passed Passed Passed - invoke-ejb-cdi protectedInvokeEJBFromSecureResponse Passed Passed Passed Passed Passed - invoke-ejb-cdi protectedInvokeEJBFromCleanSubject Passed Passed Passed Passed Passed - invoke-ejb-cdi protectedInvokeEJBFromValidateRequest Passed Passed Passed Passed Passed - invoke-ejb-cdi publicInvokeEJBFromSecureResponse Passed Passed Passed Passed Passed - invoke-ejb-cdi publicInvokeEJBFromValidateRequest Passed Passed Passed Passed Passed - invoke-ejb-cdi publicInvokeEJBFromCleanSubject Passed Passed Passed Passed Passed - jacc-propagation callingJACCWhenAuthenticated Passed Passed Failure Failure Failure - jacc-propagation callingJACCWhenAuthenticated Passed Passed Failure Failure Failure - jacc-propagation callingJACCWhenNotAuthenticated Passed Passed Failure Failure Failure - lifecycle testBasicSAMMethodsCalled Passed Passed Passed Passed Passed Passed lifecycle testLogout Passed Passed Passed Passed Passed Passed lifecycle testProtectedIsMandatory Passed Passed Passed Passed Passed Passed lifecycle testPublicIsNonMandatory Passed Passed Passed Passed Passed Passed programmatic-authentication testAuthenticateFailFirstTwice Passed Passed Passed Passed Passed Passed programmatic-authentication testAuthenticate Passed Passed Passed Passed Passed Passed programmatic-authentication testAuthenticateFailFirstOnce Passed Passed Passed Passed Passed Passed register-session testJoinSessionIsOptional Passed Passed Passed Failure Passed Passed register-session testRemembersSession Passed Passed Passed Failure Passed Passed register-session testJoinSessionIsOptional Passed Passed Passed Failure Passed Passed register-session testRemembersSession Passed Passed Passed Failure Passed Passed status-codes test404inResponse Passed Passed Passed Passed Passed Passed status-codes test404inResponse Passed Passed Passed Passed Passed Passed wrapping testResponseWrapping Passed Passed Passed Passed Passed Passed wrapping testDeclaredFilterResponseWrapping Passed Passed Failure Passed Passed Passed wrapping testProgrammaticFilterResponseWrapping Passed Passed Failure Passed Passed Passed wrapping testDeclaredFilterRequestWrapping Passed Passed Failure Passed Passed Passed wrapping testProgrammaticFilterRequestWrapping Passed Passed Failure Passed Passed Passed wrapping testRequestWrapping Passed Passed Passed Passed Passed Passed

As can be see we now have 3 perfectly scoring servers; Payara, WildFly and Tomcat now pass every test being thrown at them. All 3 do this for the first time, and the slightly older versions of each of them contain a variety of (small) bugs that prevents a perfect score.

Liberty still fails the CDI tests as before, but there's a hack available that can let Liberty pass these. Since we test as much as possible out of the box here this hack was not applied. JACC propagation also doesn't work, but this is because there's no JACC provider available by default.

Perhaps the most surprising result of this round is that WebLogic is not longer completely broken when it comes to JASPIC. While WebLogic 12.1.x had decent support for JASPIC, a major regression was introduced in 12.2.1.0 were the basic cases didn't work properly anymore. When the basic cases fail it's almost pointless to continue testing since in practice when an authentication itself doesn't work anymore it doesn't matter so much whether say request wrapping is supported or not. WebLogic does have major problems with the "register session" feature (which likely caused the regression with basic authentication in the first place), doesn't support setting a custom principal (a major feature of JASPIC), and has problems with propagating to EJB. WebLogic fails the same CDI and JACC tests as Liberty does. Contrary to Liberty, WebLogic -does- have a default JACC provider but it's not activated out of the box.

TomEE performs very well, but has a small hick up when an injected request is used in a forwarded resource. It also fails the JACC propagation, but this is because TomEE doesn't implement JACC at all for the Web module (only for EJB).

An important additional aspect not shown in the test table above is whether JASPIC works out of the box (like e.g. a Servlet Filter does), or whether it needs (server specific) activation of some sort.

In this regard, Liberty, WebLogic, TomEE and Tomcat need zero activation. There's a small caveat with Liberty and that's if you use the server specific group to role mapping in combination with a JASPIC SAM, all role checking fails. This is a somewhat peculiar incompatibility. All other servers don't care whether the groups that are set come from a portable JASPIC SAM or from something proprietary when doing the server specific/proprietary group to role mapping.

Payara and WildFly both need an activation of some kind. Payara needs a glassfish-web.xml where default group to role mapping is activated (or alternatively, where groups are actually mapped to roles), while WildFly needs a jboss-web.xml where the "jaspitest" security domain is set. WildFly should eventually lift this mandated activation when the new Elytron security system is introduced, which could possibly happen in WildFly 11. Payara could eventually default to default group to role mapping. WebLogic made this move before, but this is somewhat of a major change that should be done carefully if indeed planned.

Conclusion

Three servers, namely WildFly 10.1, Payara 164 and Tomcat 8.5.6 now pass all tests, but two of them (WildFly and Payara) need some kind of activation so are not perfect-perfect, but still very, very close to it. TomEE performs very well too, with only a minor regression and all the major core authentication functionality working perfectly.

Liberty and WebLogic have a bit more work left to be done, where as core features are concerned Liberty fails the request/response wrapping partially, while WebLogic fails the custom principal, register session and EJB propagation.

In a next article we'll be looking at what things look like after we applied the CDI/Weld hack for Liberty and Weblogic, and if JACC propagation works on them when we install/activate a JACC provider.

Arjan Tijms