To protect our apps from man-in-the-middle attacks one of the first things that usually springs to mind is certificate pinning. Indeed, in early 2017 I published an article that discusses implementing SSL Pinning on Android.

At the time little did I know that in late 2017 Google were to announce that Chrome 68 would deprecate support for HTTP public key pinning (HPKP). Chrome 68 was released on 24 July 2018.

The issues of certificate pinning are numerous. Firstly deciding on a reliable set of keys to pin against is tough. Once you made that decision if your expectations don’t match reality your users suffer from not being able to access your app or website. Smashing Magazine learnt about this the hard way in late 2016 when they blocked users access for up to a year because of a mismatch between the pins and the certificates. On mobile fixing an invalid pin means pushing out a new version of an app which can still take a while to reach every user.

But pinning is terrible — and harms the ecosystem more than helps, as we’ve seen. It was a bad thing to standardize — Ryan Sleevi (Chromium developer)

So with certificate pinning falling out of favour, what should you do? The new kid in town is certificate transparency.

If you don’t like reading watch my talk from Londroid:

What is Certificate Transparency?

Certificate Transparency helps eliminate these flaws by providing an open framework for monitoring and auditing SSL certificates in nearly real time. Specifically, Certificate Transparency makes it possible to detect SSL certificates that have been mistakenly issued by a certificate authority or maliciously acquired from an otherwise unimpeachable certificate authority. It also makes it possible to identify certificate authorities that have gone rogue and are maliciously issuing certificates. — https://www.certificate-transparency.org

Certificate transparency works by having a network of publicly accessible log servers that provide cryptographic evidence when a certificate authority issues new certificates for any domain. These log servers can then be monitored to look out for suspicious certificates as well as audited to prove the logs are working as expected.

These log servers help achieve the three main goals:

Make it hard to issue certificates without the domain owners knowledge

Provide auditing and monitoring to spot mis-issued certificates

Protect users from mis-issued certificates

When you submit a certificate to a log server, the server responds with a signed certificate timestamp (SCT), which is a promise that the certificate will be added to the logs within 24 hours (the maximum merge delay). User agents, such as web browsers and mobile apps, use this SCT to verify the validity of a domain.

For an overview, please watch this excellent video from Networking @Scale 2017:

The Very Best of Certificate Transparency (2011-) by Al Cutter & Kat Joyce (Google)

Implementing Certificate Transparency

Monitor your domains

The most critical part of certificate transparency is to be alerted to the issuance of new certificates so you can spot mis-issuance promptly. One tool for such monitoring is the open source ct_advisor hosted at https://ctadvisor.lolware.net. Another option is Facebook’s Certificate Transparency Monitoring tool which additionally can help you spot certificates created on domains trying to spoof yours.

There are also tools such as Comodo’s Certificate Search tool that make it easy to search for your certificates in the public log files to help verify your expectations.

Verifying signed certificate timestamps

The simplest way to improve security for your users using certificate transparency is to confirm that SSL certificates used in a secure connection have a reasonable number of SCTs with valid signatures associated.

To verify the signature of an SCT, you need a list of trusted log servers along with their public keys.

There are three ways to get a list of SCTs for an SSL connection.

The most common is to embed the SCT in the certificate as an X.509v3 extension. The certificate authority generates a pre-certificate, submits this to a log server to create an SCT and uses both of these to issue the end certificate with an SCT. Using X.509v3 extensions to provide SCTs requires no changes to your backend server with changes implemented at the certificate authority.

X.509v3 extension

The next most common is to provide the SCT as part of the TLS handshake. In this instance, the server operator must submit their domain to a log server to generate the SCT and requires their server to support adding SCTs as a TLS extension.

TLS extension

The least popular is providing the SCT as an Online Certificate Status Protocol (OCSP) extension through OCSP stapling. In this scenario when a client connects the server must make an OCSP request to the certificate authority to retrieve the SCT before sending it to the client. Again, as with TLS extensions, the server needs to support this functionality explicitly.

OCSP stapling

The specification states that “TLS clients MUST implement all three mechanisms”, however, where over 80% of the top 1-million websites provide SCTs only through X.509v3 extensions plus a little over 19% through just TLS extensions, by verifying the SCTs in the certificate chain alone, we cover the majority of use-cases.

Once the SCTs are retrieved, verification is then as simple as verifying its signature using the associated log servers public key extracted from a log list. TLS clients trust the connection if it provides a certain number of valid SCTs. Chrome’s policy ensures a minimum of 2 valid SCTs extending to 5 based on certificate lifetime. Apple’s policy is very similar as it is currently heavily based on Chrome’s policy along with the list of trusted log servers.

Certificate Transparency on Android

Before this article, the main projects available to help implement certificate transparency on Android were Conscrypt and certificate-transparency-java libraries.

Conscrypt

Google provides the Conscrypt Java Security Provider for Android that contains code to verify signed certificate timestamps, however with little to no documentation and only a partial implementation (as of October 2018) it’s just too hacky to get it working. You have to:

Create your own CTLogStore to load up your provided log list as the built-in files the library relies upon don’t exist

to load up your provided log list as the built-in files the library relies upon don’t exist Create your own CTPolicy as the built-in CTPolicyImpl is hidden

as the built-in is hidden Create your own TrustManager , using the libraries TrustManagerImpl , and SSLSocketFactory to provide to OkHttp

, using the libraries , and to provide to OkHttp Set various hidden security settings such as Security.setProperty("conscrypt.ct.enable", "true")

Additionally, the library depends on native libraries which doesn’t make it suitable for all projects.

certificate-transparency-java

The open source certificate-transparency-java project has no Android support, uses the deprecated Apache HttpClient, and has unnecessary dependencies on protobuf.

Introducing certificate-transparency-android

Since neither of the existing implementations is suitable for our needs at Babylon and we wanted something easy to integrate with standard networking libraries such as OkHttp we created certificate-transparency-android. The library is derived from certificate-transparency-java but modified to make it more suitable for use on Android while remaining fully compatible with use in pure Java projects.

Certificate Transparency with OkHttp

The library allows you to create a network interceptor for use with OkHttp where you specify which hosts to perform certificate transparency checks on. Wildcards are accepted but note that *.babylonhealth.com will match any sub-domain but not “babylonhealth.com” with no subdomain.

val interceptor = certificateTransparencyInterceptor {

+"*.babylonhealth.com"

}



val client = OkHttpClient.Builder().apply {

addNetworkInterceptor(interceptor)

}.build()

In Java, you can create the network interceptor through CTInterceptorBuilder .

See the GitHub project for all the available configuration options.

Certificate Transparency with Retrofit

With Retrofit built on top of OkHttp, configuring it for certificate transparency is as simple as setting up an OkHttpClient as shown above and supplying that to your Retrofit.Builder.

val retrofit = Retrofit.Builder()

.baseUrl("https://babylonhealth.com")

.addConverterFactory(GsonConverterFactory.create())

.client(okHttpClient)

.build()

Certificate Transparency with HttpURLConnection

Firstly if you are still using HttpURLConnection consider upgrading to OkHttp. The version built into Android, naturally, is a fixed version so you won’t get any security updates or bug fixes.

To use with HttpURLConnection you wrap the original hostname verifier before calling connect() on the connection:

val connection = URL("https://www.babylonhealth.com")

.openConnection() if (connection is HttpsURLConnection) {

connection.hostnameVerifier =

certificateTransparencyHostnameVerifier(

connection.hostnameVerifier

) {

+"*.babylonhealth.com"

}

}

In Java, you can create the hostname verifier through CTHostnameVerifierBuilder .

Certificate Transparency with Volley

Overriding the HostnameVerifier can be achieved by overriding createConnection when creating the RequestQueue :

val requestQueue = Volley.newRequestQueue(applicationContext,

object : HurlStack() {

override fun createConnection(url: URL): HttpURLConnection {

val connection = super.createConnection(url)

if (connection is HttpsURLConnection) {

connection.hostnameVerifier =

certificateTransparencyHostnameVerifier(

connection.hostnameVerifier

) {

+"*.babylonhealth.com"

}

}

return connection

}

})

Certificate Transparency with Apache HttpClient

As with HttpUrlConnection you really shouldn’t be using HttpClient anymore especially with the Apache HTTP Client Removal in Android 6.0. Since API 1 Android uses Apache HttpClient v4.0 and has not been updated since.

Currently, there is no support in the library for Apache HttpClient. However, adding the functionality would be relatively easy to add if there is enough demand.

Certificate Transparency in WebViews

With WebViews on Android now being provided by Chrome, hopefully in the long-term certificate transparency support will come for free. There is a proposal to add an Expect-CT header to instruct user agents to expect valid SCTs which would help enforce this.

Assuming that never happens, WebViews are tricky, not least because there is no perfect way to implement certificate transparency in them. The best you can do is override shouldInterceptRequest and implement the network calls yourself using one of the above methods. However, you can only intercept GET requests so if your WebViews use POST requests then you are out of luck.

Testing

Given you have implemented certificate transparency how do you ensure that your implementation works?

Firstly the library allows you to provide a logger which will allow you to inspect the reasons for both failures as well as successes:

val interceptor = certificateTransparencyInterceptor {

+"*.babylonhealth.com"



logger = object : Logger {

override fun log(host: String, result: VerificationResult) {

println("$host -> $result")

}

}

}

mitmproxy

Tools such as mitmproxy help with testing by allowing you to perform a man-in-the-middle proxy for HTTP and HTTPS connections. It provides an interactive console interface that allows network traffic to be intercepted, inspected, modified and replayed.

Start mitmproxy using the command below and install mitmproxy’s root cert by visiting http://mitm.it/ on your device.

mitmproxy --add-upstream-certs-to-client-chain --insecure

If when you run your app, certificate transparency causes a refused connection then you are all set. In this scenario, you also shouldn’t see your requests in the mitmproxy console window. Note that apps on Android N devices do not trust user installed certificates by default so in debug mode you have to enable this through the Network Security Configuration.

Alternative tools

Another tool worth investigating is SSLsplit — transparent SSL/TLS interception.

Playing with certificate transparency

The certificate-transparency-android library contains a sample app which provides sample code for OkHttp, HttpURLConnection and Volley along with a way to test the library against your backend.

certificate-transparency-android sample app

Conclusions

Certificate transparency helps build secure mobile apps by ensuring a client accepts only publicly logged certificates. However, it does nothing to protect against rogue certificates that were publicly logged and thus revoked when spotted as Android does not currently check revocation status.

To keep up to date with certificate transparency, please join the Google group.

#buildsecureapps