I've been following the DigiNotar story as it evolved for a few days now with growing concern and increasing alarm.

I'm by far not privy to the inside information to be able to really assess and audit the situation, so this is purely based on what is publicly known. Being a Dutch native speaker I have access to what the press in the Netherlands writes about it with the subtle nuances that an automated translation will not capture. I do lack the resources to independently double verify everything and as such some errors might still be in it, consider this a best effort at creating some overview and leading up to conclusions with the limited information that is available.

If we do attract the attention of DigiNotar and/or Vasco: please do contact us, we'd love to talk to you and get more information!

So who is DigiNotar and what do they do when all is normal?

DigiNotar is a CA. They sell SSL certificates, also the EV kind.

But there is more that's mostly of interest to those in the EU or the Netherlands only:

They are also (I'm simplifying a bit, I know) an accredited provider in the EU and provide qualified certificates and approved SSCDs to customers to create digital signatures that -by law- in the EU are automatically considered to be qualified digital signatures and as such they are automatically equivalent to manual signatures. This status forces regular 3rd party audits against the relevant Dutch law and standards such as ETSI TS 101 456.

They also provide certificates services under the PKIOverheid umbrella in the Netherlands. This has even more and stricter rules. e.g. Things that are suggested in the ETSI standards, but not mandatory, can become mandatory for PKIOverheid.

DigiNotar is a 100% daughter company of Vasco (since Jan 2011), so if you see Vasco sometimes doing things like press releases regarding the incident, that's why.

So what do we know in a chronological order ?

Analysis of the spreadsheet published by the torproject

A list of CAs was affected and has issued rogue certificates:

DigiNotar Cyber CA

DigiNotar Extended Validation CA

DigiNotar Public CA - G2

DigiNotar Public CA 2025

Koninklijke Notariele Beroepsorganisatie CA

Stichting TTP Infos CA

Revocations:

128 revocations on July 19th

130 revocations on July 20th

75 revocations on July 27th

1 revocation in August 29th

198 with a revocation status of "unknown"

The CN list (some had multiple certificates):

*.*.com

*.*.org

*.10million.org

*.android.com

*.aol.com

*.azadegi.com

*.balatarin.com

*.comodo.com

*.digicert.com

*.globalsign.com

*.google.com

*.JanamFadayeRahbar.com

*.logmein.com

*.microsoft.com

*.mozilla.org

*.RamzShekaneBozorg.com

*.SahebeDonyayeDigital.com

*.skype.com

*.startssl.com

*.thawthe.com

*.torproject.org

*.walla.co.il

*.windowsupdate.com

*.wordpress.com

addons.mozilla.org

azadegi.com

Comodo Root CA

CyberTrust Root CA

DigiCert Root CA

Equifax Root CA

friends.walla.co.il

GlobalSign Root CA

login.live.com

login.yahoo.com

my.screenname.aol.com

secure.logmein.com

Thawte Root CA

twitter.com

Verisign Root CA

wordpress.com

www.10million.org

www.balatarin.com

www.cia.gov

www.cybertrust.com

www.Equifax.com

www.facebook.com

www.google.com

www.hamdami.com

www.mossad.gov.il

www.sis.gov.uk

www.update.microsoft.com

This spreadsheet shows revocation of 205 certificates between July 19th and the time of the interview with Jan Valcke, Operational director at Vasco where he claimed DigiNotar had revoked all but the *.google.com one that was found in the wild on the 19th.

Analysis of the Public 2025 CRL

DigiNotar claims all breaches were under the "Public 2025 Root" ref [in Dutch]. What "root" does in there is somewhat unclear to the technical inclined mind, and the "public 2025" just seems to be some sort of internal name. Let's assume they meant the fraudulent certificates all were signed by the same intermediate.

The CRL indicated in the fraudulent *.google.com certificate does indeed point in the same "public 2025" direction, so let's get that CRL:

$ wget http://service.diginotar.nl/crl/public2025/latestCRL.crl

Let's make this file human readable:

$ openssl crl -text -inform DER -in latestCRL.crl >/tmp/t

And let's verify there is indeed the Serial Number in there of the *.google.com fake certificate we found on pastebin:

$ grep -i "05e2e6a4cd09ea54d665b075fe22a256" /tmp/t Serial Number: 05E2E6A4CD09EA54D665B075FE22A256

So yes, it's revoked. Getting the other relevant lines (it means first figuring out how many, but I skip the boring part).

$ grep -i -A4 "05e2e6a4cd09ea54d665b075fe22a256" /tmp/t Serial Number: 05E2E6A4CD09EA54D665B075FE22A256 Revocation Date: Aug 29 16:59:03 2011 GMT CRL entry extensions: Invalidity Date: Aug 29 16:58:47 2011 GMT

So that checks out nicely. [One should of course check that all signatures are valid everywhere]

Unfortunately one can only see the Serial Number of the certificates revoked, not the more juicy fields like the CN or so that would allow to see what and when other (fake) certificates were revoked.

But since we have the revocation date, maybe we can see the peak where they revoked the fraudulent certificates. I know the nature of revocation and any other work in a CA/RA can be highly cyclic with huge peaks in it, and I know not to worry about any revocation as such, users loosing control over a certificate happens all the time.

So let's see revocation activity in July 2011 split out per day:

$ grep "Revocation Date:" /tmp/t | sed 's/^.*Date: //' | sed 's/..:..:.. //' |sed 's/GMT//' | sort -n | uniq -c | grep 'Jul .* 2011' 1 Jul 1 2011 3 Jul 4 2011 3 Jul 5 2011 6 Jul 6 2011 6 Jul 7 2011 1 Jul 8 2011 2 Jul 11 2011 6 Jul 14 2011 1 Jul 15 2011 1 Jul 18 2011 2 Jul 19 2011 1 Jul 20 2011 1 Jul 21 2011 3 Jul 22 2011 3 Jul 26 2011 7 Jul 28 2011 5 Jul 29 2011

Uhmm, where is the "dozens" on July 19th ?

Since the *.google.com one was made on Jul 10th, there is no dozens neither before nor shortly after the 19th.

They might have been added to another CRL, hard to say as DigiNotar does not allow directory listing and doesn't have an easy to find list of CRLs they publish either.

Still, even if we look at the "normal" workload in 2011:

$ grep "Revocation Date:" /tmp/t | sed 's/^.*Date: //' | sed 's/..:..:.. //' |sed 's/GMT//' |grep 2011| sed 's/ .. 2011//'| sort -n | uniq -c 93 Apr 34 Aug 112 Feb 144 Jan 52 Jul 18 Jun 118 Mar 118 May

We see that the Jun/Jul and Aug months are very light on revocations. [Note that August was not yet complete in GMT time when I downloaded the CRL file].

I know my sed, grep commands could be optimized to save a few CPU cycles, but this isn't a unix lesson.

I'd love to see the "dozens" of revocations around July 19th in a DigiNotar CRL, but I simply cannot find them.

[for a time we showed here that a number of SNs we had found (246 to be exact) were not included in this CRL]

It's now clear that by far not all fraudulent certificates were revoked in this CRL as the initial information from DigiNotar that all was under the public 2025 CA is simply shown to be untrue. Still it should be carefully examined that all of the known bad certificates were indeed revoked and when that was done.

So what's the known impact right now:

If you're a general Internet user: you're unlikely to see much impact, maybe you'll run into a website with a DigiNotar certificate somewhere that will now warn the certificate is not trusted anymore.

Keep your browser up to date!

The longer term impact will still have to manifest itself, and for sure breaches such as these will prompt thinking of other solutions.

If the add-ons of Mozilla were indeed attacked using a MitM approach, impact might be more widespread, but that becomes somewhat less likely.

If you really need to access a website that is using a DigiNotar SSL certificate that your browser is not trusting anymore, I'd encourage you not to ignore the warning of the browser, certainly not to add the yanked DigiNotar root certificate back in. Instead the safe procedure is to go examine the certificate and contact the website operator out of band (e.g. by phone). Make them tell you what the fingerprint is of their certificate, verify that with what you see and only then accept the certificate. If you want to be sure you're talking to the right website, you need to perform the work the 3rd party used to do for you, not blindly click OK.

Keep your browser up to date! The longer term impact will still have to manifest itself, and for sure breaches such as these will prompt thinking of other solutions. If the add-ons of Mozilla were indeed attacked using a MitM approach, impact might be more widespread, but that becomes somewhat less likely. If you really need to access a website that is using a DigiNotar SSL certificate that your browser is not trusting anymore, I'd encourage you not to ignore the warning of the browser, certainly not to add the yanked DigiNotar root certificate back in. Instead the safe procedure is to go examine the certificate and contact the website operator out of band (e.g. by phone). Make them tell you what the fingerprint is of their certificate, verify that with what you see and only then accept the certificate. If you want to be sure you're talking to the right website, you need to perform the work the 3rd party used to do for you, not blindly click OK. If you're a user in Iran, and had something to hide from your government, odds are you're in trouble with your government.

Tor users: the torproject confirms the tor network itself is not reliant on SSL certificates. Downloading their client should be done with great care, but the fraudulent certificates that DigiNotar informed them about have by now all expired on their own - revocation can't be confirmed yet.

You're a customer of DigiNotar: DigiNotar lost the trust from the browser makers and the Dutch government, how permanent that is is too soon to say, but it's a huge unprecedented dent.

Your best option is to seek another provider, if you have not done so already.

Your best option is to seek another provider, if you have not done so already. If you're a CA or RA, this is yet another big wake-up call. If you're a 3rd party auditor of said, it's the exact same thing. CAs are now a target. Trying to keep breaches such as these secret is hopefully proven to be a disastrous recipe.

What is the biggest thing we all lack to better see what impact there is/was ?

Full list of fraudulent certificates (CN, SN fields at the very least)

After the publication of the list on Sept 4th we're getting there slowly, but there are still a lot of certificates with an SN as "unknown".

After the publication of the list on Sept 4th we're getting there slowly, but there are still a lot of certificates with an SN as "unknown". Clarity on when each certificate was created and when it was revoked

After the publication of the list on Sept 4th, we're getting there slowly, but there are still a lot of certificates with a status of revocation as "unknown".

After the publication of the list on Sept 4th, we're getting there slowly, but there are still a lot of certificates with a status of revocation as "unknown". In order too keep trust in other CAs/RAs that were audited by the same auditor that missed the fraudulent *.google.com certificate as well as the defaced pages on the portal, it becomes critical for the auditor in question to speak up and reassure the world this will not repeat itself. This is not intended to publicly humiliate the auditor, but much more a matter of getting confidence back into the system. So a compromise that an unnamed auditor working for well known audit company X is now not an auditor anymore due to this incident is maybe a good start. Add in some measures and guarantees to prevent it from occurring again.

Clarity over what was affected by the hackers, a full report would be really nice to read. Special attention should be given to explain how it is sure PKIOverheid, the qualified certificates etc. are for sure not affected and how privacy of other customers e.g. was affected. Similarly the defacements should be covered in detail as well as how they could be missed for so long.

Obviously it's unlikely we'll get all those details publicly, but the more we get the easier it will be to keep the trust in the SSL "system" in general.

Trust

A CA has the function of a trusted third party. Their business at the core is based on getting and keeping that trust.

Getting hacked, being reluctant to report the hack, being slow in detecting the hack, being slow in responding to the hack, not being willing or able to provide clarity about the impact of the hack, ... all are tremendously damaging to that trust.

Getting caught spreading incorrect or incomplete information about the hack is however of a whole other magnitude in loosing trust. Compare e.g. this:

A press release on Aug 30th by DigiNotar containing a statement in Dutch " Gebleken is dat vanuit één subroot (de zogenaamde Public 2025 Root) voor een aantal domeinen ten onrechte certificaten in omloop zijn gekomen. Uit het ingestelde onderzoek is gebleken dat het hier louter om SSL certificaten en EVSSL certificaten gaat die onder deze specifieke subroot zijn uitgegeven. Andere roots zijn onaangetast gebleven, zo ook de root waaruit PKIOverheid certificaten worden uitgegeven en de subroot waaruit de DigiNotar gekwalificeerde certificaten worden uitgegeven. DigiNotar trok toen onmiddellijk alle gecompromitteerde certificaten die uit het onderzoek naar voren zijn gekomen, inwaardoor deze onbruikbaar zijn. " my (somewhat free) translation to English: "It is apparent that from one subroot (the so-called Public 2025 Root) certificates for a number of domains have unrightfully been put in circulation. From the investigation it is apparent that it is limited to SSL certificates and EVSSL certificates issued under this specific subroot. Other roots have been unaffected, such as the root from wich the PKIOverheid certificated are issued and the subroot from where the qualified DigiNotar certificates are issued. DigiNotar immediately revoked all compromised certificates revealed by the investigation, making them unusable". It then goes on to cover the missed *.google.com one.

to the analysis of the Sept 4th excel reportedly from the Dutch government that lists:

6 CAs that issued rogue certificates, including the DigiNotar Public CA 2025 one.

That the revocations happened on July 19th, 21st and 27th, and that almost 200 still have an unknown revocation status. [The rogue certificates were issued on July 10th, 18th and 20th].

With the browser vendors and the Dutch government cancelling their trust in DigiNotar and said government taking over the operational management till it can transition, the fate of DigiNotar seems sealed.

Glossary

CA: Certificate Authority

CN: Common Name, in case of a SSL certificate for a web server this contains the name of the website, can be a wildcard as well in that case.

CRL: Certificate Revocation List a machine readable list of revoked certificates, typically published over http. Contains the Serial Numbers (SN) of the revoked certificates along with some minimal supporting data.

"dozens" used in my text above is a somewhat freely translation of the Dutch "tientallen", literally, "multiple tens"

ETSI TS 101 456: A technical specification on "policy requirements for certification authorities issuing qualified certificates"; used as a norm in audits of said providers.

Can be freely downloaded from ETSI: version 1.4.3.

Can be freely downloaded from ETSI: version 1.4.3. EV: extended validation: essentially the same SSL certificate, but with a slightly stricter set of rules on issuing. Most browsers render something like the URL in the address bar in a green color when they see such a certificate

PKIOverheid: a PKI system run under very strict requirements by and for the Dutch Government. There are 7 providers recognized to deliver certificates under a root certificate held by the Dutch government. This PKI not only issues certificates to (web) servers, but also to companies and individuals to do client authentication against government websites as well as provide means to create qualified digital signatures.

RA: Registration Authority

SN: Serial Number

SSCD: Secure Signature Creation Device. Mostly a smartcard or smart USB token that holds key pairs used for signing and protects the secret keys

Update History

version 1: initial release

version 2: updated with more information from the torproject, thanks for the pointer Gary!

version 3: update to include the DigiNotar press release of Aug 30th.

version 4: update to add information from the chromium source code

version 5: update after checking the SNs found in the chromium source code if they appear in the known CRL

version 6: update with information of the Dutch government as published by the torproject

version 7: update with information that the Dutch government is stopping their trust in Diginotar and moving away from them

version 8: some clean up and additions

--

Swa Frantzen -- Section 66