Security Certificate Revocation Awareness

An Evaluation of the Effectiveness

of Chrome's CRLSets

Google tells us how broken the traditional security certificate revocation system is, that we should not use it, and that Chrome's unique CRLSet solution provides all the protection we need . . .



Does it? ZDNet's Larry Seltzer: As a Google spokesperson said to me, “...[W]hen we identify a bad cert, we update our revocation list, and Chrome users are automatically protected.” This page answers the question: Is that true? Executive Summary Eschewing the industry's standard and increasingly strong and effective solutions for enforcing the revocations of Internet security certificates, Google's Chrome web browser invented its own system known as CRLSets.

Though presented by Google as a superior solution, the data below conclusively demonstrates that CRLSets are actually quite ineffective in protecting Chrome's users.

Despite the fact that hundreds of certificate authorities are revoking and publishing updated revocation lists daily, Chrome's current CRLSet contains entries for just 53 certificate authorities. Chrome implicitly trusts all certificates revoked by all other issuers.

The Internet certificate revocation lists enumerate more than two million revoked and untrustworthy certificates. Yet Chrome's CRLSet presently lists approximately 24 thousand revoked certificates. Every other one of the more than two million is implicitly trusted by Chrome.

To obscure the fact that Chrome's CRLSet mechanism is ineffective, high-profile websites using revoked certificates must be manually revoked by listing them in a special “header” section of the CRLSet file. (In contrast, Mozilla's Firefox browser automatically blocked our revoked.grc.com site instantly, several days before Chrome's developers added a manual override.)

On May 8th, 2014, The official Certificate Authority (CA) Security Council weighed-in on Google's Chrome CRLSet effectiveness. (They're not happy either.) For Google's response to this page, among others, please see the end of this page. For Google's response to this page, among others, please see the end of this page. What is Chrome's CRLSet?



A “CRL” is a Certificate Revocation List, a list of the serial numbers of unexpired security certificates which have been revoked by their issuer and should no longer be trusted. Each issuing “certificate authority” (CA) maintains and publishes its own CRL containing the current list of certificates it once signed and vouched for, but which should no longer be trusted. CRLs are in a continual state of change as old certificates expire off the lists (no longer trusted anyway due to their age) and serial numbers of newly revoked certificates are added. The Chromium project calls its custom implementation of a CRL a “CRLSet” because it contains a carefully selected collection of revoked-certificate serial numbers published by many different certificate authorities. In other words, the CRLSet consists of a set of individually curated CA CRLs. The Chromium project defines the intent and goals for their CRLSet clearly on this page. To keep the CRLSet manageable, its size is strictly limited to 256k bytes. Consequently, an oft repeated admission by Google's engineers and spokespeople is that their CRLSet “does not contain all revoked certificates”. This page carefully examines that admission. We believe that this examination is important because the topic of certificate revocation is quite complex. In a factual vacuum sound-bites are powerful. This creates a great deal of confusion within the industry, resulting in articles such as “Chrome does certificate revocation better”, recently written by industry veteran Larry Seltzer, published April 21st, 2014 by ZDNet. We'll come back to Larry's article, and the Google spokesperson he quotes, once we've established some context for evaluating his and Google's claims. (Larry also wrote another piece following up after this page was published. See the end of this page for more.) What, exactly, does Chrome's CRLSet contain?



In order to evaluate the real world performance of Chrome's CRLSet, we need to understand exactly what CRLSets are, and how they operate. When you reach the end of this page you will understand everything about Chrome's CRLSets. Chrome's CRLSet file format is well documented, straightforward and easily understood by non-programmers. It is divided into two parts: The first part is a short JSON-format file header identifying the file version, content type, the CRLSet's sequence number, the count of certificate authority CRL sets appearing after the header, a short list of explicitly blocked certificates, and an expiration date for the data. You can see these data fields in the box below. The balance of the CRLSet consists of repeating sets of carefully curated CA CRLs. The first identifier in each set is the signature of the issuing authority's certificate whose key was used to sign each of the revoked certificates making up the balance of the set. Think of it as a parent followed by all of its children. Then a different parent and all of its children, and so on. This pattern repeats for the number of sets specified in the “NumParents” header field (which is 53 in this instance). Below is Chrome's CRLSet #1596, the first 1,000 lines of 24,275, retrieved from the “clients2.google.com” server on April 23rd, 2014. (The entire file is available, contained within our DIY Kit referred to below.) It is displayed within a scrolling window so you may browse around to explore the file's contents. { "Version":0, "ContentType":"CRLSet", "Sequence":1596, "DeltaFrom":0, "NumParents":53, "BlockedSPKIs": [ "GvVsmP8EPvkr6/9UzrtN1nolupVsgX8+bdPB5S61hME=", "PtvZrOY5uhotStBHGHEf2iPoWbL79dE31CQEXnkZ37k=", "Jdoa1Yu/z7In2HI7GFfUwY57qnQXtPnv+TZrXoafizk=", "li5LVLuYp+5dX+uWM/mR08MwDpUU2t57DU+CjHlPjoc=", "yP3cdcsb27WMB7TqhHKH9iZlndZrwQomrdm1dbOgo40=" ], "NotAfter":1398627981 } 019406d575cf285a3c2d8bbf8133e0cfae4839c99cc1815bdf244487259e7cd2 11270b1308d38971db8f728e2956c8d38bc7 11270b612ddbed52a12dfa3ab5c9317c486a 11270e0d85729204151e40e55374d140b395 11272184dfda95993c83e575142b0d209185 112723158fe05c8de8c2bb7357a23c13b9d0 11272874436692a4fbd793d737c7ba4fd494 11272f54911df5a61d9ba8d0e9837093a23a 1127367d2d5a6e0b40c12686185e6bee8bc2 11274f9fca0f134199d919bcd5c834ef4c99 11275179516d2006d1f4b6d69db7f8888550 11275928b0b00ec56b091c982884d43849a0 11275e9345cf322113a7734dbeb3812c802f 1127629afc9c0e78711b8d39482815158e04 11276feb7d814316ef9bfff72383cd70ddbe 11276ff7431188b4f2244b0f9fa5c1b8cc51 11277112e2615faf0f7b7c0680aa57b9d70b 11277235c0c4e3721ca04c546b9d0b17b9a7 112779fd75b9555ecd2bb0f39fa582cf4c33 112782f76421ed8856978276ba212d8bd86b 112783536ea08730d778bd14d7a9164ab451 112787d33c8f75e5b12dba7c68fd95d6f5e7 112787e72f3d224df439f0735dbb9c471bfa 11278e0cd72d0d031fd549f38f57b6b521d8 1127928e1ada9795875fb7ae78f78377a337 11279cff4f7c46a800f84f7dfe64febe1ed4 11279d0e8d78728756b63f5fdbe11827c453 11279fea6f39a7d0e67ec7fca60358f8dddf 1127aade5d4be26bc4629f392174c9e540fd 1127ab8541690e87629a1b932896e5ca87e5 1127acd012e79d32d096c872c61acb44385c 1127bde8c0f8015a2afcd8af67939b53044f 1127c1c465aef729cc2a53b9f6664da7f7a4 1127c8c2a337982c3be14ad5ca0ce0e6ff25 1127ca5e1e5d2afd9943606af13fca32ebdb 1127d0b9eee07a5c0f2ae14d82dfebda78da 1127e8f7e8ae2746340cfa497c0d2d3fcb2d 1127ea5a69561302be4a96d834775a6e42d5 1127f825909ab002b795e07e8425337580c3 1127fecd29259d1bff4e5509992e156cb824 051cf9fa95e40e9b83edaeda6961f6168c7879c4660172479cdd51ab03cea62b 15dab72c00000000320c 40388f43 40388f59 40388f5b 40388f67 40388f68 40388f6a 40388f73 40388f76 40388f77 40388f7a 40388f85 40388f8a 40388f8b 40388f8c 40388f8d 40388f8e 40388f8f 40388f90 40388f92 40388f95 40388f96 40388f98 40388f99 40388f9c 40388f9d 40388f9f 40388fa1 40388fa2 40388fa3 40388faa 40388fb0 40388fb9 40388fba 40388ff5 40388ff6 40388ff7 40388ff8 40388ff9 40388ffa 40388ffb 40388ffc 40388ffd 40388ffe 40388fff 40389000 40389001 40389002 40389003 40389004 40389005 40389006 40389007 40389008 40389009 4038900a 4038900b 4038900c 4038900d 4038900e 4038900f 40389010 40389011 40389012 40389013 40389014 40389015 40389016 40389017 40389018 40389019 4038901a 4038901b 4038901c 4038901d 4038901e 4038901f 40389020 40389021 40389022 40389023 40389024 40389025 40389026 40389027 40389028 40389029 4038902a 4038902b 4038902c 4038902d 4038902e 4038902f 40389030 40389031 40389032 40389033 40389034 40389035 40389036 40389037 40389038 40389039 4038903a 4038903b 4038903c 4038903d 4038903e 4038903f 40389040 40389041 40389042 40389043 40389044 40389045 40389046 40389047 40389048 40389049 4038904a 4038904b 4038904c 4038904d 4038904e 4038904f 40389050 40389051 40389052 40389053 40389054 40389055 40389056 40389057 40389058 40389059 4038905a 4038905b 4038905c 4038905d 4038905e 4038905f 40389060 40389061 40389062 40389063 40389064 40389065 40389066 40389067 40389068 40389069 4038906a 4038906b 4038906c 4038906d 4038906e 4038906f 40389070 40389071 40389072 40389073 40389074 40389075 40389076 40389077 40389078 40389079 4038907a 4038907b 4038907c 4038907d 4038907e 4038907f 40389080 40389081 40389082 40389083 40389084 40389085 40389086 40389087 40389088 40389089 4038908a 4038908b 4038908c 4038908d 4038908e 4038908f 40389090 40389091 40389092 40389093 40389094 40389095 40389096 40389097 40389098 40389099 4038909a 4038909b 4038909c 4038909d 4038909e 4038909f 403890a0 403890a1 403890a2 403890a3 403890a4 403890a5 403890a6 403890a7 403890a8 403890a9 403890aa 403890ab 403890ac 403890ad 403890ae 403890af 403890b0 403890b1 403890b2 403890b3 403890b4 403890b5 403890b6 403890b7 403890b8 403890b9 403890ba 403890bb 403890bc 403890bd 403890be 403890bf 403890c0 403890c1 403890c2 403890c3 403890c4 403890c5 403890c6 403890c7 403890c8 403890c9 403890ca 403890cb 403890cc 403890cd 403890ce 403890cf 403890d0 403890d1 403890d2 403890d3 403890d4 403890d5 403890d6 403890d7 403890d8 403890d9 403890da 403890db 403890dc 403890dd 403890de 403890df 403890e0 403890e1 403890e2 403890e3 403890e4 403890e5 403890e6 403890e7 403890e8 403890e9 403890ea 403890eb 403890ec 403890ed 403890ee 403890ef 403890f0 403890f1 403890f2 403890f3 403890f4 403890f5 403890f6 403890f7 403890f8 403890f9 403890fa 403890fb 403890fc 403890fd 403890fe 403890ff 40389100 40389101 40389102 40389103 40389104 40389105 40389106 40389107 40389108 40389109 4038910a 4038910b 4038910c 4038910d 4038910e 4038910f 40389110 40389111 40389112 40389113 4038911d 4038911e 4038911f 40389120 40389121 40389123 40389124 4038912b 4038912c 09451613534eb3495e503782f9c3951e57785f9bd95bd4756f594426e40fbdff 01000000000132825f3d92 010000000001339385b1effcd6db 01000000000133c6ebf4855d87a4 01000000000133c6f18ac50041ac 01000000000133cbe294998bdecd 010000000001356343f91435ce5f 01000000000135eebbcdb589fe85 010000000001362c710d535135f7 01000000000136c5ee63b356b43c 01000000000136eec29faadd765f 01000000000136eec328df50b27b 01000000000136eec4eb4fbeadf4 010000000001379a93eaaab2b390 01000000000137cca26bc6fb18c4 0100000000013907edfe91a009fb 0100000000013907eeafa591ada8 0100000000013907ef244a67b129 0100000000013907ef866e2c625f 0100000000013907efe8879a2df9 010000000001390cca3988bd517c 010000000001390ccb1157d28023 0100000000013954b83c8f8666e9 0100000000013958e6534c19c436 010000000001397351a7ff022656 010000000001397dd0fe52c3bc57 0100000000013a673fb5b0d8bdca 0100000000013b8bf1e79017bad0 0100000000013bb5497e1a515e69 0100000000013bbd7ce63e4d5195 0100000000013c26640466dc58cc 0100000000013c4f738409881820 0100000000013c4f7447929842a9 0100000000013c6f0ee0fe831fac 0100000000013c8ba946358f4231 0100000000013d69a499b97ae7c5 0100000000013d8d4c437ccf4563 0100000000013d8e949ed723499e 0100000000013db7115ced879634 0100000000013df5bf91e9462888 0100000000013df6a52a1681491b 0100000000013df6a63bd105d7fa 0100000000013df6bc279887cccf 0100000000013e0f2793d32f4512 0100000000013e13437867dfac9d 0100000000013e193f67f7911a4e 0100000000013e194262990b98f0 0100000000013e194570c530edee 0100000000013e1e1ff22164fe43 0100000000013e7c73897bbf8f82 0100000000013e7cde294ac08e14 0100000000013e81c199b43ae847 0100000000013ea8dac293f6d94d 0100000000013ec4d3f6cbabea8b 0100000000013ec8cd956760cc7f 0100000000013ec8d2a1af4adf17 0100000000013ed1f41163fa3659 0100000000013f148aeb5f577536 0100000000013f1c2071f32f1adf 0100000000013f1c213587e0fa72 0100000000013f378f4e7bb23d39 0100000000013f43ec169fc4aba0 0100000000013f5ee29fa6d5fdd6 0100000000013f5ee4afb3eefc15 0100000000013f5ee65e01f9c0af 0100000000013f5ee6d35b3b1c1d 0100000000013f5ee748d190f385 0100000000013f6358de63696994 0100000000013f635967611ed2a5 0100000000013f9e9507fb449919 0100000000013f9e9b5eef548a20 0100000000013fc0ce217cea86ea 0100000000013fc51f47b1a5c102 0100000000013ff33bb4fefaf1e9 0100000000013ff33d633a2d7166 010000000001402b7c312b3f9f75 010000000001402b7ca67c1d3cfe 01000000000140543a4841974bd4 0100000000014054b2f86a91b7d3 0100000000014054bf586baa9c4e 010000000001407e037546f2dffa 010000000001409c04be05e2f546 01000000000140a16f9a48745b1e 01000000000140a17ceea99bd002 01000000000140a6bbe359ce7531 01000000000140a7724b7d7833ca 01000000000140a779db4ca69854 01000000000140bf37c533d3b645 01000000000140dee6e2339edd25 0100000000014118e1b22dad6bd3 0100000000014127d3de6774ff20 010000000001412ab942efe68ed4 010000000001413b462fa506d87b 010000000001415664ff37c0ef13 010000000001416ea2f71a68a44b 010000000001416eaf6a5f32c9e2 010000000001417477d8e876f777 0100000000014198bf3100f9d2ef 01000000000141a1ea2577986d9b 01000000000141a2863817322a77 01000000000141a29b125c9be44e 01000000000141c1970ca6008a74 01000000000141da8c8420eadafa 01000000000141decac498950a4d 0100000000014208f09e5f6875e2 010000000001422ed6cc380297de 0100000000014236aba0d45f4781 010000000001425de11526689ab9 0200000000013326febf5384f271 020000000001339385ab65de37d3 02000000000133b2595738988647 020000000001350bbb79711c62d2 020000000001356343fe2a4270c7 020000000001361c42e1e08e6195 02000000000136543536a0b172b2 020000000001367e1bcf899b81dd 02000000000136c5ef97e82c129f 02000000000136eec881732ffb37 020000000001372cc14ef4a2518f 02000000000137a42dd7c891b8c9 02000000000137ebe9b5010b32f7 020000000001385c8dd47e0b3bf2 020000000001390bd0f816608e9a 020000000001390cb666f43402b5 02000000000139551b0576c8e865 020000000001397dbe30741e7ece 020000000001397dbef5ea9f181e 020000000001399b481cee43e7f7 0200000000013a07cafd8f1f2cbc 0200000000013b3e1f98272aa77f 0200000000013b41a05d69540f29 0200000000013b43e53e2468cca6 0200000000013b66ba2a101bb700 0200000000013b963ca9a9b0d54a 0200000000013bae7850eb21a744 0200000000013bd79fba80325370 0200000000013c1af86a85953a09 0200000000013c4a28e26ed895e7 0200000000013c4f4daf3e4d7ef3 0200000000013c737d27dd0c8d8c 0200000000013cad4eb822de600b 0200000000013d1bf17a69bdd14e 0200000000013d645ff1f2a334c0 0200000000013d653621d76eb3db 0200000000013d6931ec52e53051 0200000000013d8957a45bda8d5d 0200000000013dcbe8b61c8a8711 0200000000013df697ecfe4b5f48 0200000000013df698b090abb5b5 0200000000013df69a724b1916f3 0200000000013e1327597411252b 0200000000013e143f129a8bacd1 0200000000013e17effedd5c27dc 0200000000013e1917f44d75b8ab 0200000000013e191b161fa5122b 0200000000013e191c3b6478f055 0200000000013e191e4b72213501 0200000000013e1921327c8e15f1 0200000000013e1924a277ad5869 0200000000013e1925c7c0324d3b 0200000000013e1926ed30b5c283 0200000000013e23c6cca6746b2c 0200000000013e3878fc6bbde200 0200000000013e3dc898fcdc2aae 0200000000013e57ce7b9c814b04 0200000000013e58560fb47079ff 0200000000013e65f370bcceaef9 0200000000013e8afd535fb90f58 0200000000013ea3e41e55d35c1d 0200000000013ea3e5b907c2b31a 0200000000013ea477892856e19b 0200000000013ec4b25053be79e5 0200000000013ec8bcfb11e2ced9 0200000000013ed244a6d94aa341 0200000000013ef1ed99a3bd0137 0200000000013efc9cda7b565040 0200000000013f1ad8ea715da345 0200000000013f1c1fc8d0ae5edb 0200000000013f1c20c6fda6dc8a 0200000000013f1c2338ea6e70cc 0200000000013f3539130004082b 0200000000013f375f0ad4d6f8bb 0200000000013f5ee4d9b2383e57 0200000000013f5ee5629f106f43 0200000000013f5ee6d625cb14b1 0200000000013f5ee737f75e692a 0200000000013f5ee7ad5759bd2f 0200000000013f5ee95c874f3a04 0200000000013f5ee9d1f723e430 0200000000013f5eea33da7ddcbc 0200000000013f67674ea36d3d67 0200000000013f9a172b50ec4da8 0200000000013fd24c51f7d9b187 0200000000013fd4238f3ec741ad 0200000000013fe69a526b62cfd8 0200000000014006c5cfb8716869 0200000000014006c7dff63d65ee 0200000000014008ca6fca575e24 0200000000014008cb5a6ffd08f5 020000000001401bc17eba24c718 020000000001401c822a35cd5e43 020000000001402b7ef2d74328e7 0200000000014053117819860b7a 02000000000140534b5dc7343990 0200000000014053c9db6409306c 0200000000014053d66250ed450a 0200000000014053eec028645071 02000000000140541f2ea55bf601 02000000000140543010380fca2a 020000000001405435cafa1a98ab 0200000000014054449cf391b3a0 02000000000140544a4428fb2a91 0200000000014054bbda92ebadb8 0200000000014054ca98c8aeaff0 0200000000014083708d4c75207e 02000000000140a770e45901d659 02000000000140a774dd49b61119 02000000000140c07f96e5f96720 02000000000140c080cfcb8bb53f 020000000001410165472f40e7c0 0200000000014126dd006670fec8 02000000000141279b10db897488 020000000001416eb82f2aebf038 020000000001419cc32cf851df7b 02000000000141a18087bb594db0 02000000000141a263b0647749f0 02000000000141c3cf9883ffb9d7 02000000000141cba36aee89c309 02000000000141e70ddb1c4c2278 020000000001420cd2bab39b5852 02000000000142229e960175bfe6 020000000001422879a84ac9fefa 0200000000014229648decc12fbb 0200000000014229676176cc3826 02000000000144085d02a9d1587e 0c97420a5c8c51800720f65508f3c95adcbca45922811992aae6bc452f142699 7e240a4986f84ba7 0ee0d04926b8b9b693cda6f93018e1d8dbeb1240f3b75d243c47670d050f9d9d 011444d7fa7044453c6bd7a165fce70b 01eb4d84b2f2cfd2817249d4ee7fef61 0243352a12001fef31a99b0b16c30f7a 0246ed711cb180f92b0b4c03ba865fae 032f058e63abce632ee3271278361d48 033c1520fbd0c1c630cca3b55a27d3f6 03671f8ba5276f57b972aa9467eda14c 03f917eae05b2a952c45b2b728578e9e 044346af53f06e3192a7c976d140b28c 045c5c8349844af9d1f923c24f3d2c13 0474462702771ad4a63f3e999660c41c 048de9ddc53ef7ffd253f8e296046957 04bbebaf3d94526096fd5a644cb4d6a6 05f68f60c30e66e1b5de9d84b3135e93 06449fc6f92b397e24cb66933a804581 06c6f4503d93bf70d861a37af8559079 06da57a57bd63ef5be35ac43162ca1cf 071590952587be7a2b37e44e5ee361e8 07ac9be3d25475efd1a6ee5aa2cd17e9 07f7095350df13121ef00158772d73cb 098fac5de86ea51b8342bde5a5cb12d8 09fbefffbe3034cd00fa03daf5ab2990 0a419f4f9196e8804c1adceb6e29f8d6 0a9ef4166ae8a26f5c52b0014996eb9f 0af8ba45194b2b765a75edabe813c7f8 0b0b6e055947268e41c0caad185f79ea 0b15e143be1788af83916ed3967662a7 0bc4c59ae662999defd7342f8dfae0c7 0bcd8d5c6e6930335e19e16d531b48fe 0bf447c3a803284eb2758309f1ad3195 0bfd5e13fca843e73456aaf758718380 0c4930845f1765e652c5a2c4f90fc11d 0c8270de4f8731d7bdf79be97746e46d 0cc8ebea550e5223a493033cd785b5c0 0e7a1b052623137135d676497d44be87 0e7f498919a466274b7e878bdf10e464 0e9bac5262bb0ef3bc3e56a5ae112fa4 0f3ec633d8b7b3f1b3ba1438f5eedb98 0fb349cd8d26d0d803b26d8c95ca2f61 0ff2bb486a62c7db824f1bd6a940316a 1055dd788182e46f84ce68e340ceb3b9 1075c9f76bc155d6fe9f5f82abec1f89 109e7d126aeca06e4f0416ac8b3ea65f 10bb1c8947a45412da0ba22601205094 110f2ab26d4f7e69f83dd9386916e658 11469cc16f133fccf4c4ec1579d92c0f 11d34dbc3ac6f5612ec8d262393f3e35 122c16da3996271019a387ea02abcfe5 125d31f099a9119143d4cff29d19623e 12c7d823c688dba5d446628a411195ae 12c7ef9cd052b491ab9b5958ccc161c8 12e2cecea1dcc35f2ffa5c5107a6aeb2 13b90280a57432f4cfda87e124653f9a 13e164ae7a9a164d1562993db38a3556 148f1dac7a1a56aa48acd4f2739480e6 14a8fbad33fe3f7bf165ea718c04c836 14c98efb21ab557ea7037bc84bdb5d90 14d5019881267235f8971d64e8d5daa3 15b5aa9c6fef2c614152d06306a00786 15f07b07ba023d8ee8b2844b646a8421 1628ab8474af95d4ce24d39b770a7808 165b8fa08d9de04bcd931bff924d70bf 170c3af70195145fd6ed4c166d7d1584 171d0b83baed760aceddf831bbe26fa5 177f7bfdd9c1baf40ef116d1561e2fad 17bcc83bcc699ae576823e187c3ba541 17c81503006e153a868555325360e743 17d7170f6d60405333624bf669a11a0d 180a520e4b4e0b08270b3034c67c0241 180ed0ebd9838a85add8469cd85e6364 182c62111a835ebc311467b49b21c8e8 183f9c1be17539fab36aa3f0eb9ab32f 188aea9b4807ed1c0cf53ba526f358ba 18bfa63a45bac3de78708ae6f16940b8 1908986446940a2afb495c211d1a335f 199fabf08197c33b9b5825717b09f5c2 1a40283e63e05d351f2e12d8353a6b8b 1a4d4ea963f7825e850377e369e10123 1a91cc553510fe2400269702a47a4717 1a94769fe93ea351c6aeb68eebb04d6c 1aa27e98a10546489054ba35d208d1fb 1aa64d4815194d38fb4d463eabf86628 1aa8cf10f0b9a355d2e0c908d1b23c3c 1b192c9e0a318346dbdf514c3a684531 1b3d44f60a3065bc3e3990bdd06ffce2 1c119e4f12ee1ceef28c1ee5cc197735 1c61b2b17a9206ba272eb0a281107f8c 1e4d5f25f9a4427f6951f85e1ad5d462 1e62e9942fdbc7448f385541b3be93e7 1f13cb784ca161daa02772f2a9332dc2 1f3a52a9f16b937ee6be3c82bf00cb0b 2008207f05d26f5feb18e874feea89c8 2017227e137f723291d07ceea69ececd 20448b1a678ecaec722d4e0efaad0bc5 20779e46efbc82c4f3a1d5a4d8547960 21878f3deb48aa30e677a0bc5b8610e8 219435fdd5174af03587f00b84daf5a4 2203088686117bf249401138b4e2631d 2210e965e15d4e8e960efe3e3f482657 227e60247b88c9516e6d5ba89c7b758b 2356da3a5c0251f81152c87b4b18b956 235b709a64d073bc9bb01094293d214f 23e8c2ef8f7344afbcdeba72a956e4ee 2442ee9443c759b63ae97a9f07826595 250841e1712d727099064010fd562251 2513a6291085784bb2abd13103b51a19 26b434ecb6df042244604b5a3be5fded 273de69b25acc9ae9950a09bcd49747b 2785977725c666504505bb713d1b4967 27c8f76674a802b41f0c56422fee276f 282c88d34d8fbdc4b315fa18efc170d0 2857c004aa0e0ca33f20b65b03381437 28f58f4c34983056e6fd323bcdd2edba 2918a8b67ff6cf5474cbc0af03e54d3d 29a7c2d77ef0053cfa8042e5fd21a06b 29dbe551650ed26e309cd2650c8bc736 29e0811b9c9a17b96b3f29c45306a70c 2ad301eabb3048774bee7d1af8c121e9 2b857c7d258e3836ea3d83d2255200ca 2b96dc6990a46a250c958da93d8a608c 2bd63501d165548982cb047069bdd0f2 2c0ec7f77c660e3808ef5cb61c4de85c 2caff2473aa441051394a0f1d857f4d5 2cee285860ba8899dd0f648bd3967543 2cfa67777a60b42075a72075d88e8e3f 2d31d98aaa7a15db3b9015a2cd64cf44 2d4ad01dc2675ce44a80d8dd85f7e748 2e34d255e364997fa439ebe2e638721f 2f71b417c857026ccb5abae10424a21f 2f99328217a025289cc65b3703b85a8b 30664f1e6df68f387376106ecf7475be 30fba900b263d7328c4f73a22ce56749 31980603dea9dfe743b8961669e5ee2f 31ba7b863598f6501ab9b7a0488c1c35 32ebbadb708b3239693af8c9c6bf017c 33e724bde4c2f5fb1ec83ba64f4e06b2 34268631936fc0e9a5a9c7f790e9d97a 344cf23913c2c490d545c48f70928ea0 34549f318ab80416fe26a5abdf353b85 3486101d9131f0cc7bb40b75df16be65 349a9818abe3cf1dec8a3c3e02f0fbff 349d6c8fb1f01ddd160ff4f02f1b2e19 34d48b0b3e27f38f9b5a634b48dc4867 34e8dd18af32cc7b96711e94c9efe4ad 350c17924ca62dc1bc2e10299e4e2880 35223d2129d3de29dcbcc1de0e175d1b 35c8a0312cc175c142fbe17ddd4b715a 360a0df121739273e4cd2aab969d67dc 36bb17555bf3def0a5694ee085bf011b 388fbb4981188c8122d46d98018d5500 38924baca5f21b60955ba507e1c9628b 3945f8594da353b262b5c914506c5981 3947025d202eb30327f34bea804588e8 3a78099d9280945ce553366d5593f707 3ab37c7eac7aec0114359b694c92288a 3af0645a5362657c18272bf6f2fc025f 3b2333ea3936edab7ebd26395b34551a 3b7caac5aaf695041e402fd046374f69 3c1ae78bff841dc9fbf7c4a91459e125 3c8e4249ef142786e9b56601191ad424 3d09c14c080c2664e4993dac274e5411 3d2622bf1a04079f86e350a08d6b5f36 3e0ca0282ca32194290cf18238f1d949 3f56cbaafa810d003dbb585f9f6dd1bd 3f5a567c2c1635994d4bd884d930f355 40053fe7934d1e28fbfd4bc932c96d45 405d2b2b47b9c0abfad3c7e430eeac56 40c81277d901a592fbc0b771d7417db8 410fda288acf35eef957778022f6733a 426effdcdba456988e9f98667de68a34 42d6835df03b65557ab48a5fad4fa6e1 43c38ca7b6d89d43b040293c5fe53084 4488005d506b2a5c59b02f15179f73ab 44bed28abe75286a2ea0e74654ac6513 44c405edfa5d02a9742dd8532dee4676 454460c09b689b5aa765efa050884ae6 466224db2e5cec56ae1a9fb57db83969 47757909a6008c2cce0706723208e3c3 493667ea6218ceb59f3ba54240b0a49d 4994e8fcc0a121f732d21478d026af27 49dddd641ce8941ad9ab258e18ba1038 4a2ebc5e51290a3f76579f5a0bb5781c 4a38d3280a5c0c75e39f18a5ae943c33 4aab4948d7455e065802199e41b94163 4acb98faa730715965201c5de234a75f 4b4db6ee2d2ffda92ddbd1ac0cd03548 4b9c95aeabacb75e5726cf8bdb368db6 4bafe6bbaae04b640a44b56910aa9d74 4be19fedb6c8ece012bf7743c33d5f9c 4c264b11237cbfa937c9fc1af8128321 4c87795576af05327d258d061bebecf7 4da6a7a70766b0afa47bc2e08f0396c5 4da7193f58f61f769525d8eb6d249870 4daf282c20b6d47e2ac4694345eabbed 4de33e5d12a2e2550aba87986f152667 4e517525511b42dec0442eb821b09aed 4e639a56bc10a4664b2876c26df03df1 4ea50cdcd9046d04e9716945e34148c5 4f1955b52ad021965a659e90bddb80de 4f30d79a1e899e81e30bf084c9027c53 4f32804c0ed749daa1a21ae7d25aca12 4f3d13a64184151c5f97ab942510c268 4f5e826493633d0072f9e6da81cec632 4f8e55122df71469bc2fe7cccdcd1606 51434414ed379477cd91ba648bc011d7 519c628c5b8df952ba4601026464334b 51cce734ba5d2d012e1911c96b50face 52111405542fbcf12c82f12cf0c00582 526d03fa0cac900541bfbf1ffad23b16 526e1df56122a829d0ef2c4e0136474b 537beb024f9d46175bf2e1cf7575f86b 53fc64c3d81e30a461a5abd7c5504de6 5413b950e0851a56c34afd32e9db1448 546b76c3e98cc943dfdd6a8aa48b858b 54d8c910519fe20eb1064cd1e6dbc0d8 552e9df12a0a3e26d92145499f2fcdee 5536c833e99539190b632a6ccc712055 558596f0708c61acbb377e82b806012c 55e2a5605d5a91838229ff8b6729b531 55f73d3cc76679ab69fa4336e3e93a97 55ffe37a94f80ec87d0933bac3fd6363 56048fd56ae5ba2ab60e441981d7c8b9 56238baeb37cb9143d914c36bdba1f8f 5703fa0e43719d2c9eb176b5800018f1 57dce7bebd7cca21021b0cf634707ed9 57f20dde3851d0449d497afdf5e42ec3 58223457a2c1f86f88571b75309548db 584ec09d01289577a687e7b1bac1c9cc 5895462d8bb46e6f1832ab937e7ccec7 594bc76107ccdf3fe014e8b18159955a 597f365dbd78b46c5a8a613ea726b69a 59d8b2570f94d04138ddda1c1be6dd02 5a08e81c7c168e08935eb0564c0e4c22 5aa95f2d5b359df58e8574f1e1bc0b12 5bf74017b1e14f9c126dda9c70701da6 5d487986e597f44054c7841a23945c76 5ddbef0aaa802f33e7ef94fdcb9e1850 5e2b73f9d36eb346be9c77327c17477a 5e3416f143b277b9bd27d07a40323eec 5e394787dc4437cd36eb0cea892e5e23 5ed3b72bd3abadd8fdc1c63f576310d5 5ee137c48fa8befe771cd13bdb936b03 5eee69793bc721810aab1fee2e83e2a5 5f808536172ca2b40ad916eb8ae6ebbf 60195ec309123c1d703fe7c6a5e7ebf3 606f0feace4ddfae08b18037a6ed3493 609091fcf9ac0bb720b22be37851c785 60a07b5a69c7978dbe8c1d7c505a38a9 60a1a47ab7773540b0a3f1be3dfb21d8 6137fbfe99405a4c6fc03ec04428d04c 61dbe19741460b40cd8a02dbf5417e97 627f4b0b3c71ce6a19d9499b8c4567f7 62bf12a978c15673b74dcaaf3a5b7b2e 62c6f656ceb5b9f8f50c2f58c74d6196 6349fc6003aea4d2224049fd016bef4f 63f7a840e91201c81cc6e386dfe0871f 64510c9b8580f4f1ec8e335847bd742d 649e2b5c681f25c105a8f41d05e66708 65852e60b24a2fa63b8b075717987a2f 65b85cbcd9559c6a806c622447df089e 6660a0b8ff4f4a1b1285355c5474c760 66662878f61a505331f04ba5dbfed5d8 6673cfa5b13e2e048b3c6d4e2c7c9f0b 678f231bd5996a7e650705725c5e2785 67c8c0204354c409ece0b56a6f80b39a 67c97da56bbd0cd68a7d42eb1514bf7a 681f617d1babb4e70542ff2ab8b51cc1 68af86a478eed00c4284b221836dda0f 68da4b0e45510bb9aa0a5f7af760317a 68effa320fa74779c0e7270c9ccab69b 69690f90799aeb398bc63ea651874787 6a63e20e73bb4115e5d3490b82e44b9f 6b07732933eb37aeadd3c43f58b266c7 6c4c007e203d356971e2954614cb13df 6ce4b1c2477a4342f8e81768e7c1f972 6cffed95647a23b6bd802ef3f2b53692 6d9fea49db5300c1f035898b76a4a974 6db039df27841220fb7a65874f74d86d 6de5b77740d5a200b3aee50198e6ad09 6e1b5baa8ad5bc0ccf4bdd9efc0dea2d 6e268f19e88612d1a0e030cc26b60034 6e7979d57209752a982950a23dc4b481 6e7dcb4914f3a331fc60df4fd3f34438 6e9ae7a008054423dbb70f55673cf02f 6f59a96e9f1b91dc989992c4f6253a30 6f65d2bc9022a8047455c18eedd3390a 6fda53dc72b88b83e9c9e999428fd3ef 6feba3f747593431e629743308159567 701a8884729a1aa4306d27b2965128fc 70ace8db41ed7171c8422d883507a006 70e428b3987aea50eb52676ec85c1b7a 70f03cffa6f6efeae0ea8ba95455d9ae 7126f793d6b14255425cb0046b0a480a 7127d3b417582faaf3054033b91d7f55 7131c25e5c3649785bc557fce7a82c2a 724feff7b4a1e52be4b1cb20e0660e39 728e928290cb5bb8242f1d97604ec84f 72d1ceaf172c8b15c4d2022fa8444557 73203e6a488a006b9216819b40d9a1e6 732dcf309bdbec724a887c1c66d856c7 73b3cdc5e9ccfec6438dae9999c20a8f 73dee78b55e813221b835b152734bb99 74a13d08dce8b01cbd493eedc99d01a5 74aee724c459324ae2912fa1bc11944f 74bfa1f51843ceaae2b8fb585a1b76a1 74f5bc244233966c50fca313b25b35a1 7615859c410f68bdb20dad9901f0abec 765c0993515112e16684c6ec2d898f7b 76a83ce2a444ab45177d8889804895df 783e69223432b6aeca46703d3c63fb8a 78683fab09d92efef92a9952c3e5ec40 789e91750a6371c15dcaf6a1a9209f50 79706345766c0b7a0b64ea5ef365d46d 79a547780408ea77d77cccf2abf16131 79ae8fa83aee118af998bdd5bd5da1a9 7a2ead9ea031ae61d8979f512cb2ce21 7a3d3e7d78d79db4be410cd789beed2b 7a907628e707ff0893ce34e34cb7b213 7b21d32904d3dd51d1f51adbd266adda 7b3fa1123366c497895d0ff943a2af1c 7b63b271963d909b0eb1c36ddeeeb3e7 7c0a6fe56598e4d6ac639a7bc952488e 7c13ff4aadc9a53b278797d41aea5355 7c41becb222fb014fc88e13fce440fba 7c66dc1361753f734779bd6a3c89699b 7cf1af48c60c5fcec5042584888bb4a4 7d2fb015dc0572b2a2650491a2465633 7e1f2e938b13ccb77e828f395000352e 7e7105e37bdaa09b64d03f58da4608fd 7f458440c7d31dce12c233f4dd3aa34b 7f9f2084bc39a1e34d38494bc1a2778e 7fbd6c8b4ce60f42baa33597cb578cc1 7ff67c56a3055b1624a954da635d02c4 808bb3995968192c29e0ddc69cd44166 81a0b84c73057e97a5199b529d41d4df 81b26f1313208fafba04552cb9d9f3f7 81bf4e36c5ec152237635e72cf8ef0b7 821b7dd5cf27ea24049aecd9ac2acb02 826332738db15d82229610c1b895b24c 826b23c18ece07eb90b90c4ae453434d 82ac6548dbbfaf6fd3dc359e13b17845 82d6030b11fff3affa94c1e83c5af9c7 843bcdce77bf62f85027d5006df5f897 849c20fbb522f729c075122ce387ceb8 84cac522c7a7048585e60db6694f3b0b 84e7a677e5e34687c4368266d1d3a680 861535ed93decea3d9ea8a0ff59c551a 861f653c69faa71c4b7ee5969fb0b536 8647b8be37b0abbd7b9776827ecd325f 866edf67adff94f9e0a7aea1eae3e241 867487d0a5074f09d665df4130e3e306 8683f5d5c43299c550c1d3ff997083b8 872624e12a7d67d475f383c9515ad72c 87f3d84aac9aba1aedb9441de7956d26 8830f6a6ace9e7a97ba924b0a40b9a84 8853bcd7110749441f22833efb52db3a 887450f33fd07e3d679337d9946f33ab 88b0eada7375ded151e4018461cb4947 8910757c5bce2781e994649355579ffd 8921e8ade41d23ede958fb7dfad4148d 89b16d369feb37c240329f2ca8dd9827 8a1e552dabbcb3526576b054557b9861 8a2fa3a8eae7fff276ad943dbda27747 8a62699540a680ddb398904820843049 8af46d329d02743f05b49f9d08fb5575 8b3ffcbb510a715e4587ea7fa1255240 8b544f463d587bfda148c34df1e36bbd 8b8b9687ee7e8f9a51e36f5fd3ba60b1 8d14919a573ad9f35880dc8dab175e46 8dffbe074314ab4fa55e748859c2299c 8e7d0dd9914d6a504cc2c18677d10086 8e9cd992b604bb239c9bf7e7ff8d0120 8e9f582d11c88c6b3647a0183869ca4e 8ec7f560347625e2038f373786c7862a 8f40f8d6c12870c9593334ecfe9cf7c3 What do these characteristics mean for the CRLSet's effectiveness? The header indicates that the body of the file contains 53 sets of revoked certificates (NumParents:53). In other words, the entire current CRLSet contains sets of revoked certificates signed by 53 certificate authorities.

Let's look at some numbers:



Apple's site provides their list of 211 root certificate authorities which are built into and trusted by Apple products, and Microsoft's site provides their listings (on four pages) of 353 root certificate authorities built into and trusted by Microsoft products. Apple Microsoft Number of vendor-trusted CAs: 211 353 Number of CAs represented in CRLSet: -53 -53 Number of CAs missing from CRLSet: 158 300 Percentage of CAs missing from CRLSet: 75% 85% If you are using Chrome on an Apple product, which trusts the certificates signed by 211 certificate authorities, the limitations inherent in Chrome's CRLSet blinds it to ALL of the certificates which have been revoked by 158 certificate authorities trusted by Apple's products. To rephrase that: Chrome will blindly trust every revoked certificate that was originally signed by three quarters of the certificate authorities Apple trusts. If you are using Chrome on a Microsoft system, which trusts the certificates signed by 353 certificate authorities, the limitations inherent in Chrome's CRLSet blinds it to ALL of the certificates which have been revoked by 300 certificate authorities trusted by Microsoft's systems. To rephrase that: Chrome will blindly trust every revoked certificate that was originally signed by more than four fifths of the certificate authorities Microsoft trusts. Now we know that Chrome's CRLSet ignores the revocation warnings published by a large majority of the Internet's certificate issuers. How does it perform in sheer number of revoked certificates?



Examining CRLSet #1596, we find 24,259 lines of certificate identifiers, one per line. Since we know that 53 of those lines are the certificate signatures of each section's “parent,” we subtract 53 from the total body line count to obtain 24,206 revoked serial numbers contained within CRLSet #1596. But . . . compared to what? That's the question. In July, 2013, Websense harvested data from their “ThreatSeeker Intelligence Cloud” as described in detail in this Websense Security Labs Blog article titled “Digging Into Certificate Revocation Lists.” During approximately one hour of sampling the Internet for certificate traffic, Websense gathered 10,000 records. From those 10,000 records, 9,066 contained the URL of a certificate authority's online certificate revocation list (CRL). Within that set they found 819 unique URLs (there were many repeats). Here is Websense's graph of the Top 30 requested CRL URLs:

Credit: Websense blog, July 10th, 2013 Websense notes that the Globalsign CRL (the lower line on the chart above) is the most popular, having been referenced almost twice as often as most of the other top 30, not to mention the other 481 URLs that didn't make it into the Top 30 list. We'll come back to Globalsign a bit later. Websense then compiled the certificate revocation list content from 511 of the 819 URLs that responded to their queries. Given the nature of the ad hoc URL collection, and their unknown provenance, Websense was not surprised by non-responses. Other facets of Websense's research is further discussed on our OCSP Must-Staple page. What's significant here, is that by pulling from the Internet's readily available certificate revocation lists, Websense gathered records of a total of 2,232,845 revoked certificates. This means that Chrome's CRLSet, which currently lists the serial numbers of 24,206 revoked certificates, reflects only 1.08% of the revoked certificates collected by Websense in one hour.

The 2,232,845 number should probably be regarded as a worst-case high-end count. Certificates are used and revoked for purposes other than protecting web traffic. So for the sake of argument and scrupulous fairness, lets assume that only half of the revoked certificates collected by Websense are relevant to web security. That's probably very conservative, as additional data will shortly demonstrate. But it allows us to very conservatively conclude . . . Chrome's CRLSet contains at most 2% of

the Internet's currently revoked certificates,

leaving Chrome's users 98% unprotected. So that we're not basing our entire case upon a single research study, here are two additional data points from Internet monitoring companies to further characterize the limited reach of Chrome's list of 24,206 revoked serial numbers: Websense:

Starting in the year 2000, Websense has tracked the rate of certificate revocations. This chart clearly illustrates the trend:

Credit: Websense blog, July 10th, 2013 The apparent decline in 2013 is due to the fact that data for only the first five months of 2013 was available and shown. Websense notes that during those first five months, 766,451 certificate revocations were seen. A calculation of the average monthly revocation rate based upon those first five months will significantly underestimate the year's total, because the historical trend has clearly been increasing. But it can give us a working minimum estimate. 766,451 revocations in five months means an average of 153,290 revocations per month through those first five months. Extrapolating that for the year, which, again, won't account for any rate increase, gives us a low-end estimate of 1,839,482 newly published certificate revocations occurring only during 2013. Standard web certificates have a lifetime of either two or three years (24 or 36 months) though most are issued for three years to obtain a discounted price. If we assume that revocation events are uniformly (randomly) spread across the lifetime of certificates, this means that the average revoked certificate will remain on its CRL for half its lifetime, or 18 months. Then it will be expired off the list due to age. For the sake of obtaining a scrupulously fair low-end estimate, we will ignore the effects of an accelerating revocation rate which would cause more certificates to be added every month than expire. At any given time, assuming a low-end revocation rate of 153,290 revocations per month, the Internet's collected revocation lists would contain an average of 18 months of certificate revocations, or 2,759,220 listed revoked certificates. Remember that this figure assumes a constant revocation rate based upon the observed revocation rate during the first five months of 2013. We know the real number is significantly higher. From this data, even after making every conservative assumption, Chrome's list of 24,206 revoked certificates, accounts for just 0.877% of the Internet's currently published non-expired and possibly fraudulent certificates. Update: In a communication with Netcraft engineers attempting to verify our back-of-the-envelope estimates, they wrote: “In our data, from 1800 GMT yesterday, there were 2.85M revocations in CRLs we’ve fetched after seeing them in publicly-trusted certificates. This is very close to your estimated 2.76M.” — This additional data directly confirms an aggregate global CRL size of well over two million.

Starting in the year 2000, Websense has tracked the rate of certificate revocations. This chart clearly illustrates the trend: Netcraft:

Early in the Heartbleed adventure, Netcraft, a well known and respected Internet monitoring company, wondered aloud in a posting when the expected Heartbleed “tsunami” of certificate revocations was going to show up. Most sites took longer to respond than many industry observers expected, with a great many occurring the following week. Quoting form Netcraft's post: Activity on certificate revocation lists peaked at a rate of 3,900 revocations per hour on the day the Heartbleed bug was announced (Monday April 7, 2014). On a typical Monday, we would expect to see a total of around 22,000-30,000 SSL certificates being revoked over the course of the day. On the Monday that the Heartbleed bug was announced to the public, there were 29,000 revocations. On the next day (Tuesday), 33,000 certificates were revoked, followed by 32,000 on Wednesday. These were both above average, suggesting that around 5,000 certificates were revoked in direct response to the Heartbleed bug each day. Note that fewer revocations usually take place over weekends. The crucial takeaway from Netcraft's rather matter-of-fact baseline, is that they state that on a typical Monday before the Heartbleed incident, they would expect to see between 22,000 and 30,000 certificates revoked. With most certificates having either a three year (36 month) life, we'd expect the average certificate to remain on revocation lists for half that time, 18 months or 548 days. Some will expire sooner, and some later. But Chrome's CRLSet #1596 contains a grand total of 24,206 revoked certificate serial numbers. Given Netcraft's observations, that implies that Chrome's CRLSet might be able to handle a single day's Internet certificate revocations. But given the average revoked certificate CRL presence of 548 days, Chrome can only retain approximately 1/548th of the Internet's revoked CRLs. Update: In a communication with Netcraft engineers, they explained that the numbers obtained during this particular work were fetched from all of the CRLs they have access to, including some published by certificate authorities not trusted by mainstream web browsers. So this somewhat horrifying number is reduced somewhat. The remaining question to answer is: Which certificate authorities are

represented among the chosen 53? Since signatures are made from hash functions which are deliberately “one-way,” it's not possible to take a signature hash and “decode it” to recover the value that was originally hashed. The only way to determine which certificate authorities are represented in Chrome's CRLSet is to take candidate CA signing certificates, generate their signature, and see whether that value matches any of the values in Chrome's CRLSet. Let's first take a look at Globalsign:



As we saw above, during the single hour of Internet traffic capture which Websense analyzed last summer (long before anyone knew about the critical vulnerability in OpenSSL and had imagined or named “Heartbleed”), references to Globalsign's CRL were made nearly twice as often as most of the other CRL's among the Top 30, to say nothing of the remaining 481 responding CRLs in the data set. In other words, as reflected by this sample . . . Internet users are actively establishing connections secured by Globalsign's certificates much more often than any other certificate authority. In other words, the integrity of Globalsign's certificates matters to Internet users. This makes the importance of intercepting and blocking any attempted abuse of these certificates clear. As a consequence of the Heartbleed incident, hundreds of thousands of possibly-compromised certificates were revoked and replaced across the Internet. This chart, compiled by Netcraft, shows the regular daily cycle of Internet certificate revocations per hour in the days following the Heartbleed revelation:

Credit: Netcraft Security Blog, April 18th, 2014 It also shows, separately in green, the April 16th to 17th period during which the large web performance and security company, CloudFlare, which was vulnerable to Heartbleed as were so many sites, revoked all of its web certificates issued by Globalsign. To obtain an estimate of the total number of CloudFlare's certificates revoked by Globalsign, we next turn to the SANS Technology Institute's certificate revocation tracker:

Credit: SANS Technology Institute, Internet Storm Center, SSL CRL Activity monitor At the time of this writing, the SANS Internet Storm Center is monitoring the CRLs of 81 Internet certificate authorities including Globalsign's. Although this is only a subset of all trusted certificate authorities, it is perfect for our purposes, because their monitoring of fewer means that skewing of the Globalsign data will be reduced. The two data points of interest occur on April 16th and 17th. Examining the chart, we note that the midpoint of the line connecting the data for the two days falls almost exactly across the 70,000 certs-per-day scale line. This means that the value for April 16th is as far above 70,000 as the value for the 17th is below 70,000. In other words, the average of the two days is slightly below 70,000, so the total for the two days is slightly less than twice that, or 140,000. Therefore, we can confidently conclude that just short of 140,000 possibly-compromised and now revoked certificates were originally issued by Globalsign, the most heavily used certificate authority (based upon Websense's traffic analysis). What does Chrome's CRLSet have

to say about that 140,000? The Globalsign CRL in question, which was monitored by Netcraft and SANS for their reports, is http://crl.globalsign.com/gs/gsorganizationvalg2.crl. This handy page provides the intermediate CA certificate whose private key was used to sign the Organization Validated (OV) certificates whose subsequent revocations were listed in the Globalsign OV CRL. Since this certificate is the “parent” certificate of all those revoked, its signature will introduce a set of revoked serial numbers within Chrome's CRLSet. We generate any certificate's “signature” with the following concatenation of OpenSSL commands:

openssl x509 -pubkey -noout -in gsov.cer | openssl base64 -d | openssl dgst -sha256 We provide a complete DIY Crypto Kit to allow anyone to duplicate and verify this research and findings.

This extracts the Subject's Public Key Info (SPKI) record, converts it from base64 into binary, obtains its SHA256 hash and outputs the final result in hexadecimal, which is the format Chrome's CRLSet uses for each of its certificate set's headers. When this command is run against the Globalsign OV Intermediate certificate we obtain this signature: 3aaa93962bc8e2d693083244d68abad6d02c767490ff5dc86e9d10284d47eed7 Now we perform a simple text search for that string of characters in the ASCII-converted CRLSet. And it is not found. In other words . . . Following the Heartbleed incident, none of the approximately 140,000 certificates revoked by Globalsign after the confirmed possibility that they could have been compromised, are present in Chrome's CRLSet. But . . .

What we DID discover in Chrome's CRLSet

may surprise you. It certainly surprised us. First, a brief bit of history to set the stage . . .



Certificate revocation has not been receiving sufficient attention from the Internet community. And it seemed clear that the news of the “Heartbleed” vulnerability would result in a “tsunami” of revocations of possibly-compromised certificates. So the day after Heartbleed went public, work on GRC's revocation testing “https://revoked.grc.com” website began. It was secured by a certificate that had been revoked before the site was ever made public. Its intent was clearly stated: To allow users to test the “revocation awareness” of their various browsers, devices, and computing platforms. As expected, the revoked.grc.com site immediately and vividly demonstrated that “the revocation problem” has been a well-kept secret among industry insiders. The Internet's many millions of web surfers have been encouraged to believe that revocation is working and that their web browsers are protecting them from fraudulent web sites. (They're not.) Because few people understand the reality of today's broken revocation system, Chrome's users assumed that their web browser's willing display of the revoked.grc.com demonstration page must be a bug . . . and reported it as such to the Chromium developers: Issue 362696: Missing warning on revoked certificate. (For additional background, read the thread there. It's enlightening.) And sure enough, though it took a few days, we started receiving reports that Chrome was now “properly” blocking the special revoked.grc.com test site . . . as expected. Wanting to take a closer look, we grabbed a copy of Adam's Langley's CRLSet fetch & dump utility from his github repository. (This is how we originally obtained and created the complete CRLSet listing above.) Knowing how the CRLSet worked, we simply searched it for the serial number of the certificate of our now-blocked revoked.grc.com site . . . 0519bb07ad6f78216b1a001fa72223af . . . assuming, of course, that it would be there somewhere. But it wasn't! The serial number of GRC's “revoked.grc.com”

certificate wasn't in the CRLSet file anywhere. How could that be? Chrome had begun blocking access to revoked.grc.com once people obtained the updated CRLSet. Adam Langley himself, answering in the dialog following the Issue 362696 bug report, wrote: “CRLSets push 1576 (I think) contains the revocation for revoked.grc.com. (See chrome://components.)” (Adam's reference to “chrome://components” is a means to have Chrome display the versions of its various components, among them the last retrieved and loaded CRLSet.) So we had a mystery: We have Adam Langley publicly affirming that “CRLSet push 1576 contains the revocation for revoked.grc.com.” And, sure enough, after loading any CRLSet at or after #1576, Chrome knows not to allow the revoked.grc.com page to display. Yet we know how the CRLSets operate . . . and our serial number is definitely NOT listed anywhere in the file! We have Adam Langley publicly affirming that “” And, sure enough, after loading any CRLSet at or after #1576, Chrome knows not to allow the revoked.grc.com page to display. Yet we know how the CRLSets operateand our serial number is definitelylisted anywhere in the file! Was our CA, Digicert, missing the same way Globalsign was?



We decided to see whether our parent's (Digicert) intermediate certificate was present as one of the headers for a set of its revoked certificates. So just as we did with the Globalsign certificate, we ran the OpenSSL incantation on the certificate for the key that signed “revoked.grc.com”:

openssl x509 -pubkey -noout -in digi.cer | openssl base64 -d | openssl dgst -sha256

It dutifully dumped out the signature of the certificate for the key that signed “revoked.grc.com”: 7a6ad88298cba65b176ba3a7ab25ee8f90604033da6a98ac07959f566fa3ac54 And, once again, we searched the CRLSet for that string . . . and it was not present in the file. That meant that no certificates signed by Digicert's High Assurance CA-3 intermediate certificate were present in the file, including ours. But it had to be since Chrome had started blocking the site. A bunch of us in the grc.securitynow newsgroup, who had been collaborating on this research, puzzled over this intractable mystery for a day or two. Finally, one of the brighter bulbs in the box decided to pursue a wild hunch . . . which shocked us all, but instantly solved the mystery:

“revoked.grc.com” was in the CRLSet file header. { "Version":0, "ContentType":"CRLSet", "Sequence":1596, "DeltaFrom":0, "NumParents":53, "BlockedSPKIs": [ "GvVsmP8EPvkr6/9UzrtN1nolupVsgX8+bdPB5S61hME=", "PtvZrOY5uhotStBHGHEf2iPoWbL79dE31CQEXnkZ37k=", "Jdoa1Yu/z7In2HI7GFfUwY57qnQXtPnv+TZrXoafizk=", "li5LVLuYp+5dX+uWM/mR08MwDpUU2t57DU+CjHlPjoc=", cloudflarechallenge.com "yP3cdcsb27WMB7TqhHKH9iZlndZrwQomrdm1dbOgo40=" revoked.grc.com ], "NotAfter":1398627981 }

By adding one more term to the growing OpenSSL invocation, we convert its hexadecimal signature output into base64 encoding, which is the format used within the “BlockedSPKI” header block:

openssl x509 ‑pubkey ‑noout ‑in rgrc.cer | openssl base64 ‑d | openssl dgst ‑binary ‑sha256 | openssl base64 ‑e