This is the story about how I cracked 122 million* password hashes with John the Ripper and oclHashcat-plus.

Author: m3g9tr0n, Copy Editor: Thireus.

It was several months ago, when I (m3g9tr0n) saw a tweet from KoreLogic about a torrent file containing various hash lists of passwords for a total of 146 million passwords. This very big amount of password hashes at first discouraged me, as I only own a classic computer configuration with an AMD Phenom II 4 cores at 3,2 Mhz in addition to an ATI/AMD 5770 graphics card. But I really wanted to give it a try because the field of password cracking fascinates me.

The password cracking tools I used during this long trip were John the Ripper and oclHashcat-plus. This article is about cracking the provided MD5 hashes of KoreLogic only, but the same strategy was also applied to the SHA1 hashes.

Updates:

08/29/2012 – New example in the John the Ripper section: "Crack double MD5 hashes with the help of dict2hash.pl script"

script" 08/29/2012 – New download! All in one sorted and cleaned version.

Dealing with hashes...

First of all the KoreLogic torrent file file must be decompressed, it contains a folder named "hashes". Let's check the content of this folder...

root@m3g9tr0n:~/hashes$ ls longer_salts raw-md5.hashes.txt salted_with_md5 SHA1 vBulletin-v3.8.4

We will concentrate from now on the raw-md5.hashes.txt list. This file is 4.3 GB and includes 139444502 lines according to the wc utility.

root@m3g9tr0n:~/hashes$ wc -l raw-md5.hashes.txt 139444502 raw-md5.hashes.txt

As you can assume, both John the Ripper and oclHashcat-plus are not able to load this file because it is too big. For that reason, we need to split this file. Under Linux we have a nice utility called split that does this job very well:

root@m3g9tr0n:~$ split --help Usage: split [OPTION]... [INPUT [PREFIX]] Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default size is 1000 lines, and default PREFIX is `x'. With no INPUT, or when INPUT is -, read standard input. Mandatory arguments to long options are mandatory for short options too. -a, --suffix-length=N use suffixes of length N (default 2) -b, --bytes=SIZE put SIZE bytes per output file -C, --line-bytes=SIZE put at most SIZE bytes of lines per output file -d, --numeric-suffixes use numeric suffixes instead of alphabetic -l, --lines=NUMBER put NUMBER lines per output file --verbose print a diagnostic just before each output file is opened --help display this help and exit --version output version information and exit SIZE may be (or may be an integer optionally followed by) one of following: KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T, P, E, Z, Y.

We can use the --lines=NUMBER parameter to split our raw-md5.hashes.txt file.

root@m3g9tr0n:~/hashes$ split -l 3000000 raw-md5.hashes.txt part

Note that we can also split the file based on the amount of MBs by taking into consideration that each MD5 hash is 32 bytes long.

Cracking Passwords with oclHashcat-plus

I started playing with oclHashcat-plus because it contains the -remove option, which removes the hashes from the hashfile once it is cracked and it is really convenient. The only limitation oclHashcat-plus has, is the constraint on password length. In other words, it is only able to crack passwords up to 15 characters. The rules that I used for oclHashcat-plus are base64.rule, passwordspro.rule, T0XlC.rule and in some cases d3ad0ne.rule. There rules can be found directly from the oclHashcat-plus suite.

Bruteforce techniques were not my first choice. I used wordlists which I downloaded from the g0tm1lk's blogspot. You will find on g0tmi1k's article other external links for more wordlists. The biggest part of cracking process was done by using these wordlists with the rules mentioned above. Let's see some examples...

Using a single rule:

./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove

Using Rules' combination:

./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 --remove

Bruteforce attack with mask (you can specify whichever charset you want):

./oclHashcat-plus64.bin -a 3 -1 ?l?d?u?s -m 0 ~/hashes/md5_1 ?1?1?1?1?1?1?1?1 -o Ultimate_Crack/eNtr0pY_1 --remove

Combination attack:

./oclHashcat-plus64.bin -a 1 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ~/Wordlists/list -o Ultimate_Crack/eNtr0pY_1 --remove

Combination attack with rules:

./oclHashcat-plus64.bin -a 1 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ~/Wordlists/list -r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 --remove

Permutation attack:

./oclHashcat-plus64.bin -a 4 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -o Ultimate_Crack/eNtr0pY_1 --remove

Permutation attack with rules:

./oclHashcat-plus64.bin -a 4 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove

In some cases, I used the hybrid + mask attack technique:

./oclHashcat-plus64.bin -a 6 -1 ?l?d -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ?1?1 -o Ultimate_Crack/eNtr0pY_1 --remove

Hybrid + mask attack with rules:

./oclHashcat-plus64.bin -a 6 -1 ?l?d -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ?1?1 -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove

At this point, I did not use these last two methods as they were very time consuming. I rather found a better one using KoreLogic's Rules for John the Ripper by piping the output of John the Ripper to oclHashcat-plus. As I mentioned, oclHashcat-plus is able to crack passwords up to 15 characters. For that reason, I had to define every time, via the --stdout option, the length of the produced word. If you own a very fast GPU you can skip the following example.

./john --wordlist=~/Wordlists/all.lst -rules:KoreLogicRulesPrependYears --stdout=10 | ./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 -o Ultimate_Crack/eNtr0pY_1 --remove

Of course you can use other prepend rules created from Korelogic, like KoreLogicRulesPrependNumNum, or even better create your own rules!

It was time to produce a wordlist from the cracked passwords and use it to crack the remaining hashes. From eNtr0pY_1, I removed the MD5 hashes with the following command.

cut -b34- eNtr0pY_1 > eNtr0pY_1.dic

By using the above produced wordlist, a big amount of MD5 hashes were cracked using the fingerprint attack. You can read more about this attack from Martin Bos @purehate and I guarantee you that this technique is very successful!

Of course you can also use the binaries included into hashcat-utils and pipe the output of each util to oclHashcat-plus.

root@m3g9tr0n:~/oclHashcat-plus-0.08/hashcat-utils$ ls combinator.bin expander.bin gate.bin len.bin mp32.bin permute.bin prepare.bin req.bin splitlen.bin

Cracking Passwords with John the Ripper

After testing all my wordlist collection and after several days, it was time to move to John the Ripper for cracking the rest of password hashes...

I used magnum-ripper compiled with OpenCL for ATI/AMD graphics card because I wanted to use the --format=raw-md5-opencl parameter. Compared to --format=raw-md5 , it is way faster as it uses your CPU and GPU!

The Rules that were used with John the Ripper are:

wordlist

Single

NT

Extra

KoreLogicRulesAppendNumbers_and_Specials_Simple

KoreLogicRulesAppend6Num

KoreLogicRulesPrependAndAppendSpecial

KoreLogicRulesAppendNumNum_AddSpecialEverywhere

KoreLogicRulesAppendNumNumNum_AddSpecialEverywhere

KoreLogicRulesL33t.

You can download these rules and add them to your john.conf file:

Let’s see now some examples with John the Ripper...

Using --rules=Single :

./john --format=raw-md5-opencl --wordlist=../../Wordlists/all.lst --rules:Single ~/hashes/md5_1

The results of cracked hashes are stored in the john.pot file by default. You can examine its contents with cat , more , head and tail .

root@m3g9tr0n:~/Tools/Password_Cracking/magnum-jumbo-OpenCL/run$ tail -n 9 john.pot $MD5$0fad81e7a61b47d387dde893fcf8e88a:anacarolinagu $MD5$0f82fc9a81f5db07eb9289767390fd2b:fabulousfoodsu $MD5$0e22933267b2e7df062703c4e5842029:fabuloustravelu $MD5$0d40086a54fefe993c9816d1441672ac:modularhomeu $MD5$0ed8181fc4d18e260dd8e36633124bfd:greenshoppingu $MD5$0d6e8da4017ec5c384ac5536087da44d:lawofattractionu $MD5$0eb916d3c6a66a32cedd4acc6edb1dbb:hotreportu $MD5$0e241f99b5c13d56686ec618ab54d5fa:flightsandholidaysu $MD5$0f3c99478362aae389d2cbf716394269:stthomasmoresu

To generate a wordlist from the john.pot file, you can use the following command.

cut -d: -f 2- john.pot | sort -u > cracked.dic

The generated wordlist can be used to crack more hashes when combined with the abovementioned rules.

When I was cracking MD5 hashes with oclHashcat-plus, I observed that some produced passwords were rejected. This is because oclHashcat-plus has a limitation about characters' length. For that reason, I piped hashcat's output to John the Ripper with the additional advantage of using hashcat rules with John the Ripper.

./hashcat-cli64.bin --stdout ~/Wordlists/d3ad0ne.dic -r rules/best64.rule | ./john --format=raw-md5-opencl --stdin ~/hashes/md5_1

After trying all the wordlists combined with the rules mentioned above, it was time to move to bruteforce attacks with John the Ripper. Unfortunately, John the Ripper does not use the mask attacks to produce passwords when implementing bruteforce attacks. We have to create our own charset based on cracked passwords contained in john.pot.

./john --make-charset=eNtr0pY.chr Loaded 7948325 plaintexts Generating charsets... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 DONE Generating cracking order... DONE Successfully written charset file: eNtr0pY.chr (95 characters)

Many of you will wonder about "31 DONE"... This is just because I compiled John the Ripper with 31 characters length. By default, John the Ripper is compiled with support for up to 8 characters length, so it is best to change it by modifying the following lines of the header file params.h located in the scr folder of John the Ripper.

#define CHARSET_MIN ' ' #define CHARSET_MAX 0x7E #define CHARSET_SIZE (CHARSET_MAX - CHARSET_MIN + 1) #define CHARSET_LENGTH 8 //Change that to 31 or whatever you wish

At last you have to include your created charset to john.conf as provided in this example:

# Incremental modes [Incremental:eNtr0pY] File = $JOHN/eNtr0pY.chr MinLen = 0 MaxLen = 31 CharCount = 95

Now it is time to use bruteforce attacks with our own charstet!

./john --format=raw-md5-opencl --incremental=eNtr0pY ~/hashes/md5_1

If you look into john.conf you will see some bruteforce attack modes categorized as externals. These are Double, Strip, Keyboard (which uses neighbor combinations produced from keyboard characters), KnownForce, DateTime, Repeats, Sequence, Subsets and DumbForce for crazy password formats.

./john --format=raw-md5-opencl --external=DumbForce ~/hashes/md5_1

We would also like to crack double MD5 hashes with the help of the dict2hash.pl script provided here.

perl dict2hash.pl < rockyou.txt | ./john --format=raw-md5-opencl --stdin ~/md5_1

Here you can see some samples of cracked MD5s with John the Ripper:

Personally, I believe a password like "$MD5$0b26a0faf1344d6e772bf55628e10e29:n34=mn { .clipboard $me }" is impossible to crack with bruteforce attacks.

Note: All the abovementioned techniques can be used with oclHashcat-plus by defining -m 100 and with John the Ripper by defining --format=raw-sha1-opencl for SHA1 cracking with OpenCL!

Password Analysis

Finally, it worths to see an analysis using pipal (a password analyser) of a collected sample generated from cracking results.

root@m3g9tr0n:~/pipal$ ruby1.9.1 pipal.rb \ -o eNtr0pY_1 ~/Wordlists/Ultimate/Part1/eNtr0pY_5.dic Total entries = 759103 Total unique entries = 758299 Top 10 passwords niezgadniesz123 = 3 (0.0%) ubqu = 3 (0.0%) amonys = 3 (0.0%) centralitie = 3 (0.0%) bobydu = 3 (0.0%) hanghuynh = 3 (0.0%) hmadyousi = 3 (0.0%) matthewperman = 3 (0.0%) shadowninja2 = 3 (0.0%) lhz4 = 3 (0.0%) Top 10 base words august = 219 (0.03%) july = 205 (0.03%) april = 199 (0.03%) june = 195 (0.03%) march = 165 (0.02%) alex = 161 (0.02%) love = 132 (0.02%) chris = 130 (0.02%) daniel = 128 (0.02%) dragon = 122 (0.02%) Password length (length ordered) 1 = 13 (0.0%) 2 = 103 (0.01%) 3 = 1332 (0.18%) 4 = 16781 (2.21%) 5 = 19831 (2.61%) 6 = 95800 (12.62%) 7 = 202414 (26.66%) 8 = 158562 (20.89%) 9 = 103855 (13.68%) 10 = 75652 (9.97%) 11 = 46023 (6.06%) 12 = 24997 (3.29%) 13 = 8423 (1.11%) 14 = 3772 (0.5%) 15 = 1560 (0.21%) Password length (count ordered) 7 = 202414 (26.66%) 8 = 158562 (20.89%) 9 = 103855 (13.68%) 6 = 95800 (12.62%) 10 = 75652 (9.97%) 11 = 46023 (6.06%) 12 = 24997 (3.29%) 5 = 19831 (2.61%) 4 = 16781 (2.21%) 13 = 8423 (1.11%) 14 = 3772 (0.5%) 15 = 1560 (0.21%) 3 = 1332 (0.18%) 2 = 103 (0.01%) 1 = 13 (0.0%) | | | || || || || ||| |||| |||| ||||| ||||| |||||| |||||| ||||||||| ||||||||||||||||| 00000000001111111 01234567890123456 One to six characters = 133854 (17.63%) One to eight characters = 494828 (65.19%) More than eight characters = 264275 (34.81%) Only lowercase alpha = 154996 (20.42%) Only uppercase alpha = 14072 (1.85%) Only alpha = 169068 (22.27%) Only numeric = 119581 (15.75%) First capital last symbol = 6088 (0.8%) First capital last number = 73611 (9.7%) Months january = 109 (0.01%) february = 45 (0.01%) march = 247 (0.03%) april = 251 (0.03%) may = 850 (0.11%) june = 246 (0.03%) july = 223 (0.03%) august = 300 (0.04%) september = 80 (0.01%) october = 134 (0.02%) november = 113 (0.01%) december = 115 (0.02%) Days monday = 59 (0.01%) tuesday = 20 (0.0%) wednesday = 7 (0.0%) thursday = 38 (0.01%) friday = 46 (0.01%) saturday = 7 (0.0%) sunday = 70 (0.01%) Months (Abreviated) jan = 1482 (0.2%) feb = 249 (0.03%) mar = 8397 (1.11%) apr = 692 (0.09%) may = 850 (0.11%) jun = 889 (0.12%) jul = 1051 (0.14%) aug = 785 (0.1%) sept = 215 (0.03%) oct = 512 (0.07%) nov = 821 (0.11%) dec = 874 (0.12%) Days (Abreviated) mon = 4319 (0.57%) tues = 28 (0.0%) wed = 217 (0.03%) thurs = 44 (0.01%) fri = 758 (0.1%) sat = 769 (0.1%) sun = 1018 (0.13%) Includes years 1975 = 411 (0.05%) 1976 = 388 (0.05%) 1977 = 446 (0.06%) 1978 = 432 (0.06%) 1979 = 441 (0.06%) 1980 = 541 (0.07%) 1981 = 453 (0.06%) 1982 = 519 (0.07%) 1983 = 533 (0.07%) 1984 = 603 (0.08%) 1985 = 585 (0.08%) 1986 = 616 (0.08%) 1987 = 710 (0.09%) 1988 = 641 (0.08%) 1989 = 941 (0.12%) 1990 = 931 (0.12%) 1991 = 995 (0.13%) 1992 = 935 (0.12%) 1993 = 905 (0.12%) 1994 = 907 (0.12%) 1995 = 4021 (0.53%) 1996 = 858 (0.11%) 1997 = 486 (0.06%) 1998 = 443 (0.06%) 1999 = 416 (0.05%) 2000 = 1024 (0.13%) 2001 = 643 (0.08%) 2002 = 586 (0.08%) 2003 = 1132 (0.15%) 2004 = 1254 (0.17%) 2005 = 796 (0.1%) 2006 = 818 (0.11%) 2007 = 1442 (0.19%) 2008 = 1019 (0.13%) 2009 = 742 (0.1%) 2010 = 767 (0.1%) 2011 = 516 (0.07%) 2012 = 925 (0.12%) 2013 = 165 (0.02%) 2014 = 142 (0.02%) 2015 = 146 (0.02%) 2016 = 118 (0.02%) 2017 = 139 (0.02%) 2018 = 131 (0.02%) 2019 = 172 (0.02%) 2020 = 179 (0.02%) Years (Top 10) 1995 = 4021 (0.53%) 2007 = 1442 (0.19%) 2004 = 1254 (0.17%) 2003 = 1132 (0.15%) 2000 = 1024 (0.13%) 2008 = 1019 (0.13%) 1991 = 995 (0.13%) 1989 = 941 (0.12%) 1992 = 935 (0.12%) 1990 = 931 (0.12%) Colours black = 485 (0.06%) blue = 549 (0.07%) brown = 184 (0.02%) gray = 89 (0.01%) green = 348 (0.05%) orange = 125 (0.02%) pink = 262 (0.03%) purple = 73 (0.01%) red = 2974 (0.39%) white = 179 (0.02%) yellow = 85 (0.01%) violet = 63 (0.01%) indigo = 22 (0.0%) Single digit on the end = 92080 (12.13%) Two digits on the end = 87587 (11.54%) Three digits on the end = 103715 (13.66%) Last number 0 = 45407 (5.98%) 1 = 64764 (8.53%) 2 = 52570 (6.93%) 3 = 52890 (6.97%) 4 = 43719 (5.76%) 5 = 55185 (7.27%) 6 = 42826 (5.64%) 7 = 46169 (6.08%) 8 = 42475 (5.6%) 9 = 44930 (5.92%) | | | | | ||| | |||| | | | |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| |||||||||| 0123456789 Last digit 1 = 64764 (8.53%) 5 = 55185 (7.27%) 3 = 52890 (6.97%) 2 = 52570 (6.93%) 7 = 46169 (6.08%) 0 = 45407 (5.98%) 9 = 44930 (5.92%) 4 = 43719 (5.76%) 6 = 42826 (5.64%) 8 = 42475 (5.6%) Last 2 digits (Top 10) 95 = 14675 (1.93%) 23 = 12192 (1.61%) 12 = 9230 (1.22%) 11 = 8214 (1.08%) 01 = 7606 (1.0%) 00 = 7131 (0.94%) 07 = 6295 (0.83%) 10 = 6182 (0.81%) 21 = 5881 (0.77%) 99 = 5868 (0.77%) Last 3 digits (Top 10) 123 = 6857 (0.9%) 995 = 4122 (0.54%) 971 = 2916 (0.38%) 972 = 2850 (0.38%) 007 = 2514 (0.33%) 000 = 1868 (0.25%) 234 = 1725 (0.23%) 666 = 1465 (0.19%) 777 = 1389 (0.18%) 004 = 1347 (0.18%) Last 4 digits (Top 10) 1995 = 3886 (0.51%) 1234 = 1379 (0.18%) 2007 = 1325 (0.17%) 2004 = 1121 (0.15%) 2003 = 1016 (0.13%) 2008 = 869 (0.11%) 2000 = 846 (0.11%) 1991 = 819 (0.11%) 2012 = 809 (0.11%) 1990 = 789 (0.1%) Last 5 digits (Top 10) 12345 = 743 (0.1%) 23456 = 652 (0.09%) 54321 = 189 (0.02%) 23123 = 140 (0.02%) 56789 = 127 (0.02%) 34567 = 102 (0.01%) 11111 = 99 (0.01%) 45678 = 75 (0.01%) 00000 = 73 (0.01%) 88888 = 68 (0.01%) US Area Codes 971 = Oregon: Metropolitan Portland, Salem/Keizer area, incl Cricket Wireless (OR) 972 = Texas: Dallas Metro (TX) 234 = NE Ohio: Canton, Akron (OH) Character sets loweralphanum: 330937 (43.6%) loweralpha: 154996 (20.42%) numeric: 119581 (15.75%) mixedalphanum: 41121 (5.42%) upperalphanum: 41078 (5.41%) mixedalpha: 28464 (3.75%) upperalpha: 14072 (1.85%) loweralphaspecial: 10222 (1.35%) loweralphaspecialnum: 5735 (0.76%) mixedalphaspecial: 4724 (0.62%) upperalphaspecial: 2939 (0.39%) mixedalphaspecialnum: 2247 (0.3%) specialnum: 648 (0.09%) upperalphaspecialnum: 374 (0.05%) special: 47 (0.01%) Character set ordering stringdigit: 349534 (46.05%) allstring: 197532 (26.02%) alldigit: 119581 (15.75%) digitstring: 28873 (3.8%) othermask: 18649 (2.46%) stringdigitstring: 14577 (1.92%) stringspecial: 10441 (1.38%) digitstringdigit: 9981 (1.31%) stringspecialstring: 5469 (0.72%) stringspecialdigit: 3075 (0.41%) specialstring: 834 (0.11%) specialstringspecial: 510 (0.07%) allspecial: 47 (0.01%) Hashcat masks (Top 10) ?d?d?d?d?d?d?d: 85053 (11.2%) ?l?l?l?l?l?l: 38400 (5.06%) ?l?l?l?l?l?l?l?l: 36217 (4.77%) ?l?l?l?l?l?l?l: 35468 (4.67%) ?l?l?l?l?l?l?d?d?d: 24051 (3.17%) ?l?l?l?l?l?l?d?d: 18591 (2.45%) ?l?l?l?l?l?d?d?d: 18047 (2.38%) ?d?d?d?d?d?d: 16048 (2.11%) ?l?l?l?l?l?l?l?l?l: 14236 (1.88%) ?l?l?l?l?d?d?d: 13802 (1.82%)

Conclusion

This was a very time consuming and a hard job because I do not own the fastest graphics card. The whole cracking process took about 5 months to accomplish because I had to finish my studies for CCNP certification. The lesson learned from this is that with a good and smart dictionary combined with handy rules either for hashcat or John the Ripper even strong passwords can be cracked. Based on the above statement, admins should use a stronger hash algorithm (with salt) to store your passwords and on your side just change your passwords in a regular basis.

Thanks for reading.

You can find me on twitter, @m3g9tr0n.

Downloads

You can download the results of the cracked hashes:

The provided KoreLogic torrent file contains various but unique password hashes. For that reason you may find duplicated passwords in these wordlists, as a single password can be hashed using various algorithmes! Meaning that 122 million unique hashes (MD5, SHA1, double MD5, etc.) were cracked and result in 83,6 million unique passwords.

You can download the “all in one” version, cleaned and sorted:

The command used to generate this "all in one" CLEANED wordlist was:

export LC_ALL='C' && cat * | sort | uniq > eNtr0pY_ALL_sort_uniq.dic

References

hashcat Wiki [ http://hashcat.net/wiki/ ]

John the Ripper Wiki [ http://openwall.info/wiki/john ]

Related terms: