Cracking Story - How I Cracked Over 122 Million SHA1 and MD5 Hashed Passwords
This is the story about how I cracked 122 million* password hashes with John the Ripper and oclHashcat-plus.
Author: m3g9tr0n, Copy Editor: Thireus.
It was several months ago, when I (m3g9tr0n) saw a tweet from KoreLogic about a torrent file containing various hash lists of passwords for a total of 146 million passwords. This very big amount of password hashes at first discouraged me, as I only own a classic computer configuration with an AMD Phenom II 4 cores at 3,2 Mhz in addition to an ATI/AMD 5770 graphics card. But I really wanted to give it a try because the field of password cracking fascinates me.
The password cracking tools I used during this long trip were John the Ripper and oclHashcat-plus. This article is about cracking the provided MD5 hashes of KoreLogic only, but the same strategy was also applied to the SHA1 hashes.
Updates:
- 08/29/2012 – New example in the John the Ripper section: "Crack double MD5 hashes with the help of dict2hash.pl script"
- 08/29/2012 – New download! All in one sorted and cleaned version.
Dealing with hashes...
First of all the KoreLogic torrent file file must be decompressed, it contains a folder named "hashes". Let's check the content of this folder...
root@m3g9tr0n:~/hashes$ ls
longer_salts raw-md5.hashes.txt salted_with_md5 SHA1 vBulletin-v3.8.4
We will concentrate from now on the raw-md5.hashes.txt list. This file is 4.3 GB and includes 139444502 lines according to the wc utility.
root@m3g9tr0n:~/hashes$ wc -l raw-md5.hashes.txt
139444502 raw-md5.hashes.txt
As you can assume, both John the Ripper and oclHashcat-plus are not able to load this file because it is too big. For that reason, we need to split this file. Under Linux we have a nice utility called split that does this job very well:
root@m3g9tr0n:~$ split --help
Usage: split [OPTION]... [INPUT [PREFIX]]
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default
size is 1000 lines, and default PREFIX is `x'. With no INPUT, or when INPUT
is -, read standard input.
Mandatory arguments to long options are mandatory for short options too.
-a, --suffix-length=N use suffixes of length N (default 2)
-b, --bytes=SIZE put SIZE bytes per output file
-C, --line-bytes=SIZE put at most SIZE bytes of lines per output file
-d, --numeric-suffixes use numeric suffixes instead of alphabetic
-l, --lines=NUMBER put NUMBER lines per output file
--verbose print a diagnostic just before each
output file is opened
--help display this help and exit
--version output version information and exit
SIZE may be (or may be an integer optionally followed by) one of following:
KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T, P, E, Z, Y.
We can use the --lines=NUMBER parameter to split our raw-md5.hashes.txt file.
root@m3g9tr0n:~/hashes$ split -l 3000000 raw-md5.hashes.txt part
Note that we can also split the file based on the amount of MBs by taking into consideration that each MD5 hash is 32 bytes long.
Cracking Passwords with oclHashcat-plus
I started playing with oclHashcat-plus because it contains the -remove option, which removes the hashes from the hashfile once it is cracked and it is really convenient. The only limitation oclHashcat-plus has, is the constraint on password length. In other words, it is only able to crack passwords up to 15 characters. The rules that I used for oclHashcat-plus are base64.rule, passwordspro.rule, T0XlC.rule and in some cases d3ad0ne.rule. There rules can be found directly from the oclHashcat-plus suite.
Bruteforce techniques were not my first choice. I used wordlists which I downloaded from the g0tm1lk's blogspot. You will find on g0tmi1k's article other external links for more wordlists. The biggest part of cracking process was done by using these wordlists with the rules mentioned above. Let's see some examples...
Using a single rule:
./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove
Using Rules' combination:
./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 --remove
Bruteforce attack with mask (you can specify whichever charset you want):
./oclHashcat-plus64.bin -a 3 -1 ?l?d?u?s -m 0 ~/hashes/md5_1 ?1?1?1?1?1?1?1?1 -o Ultimate_Crack/eNtr0pY_1 --remove
Combination attack:
./oclHashcat-plus64.bin -a 1 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ~/Wordlists/list -o Ultimate_Crack/eNtr0pY_1 --remove
Combination attack with rules:
./oclHashcat-plus64.bin -a 1 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ~/Wordlists/list -r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 --remove
Permutation attack:
./oclHashcat-plus64.bin -a 4 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -o Ultimate_Crack/eNtr0pY_1 --remove
Permutation attack with rules:
./oclHashcat-plus64.bin -a 4 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove
In some cases, I used the hybrid + mask attack technique:
./oclHashcat-plus64.bin -a 6 -1 ?l?d -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ?1?1 -o Ultimate_Crack/eNtr0pY_1 --remove
Hybrid + mask attack with rules:
./oclHashcat-plus64.bin -a 6 -1 ?l?d -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ?1?1 -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove
At this point, I did not use these last two methods as they were very time consuming. I rather found a better one using KoreLogic's Rules for John the Ripper by piping the output of John the Ripper to oclHashcat-plus. As I mentioned, oclHashcat-plus is able to crack passwords up to 15 characters. For that reason, I had to define every time, via the --stdout option, the length of the produced word. If you own a very fast GPU you can skip the following example.
./john --wordlist=~/Wordlists/all.lst -rules:KoreLogicRulesPrependYears --stdout=10 | ./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 -o Ultimate_Crack/eNtr0pY_1 --remove
Of course you can use other prepend rules created from Korelogic, like KoreLogicRulesPrependNumNum, or even better create your own rules!
It was time to produce a wordlist from the cracked passwords and use it to crack the remaining hashes. From eNtr0pY_1, I removed the MD5 hashes with the following command.
cut -b34- eNtr0pY_1 > eNtr0pY_1.dic
By using the above produced wordlist, a big amount of MD5 hashes were cracked using the fingerprint attack. You can read more about this attack from Martin Bos @purehate and I guarantee you that this technique is very successful!
Of course you can also use the binaries included into hashcat-utils and pipe the output of each util to oclHashcat-plus.
root@m3g9tr0n:~/oclHashcat-plus-0.08/hashcat-utils$ ls
combinator.bin expander.bin gate.bin len.bin mp32.bin permute.bin prepare.bin req.bin splitlen.bin
Cracking Passwords with John the Ripper
After testing all my wordlist collection and after several days, it was time to move to John the Ripper for cracking the rest of password hashes...
I used magnum-ripper compiled with OpenCL for ATI/AMD graphics card because I wanted to use the --format=raw-md5-opencl parameter. Compared to --format=raw-md5, it is way faster as it uses your CPU and GPU!
The Rules that were used with John the Ripper are:
- wordlist
- Single
- NT
- Extra
- KoreLogicRulesAppendNumbers_and_Specials_Simple
- KoreLogicRulesAppend6Num
- KoreLogicRulesPrependAndAppendSpecial
- KoreLogicRulesAppendNumNum_AddSpecialEverywhere
- KoreLogicRulesAppendNumNumNum_AddSpecialEverywhere
- KoreLogicRulesL33t.
You can download these rules and add them to your john.conf file:
Let’s see now some examples with John the Ripper...
Using --rules=Single:
./john --format=raw-md5-opencl --wordlist=../../Wordlists/all.lst --rules:Single ~/hashes/md5_1
The results of cracked hashes are stored in the john.pot file by default. You can examine its contents with cat, more, head and tail.
root@m3g9tr0n:~/Tools/Password_Cracking/magnum-jumbo-OpenCL/run$ tail -n 9 john.pot
$MD5$0fad81e7a61b47d387dde893fcf8e88a:anacarolinagu
$MD5$0f82fc9a81f5db07eb9289767390fd2b:fabulousfoodsu
$MD5$0e22933267b2e7df062703c4e5842029:fabuloustravelu
$MD5$0d40086a54fefe993c9816d1441672ac:modularhomeu
$MD5$0ed8181fc4d18e260dd8e36633124bfd:greenshoppingu
$MD5$0d6e8da4017ec5c384ac5536087da44d:lawofattractionu
$MD5$0eb916d3c6a66a32cedd4acc6edb1dbb:hotreportu
$MD5$0e241f99b5c13d56686ec618ab54d5fa:flightsandholidaysu
$MD5$0f3c99478362aae389d2cbf716394269:stthomasmoresu
To generate a wordlist from the john.pot file, you can use the following command.
cut -d: -f 2- john.pot | sort -u > cracked.dic
The generated wordlist can be used to crack more hashes when combined with the abovementioned rules.
When I was cracking MD5 hashes with oclHashcat-plus, I observed that some produced passwords were rejected. This is because oclHashcat-plus has a limitation about characters' length. For that reason, I piped hashcat's output to John the Ripper with the additional advantage of using hashcat rules with John the Ripper.
./hashcat-cli64.bin --stdout ~/Wordlists/d3ad0ne.dic -r rules/best64.rule | ./john --format=raw-md5-opencl --stdin ~/hashes/md5_1
After trying all the wordlists combined with the rules mentioned above, it was time to move to bruteforce attacks with John the Ripper. Unfortunately, John the Ripper does not use the mask attacks to produce passwords when implementing bruteforce attacks. We have to create our own charset based on cracked passwords contained in john.pot.
./john --make-charset=eNtr0pY.chr
Loaded 7948325 plaintexts
Generating charsets... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 DONE
Generating cracking order... DONE
Successfully written charset file: eNtr0pY.chr (95 characters)
Many of you will wonder about "31 DONE"... This is just because I compiled John the Ripper with 31 characters length. By default, John the Ripper is compiled with support for up to 8 characters length, so it is best to change it by modifying the following lines of the header file params.h located in the scr folder of John the Ripper.
#define CHARSET_MIN ' '
#define CHARSET_MAX 0x7E
#define CHARSET_SIZE (CHARSET_MAX - CHARSET_MIN + 1)
#define CHARSET_LENGTH 8 //Change that to 31 or whatever you wish
At last you have to include your created charset to john.conf as provided in this example:
# Incremental modes
[Incremental:eNtr0pY]
File = $JOHN/eNtr0pY.chr
MinLen = 0
MaxLen = 31
CharCount = 95
Now it is time to use bruteforce attacks with our own charstet!
./john --format=raw-md5-opencl --incremental=eNtr0pY ~/hashes/md5_1
If you look into john.conf you will see some bruteforce attack modes categorized as externals. These are Double, Strip, Keyboard (which uses neighbor combinations produced from keyboard characters), KnownForce, DateTime, Repeats, Sequence, Subsets and DumbForce for crazy password formats.
./john --format=raw-md5-opencl --external=DumbForce ~/hashes/md5_1
We would also like to crack double MD5 hashes with the help of the dict2hash.pl script provided here.
perl dict2hash.pl < rockyou.txt | ./john --format=raw-md5-opencl --stdin ~/md5_1
Here you can see some samples of cracked MD5s with John the Ripper:
Personally, I believe a password like "$MD5$0b26a0faf1344d6e772bf55628e10e29:n34=mn { .clipboard $me }" is impossible to crack with bruteforce attacks.
Note: All the abovementioned techniques can be used with oclHashcat-plus by defining -m 100 and with John the Ripper by defining --format=raw-sha1-opencl for SHA1 cracking with OpenCL!
Password Analysis
Finally, it worths to see an analysis using pipal (a password analyser) of a collected sample generated from cracking results.
root@m3g9tr0n:~/pipal$ ruby1.9.1 pipal.rb \
-o eNtr0pY_1 ~/Wordlists/Ultimate/Part1/eNtr0pY_5.dic
Total entries = 759103
Total unique entries = 758299
Top 10 passwords
niezgadniesz123 = 3 (0.0%)
ubqu = 3 (0.0%)
amonys = 3 (0.0%)
centralitie = 3 (0.0%)
bobydu = 3 (0.0%)
hanghuynh = 3 (0.0%)
hmadyousi = 3 (0.0%)
matthewperman = 3 (0.0%)
shadowninja2 = 3 (0.0%)
lhz4 = 3 (0.0%)
Top 10 base words
august = 219 (0.03%)
july = 205 (0.03%)
april = 199 (0.03%)
june = 195 (0.03%)
march = 165 (0.02%)
alex = 161 (0.02%)
love = 132 (0.02%)
chris = 130 (0.02%)
daniel = 128 (0.02%)
dragon = 122 (0.02%)
Password length (length ordered)
1 = 13 (0.0%)
2 = 103 (0.01%)
3 = 1332 (0.18%)
4 = 16781 (2.21%)
5 = 19831 (2.61%)
6 = 95800 (12.62%)
7 = 202414 (26.66%)
8 = 158562 (20.89%)
9 = 103855 (13.68%)
10 = 75652 (9.97%)
11 = 46023 (6.06%)
12 = 24997 (3.29%)
13 = 8423 (1.11%)
14 = 3772 (0.5%)
15 = 1560 (0.21%)
Password length (count ordered)
7 = 202414 (26.66%)
8 = 158562 (20.89%)
9 = 103855 (13.68%)
6 = 95800 (12.62%)
10 = 75652 (9.97%)
11 = 46023 (6.06%)
12 = 24997 (3.29%)
5 = 19831 (2.61%)
4 = 16781 (2.21%)
13 = 8423 (1.11%)
14 = 3772 (0.5%)
15 = 1560 (0.21%)
3 = 1332 (0.18%)
2 = 103 (0.01%)
1 = 13 (0.0%)
|
|
|
||
||
||
||
|||
||||
||||
|||||
|||||
||||||
||||||
|||||||||
|||||||||||||||||
00000000001111111
01234567890123456
One to six characters = 133854 (17.63%)
One to eight characters = 494828 (65.19%)
More than eight characters = 264275 (34.81%)
Only lowercase alpha = 154996 (20.42%)
Only uppercase alpha = 14072 (1.85%)
Only alpha = 169068 (22.27%)
Only numeric = 119581 (15.75%)
First capital last symbol = 6088 (0.8%)
First capital last number = 73611 (9.7%)
Months
january = 109 (0.01%)
february = 45 (0.01%)
march = 247 (0.03%)
april = 251 (0.03%)
may = 850 (0.11%)
june = 246 (0.03%)
july = 223 (0.03%)
august = 300 (0.04%)
september = 80 (0.01%)
october = 134 (0.02%)
november = 113 (0.01%)
december = 115 (0.02%)
Days
monday = 59 (0.01%)
tuesday = 20 (0.0%)
wednesday = 7 (0.0%)
thursday = 38 (0.01%)
friday = 46 (0.01%)
saturday = 7 (0.0%)
sunday = 70 (0.01%)
Months (Abreviated)
jan = 1482 (0.2%)
feb = 249 (0.03%)
mar = 8397 (1.11%)
apr = 692 (0.09%)
may = 850 (0.11%)
jun = 889 (0.12%)
jul = 1051 (0.14%)
aug = 785 (0.1%)
sept = 215 (0.03%)
oct = 512 (0.07%)
nov = 821 (0.11%)
dec = 874 (0.12%)
Days (Abreviated)
mon = 4319 (0.57%)
tues = 28 (0.0%)
wed = 217 (0.03%)
thurs = 44 (0.01%)
fri = 758 (0.1%)
sat = 769 (0.1%)
sun = 1018 (0.13%)
Includes years
1975 = 411 (0.05%)
1976 = 388 (0.05%)
1977 = 446 (0.06%)
1978 = 432 (0.06%)
1979 = 441 (0.06%)
1980 = 541 (0.07%)
1981 = 453 (0.06%)
1982 = 519 (0.07%)
1983 = 533 (0.07%)
1984 = 603 (0.08%)
1985 = 585 (0.08%)
1986 = 616 (0.08%)
1987 = 710 (0.09%)
1988 = 641 (0.08%)
1989 = 941 (0.12%)
1990 = 931 (0.12%)
1991 = 995 (0.13%)
1992 = 935 (0.12%)
1993 = 905 (0.12%)
1994 = 907 (0.12%)
1995 = 4021 (0.53%)
1996 = 858 (0.11%)
1997 = 486 (0.06%)
1998 = 443 (0.06%)
1999 = 416 (0.05%)
2000 = 1024 (0.13%)
2001 = 643 (0.08%)
2002 = 586 (0.08%)
2003 = 1132 (0.15%)
2004 = 1254 (0.17%)
2005 = 796 (0.1%)
2006 = 818 (0.11%)
2007 = 1442 (0.19%)
2008 = 1019 (0.13%)
2009 = 742 (0.1%)
2010 = 767 (0.1%)
2011 = 516 (0.07%)
2012 = 925 (0.12%)
2013 = 165 (0.02%)
2014 = 142 (0.02%)
2015 = 146 (0.02%)
2016 = 118 (0.02%)
2017 = 139 (0.02%)
2018 = 131 (0.02%)
2019 = 172 (0.02%)
2020 = 179 (0.02%)
Years (Top 10)
1995 = 4021 (0.53%)
2007 = 1442 (0.19%)
2004 = 1254 (0.17%)
2003 = 1132 (0.15%)
2000 = 1024 (0.13%)
2008 = 1019 (0.13%)
1991 = 995 (0.13%)
1989 = 941 (0.12%)
1992 = 935 (0.12%)
1990 = 931 (0.12%)
Colours
black = 485 (0.06%)
blue = 549 (0.07%)
brown = 184 (0.02%)
gray = 89 (0.01%)
green = 348 (0.05%)
orange = 125 (0.02%)
pink = 262 (0.03%)
purple = 73 (0.01%)
red = 2974 (0.39%)
white = 179 (0.02%)
yellow = 85 (0.01%)
violet = 63 (0.01%)
indigo = 22 (0.0%)
Single digit on the end = 92080 (12.13%)
Two digits on the end = 87587 (11.54%)
Three digits on the end = 103715 (13.66%)
Last number
0 = 45407 (5.98%)
1 = 64764 (8.53%)
2 = 52570 (6.93%)
3 = 52890 (6.97%)
4 = 43719 (5.76%)
5 = 55185 (7.27%)
6 = 42826 (5.64%)
7 = 46169 (6.08%)
8 = 42475 (5.6%)
9 = 44930 (5.92%)
|
|
| | |
||| |
|||| | | |
||||||||||
||||||||||
||||||||||
||||||||||
||||||||||
||||||||||
||||||||||
||||||||||
||||||||||
||||||||||
||||||||||
0123456789
Last digit
1 = 64764 (8.53%)
5 = 55185 (7.27%)
3 = 52890 (6.97%)
2 = 52570 (6.93%)
7 = 46169 (6.08%)
0 = 45407 (5.98%)
9 = 44930 (5.92%)
4 = 43719 (5.76%)
6 = 42826 (5.64%)
8 = 42475 (5.6%)
Last 2 digits (Top 10)
95 = 14675 (1.93%)
23 = 12192 (1.61%)
12 = 9230 (1.22%)
11 = 8214 (1.08%)
01 = 7606 (1.0%)
00 = 7131 (0.94%)
07 = 6295 (0.83%)
10 = 6182 (0.81%)
21 = 5881 (0.77%)
99 = 5868 (0.77%)
Last 3 digits (Top 10)
123 = 6857 (0.9%)
995 = 4122 (0.54%)
971 = 2916 (0.38%)
972 = 2850 (0.38%)
007 = 2514 (0.33%)
000 = 1868 (0.25%)
234 = 1725 (0.23%)
666 = 1465 (0.19%)
777 = 1389 (0.18%)
004 = 1347 (0.18%)
Last 4 digits (Top 10)
1995 = 3886 (0.51%)
1234 = 1379 (0.18%)
2007 = 1325 (0.17%)
2004 = 1121 (0.15%)
2003 = 1016 (0.13%)
2008 = 869 (0.11%)
2000 = 846 (0.11%)
1991 = 819 (0.11%)
2012 = 809 (0.11%)
1990 = 789 (0.1%)
Last 5 digits (Top 10)
12345 = 743 (0.1%)
23456 = 652 (0.09%)
54321 = 189 (0.02%)
23123 = 140 (0.02%)
56789 = 127 (0.02%)
34567 = 102 (0.01%)
11111 = 99 (0.01%)
45678 = 75 (0.01%)
00000 = 73 (0.01%)
88888 = 68 (0.01%)
US Area Codes
971 = Oregon: Metropolitan Portland,
Salem/Keizer area,
incl Cricket Wireless (OR)
972 = Texas: Dallas Metro (TX)
234 = NE Ohio: Canton, Akron (OH)
Character sets
loweralphanum: 330937 (43.6%)
loweralpha: 154996 (20.42%)
numeric: 119581 (15.75%)
mixedalphanum: 41121 (5.42%)
upperalphanum: 41078 (5.41%)
mixedalpha: 28464 (3.75%)
upperalpha: 14072 (1.85%)
loweralphaspecial: 10222 (1.35%)
loweralphaspecialnum: 5735 (0.76%)
mixedalphaspecial: 4724 (0.62%)
upperalphaspecial: 2939 (0.39%)
mixedalphaspecialnum: 2247 (0.3%)
specialnum: 648 (0.09%)
upperalphaspecialnum: 374 (0.05%)
special: 47 (0.01%)
Character set ordering
stringdigit: 349534 (46.05%)
allstring: 197532 (26.02%)
alldigit: 119581 (15.75%)
digitstring: 28873 (3.8%)
othermask: 18649 (2.46%)
stringdigitstring: 14577 (1.92%)
stringspecial: 10441 (1.38%)
digitstringdigit: 9981 (1.31%)
stringspecialstring: 5469 (0.72%)
stringspecialdigit: 3075 (0.41%)
specialstring: 834 (0.11%)
specialstringspecial: 510 (0.07%)
allspecial: 47 (0.01%)
Hashcat masks (Top 10)
?d?d?d?d?d?d?d: 85053 (11.2%)
?l?l?l?l?l?l: 38400 (5.06%)
?l?l?l?l?l?l?l?l: 36217 (4.77%)
?l?l?l?l?l?l?l: 35468 (4.67%)
?l?l?l?l?l?l?d?d?d: 24051 (3.17%)
?l?l?l?l?l?l?d?d: 18591 (2.45%)
?l?l?l?l?l?d?d?d: 18047 (2.38%)
?d?d?d?d?d?d: 16048 (2.11%)
?l?l?l?l?l?l?l?l?l: 14236 (1.88%)
?l?l?l?l?d?d?d: 13802 (1.82%)
Conclusion
This was a very time consuming and a hard job because I do not own the fastest graphics card. The whole cracking process took about 5 months to accomplish because I had to finish my studies for CCNP certification. The lesson learned from this is that with a good and smart dictionary combined with handy rules either for hashcat or John the Ripper even strong passwords can be cracked. Based on the above statement, admins should use a stronger hash algorithm (with salt) to store your passwords and on your side just change your passwords in a regular basis.
Thanks for reading.
You can find me on twitter, @m3g9tr0n.
Downloads
You can download the results of the cracked hashes:
The provided KoreLogic torrent file contains various but unique password hashes. For that reason you may find duplicated passwords in these wordlists, as a single password can be hashed using various algorithmes! Meaning that 122 million unique hashes (MD5, SHA1, double MD5, etc.) were cracked and result in 83,6 million unique passwords.
You can download the “all in one” version, cleaned and sorted:
The command used to generate this "all in one" CLEANED wordlist was:
export LC_ALL='C' && cat * | sort | uniq > eNtr0pY_ALL_sort_uniq.dic
References
- hashcat Wiki [ http://hashcat.net/wiki/ ]
- John the Ripper Wiki [ http://openwall.info/wiki/john ]