Thireus' Bl0g

Cracking Story – How I Cracked Over 122 Million SHA1 and MD5 Hashed Passwords

by m3g9tr0n on Aug.28, 2012, under Crack1ng, Hack1ng. 168,536 views

This is the story about how I cracked 122 million* password hashes with John the Ripper and oclHashcat-plus.

Author: m3g9tr0n, Copy Editor: Thireus.

It was several months ago, when I (m3g9tr0n) saw a tweet from KoreLogic about a torrent file containing various password hash lists for a total of 146 million passwords. This very big amount of password hashes at first discouraged me, as I only own a classic computer configuration with an AMD Phenom II 4 cores at 3,2 Mhz in addition to an ATI/AMD 5770 graphics card. But at least, I really wanted to give them a try because the field of password cracking fascinates me.

The password cracking tools I used during this long trip were John the Ripper and oclHashcat-plus. This article is about cracking the provided MD5 hashes of KoreLogic only, but the same strategy was also applied to SHA1 hashes.

Updates:

  • 08/29/2012 – New example in John the Ripper section: “Crack double MD5 hashes with the help of dict2hash.pl script”
  • 08/29/2012 – New download! All in one sorted and cleaned version.

Dealing with hashes…

First of all the KoreLogic torrent file file must be decompressed, it contains a folder named “hashes”. Let’s see the content of this folder…

root@m3g9tr0n:~/hashes$ ls
longer_salts  raw-md5.hashes.txt  salted_with_md5  SHA1  vBulletin-v3.8.4

We will concentrate here on the raw-md5.hashes.txt list. This file is 4.3 GB and includes 139444502 lines according to wc utility.

root@m3g9tr0n:~/hashes$ wc -l raw-md5.hashes.txt 
139444502 raw-md5.hashes.txt

As you consider, both John the Ripper and oclHashcat-plus are not able to load this file because it is too big. For that reason, we need to split this file. Under Linux we have a nice utility called split that does this job very well.

root@m3g9tr0n:~$ split --help
Usage: split [OPTION]... [INPUT [PREFIX]]
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default
size is 1000 lines, and default PREFIX is `x'.  With no INPUT, or when INPUT
is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
  -a, --suffix-length=N   use suffixes of length N (default 2)
  -b, --bytes=SIZE        put SIZE bytes per output file
  -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file
  -d, --numeric-suffixes  use numeric suffixes instead of alphabetic
  -l, --lines=NUMBER      put NUMBER lines per output file
      --verbose           print a diagnostic just before each
                            output file is opened
      --help     display this help and exit
      --version  output version information and exit

SIZE may be (or may be an integer optionally followed by) one of following:
KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T, P, E, Z, Y.

We can use the –lines=NUMBER parameter to split our raw-md5.hashes.txt file.

root@m3g9tr0n:~/hashes$ split -l 3000000 raw-md5.hashes.txt part

Note that we can also split the file based on the amount of MBs by taking into consideration that each MD5 hash is 32 bytes long.

Cracking Passwords with oclHashcat-plus

I started with oclHashcat-plus because it contains the –remove option, which enable remove of hash from hashfile once it is cracked and is really convenient. The only limitation oclHashcat-plus has, is the constraint on password length. In other words, it is only able to crack passwords up to 15 characters. The rules that I used for oclHashcat-plus are base64.rule, passwordspro.rule, T0XlC.rule and in some cases d3ad0ne.rule. There rules can be found directly from the oclHashcat-plus suite.

Bruteforce techniques were not my first choice. I used wordlists which I download from g0tm1lk’s blogspot. You will find on g0tmi1k’s article other external links for more wordlists. The biggest part of cracking process was done by using these wordlists with the rules mentioned above. Let’s see some examples…

Using a single rule:

./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove

Using Rules’ combination:

./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 --remove

Bruteforce attack with mask (you can specify whatever charset you want):

./oclHashcat-plus64.bin -a 3 -1 ?l?d?u?s -m 0 ~/hashes/md5_1 ?1?1?1?1?1?1?1?1 -o Ultimate_Crack/eNtr0pY_1 --remove

Combination attack:

./oclHashcat-plus64.bin -a 1 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ~/Wordlists/list -o Ultimate_Crack/eNtr0pY_1 --remove

Combination attack with rules:

./oclHashcat-plus64.bin -a 1 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ~/Wordlists/list -r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 --remove

Permutation attack:

./oclHashcat-plus64.bin -a 4 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -o Ultimate_Crack/eNtr0pY_1 --remove

Permutation attack with rules:

./oclHashcat-plus64.bin -a 4 -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove

In some cases, I used the hybrid + mask attack technique:

./oclHashcat-plus64.bin -a 6 -1 ?l?d -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ?1?1 -o Ultimate_Crack/eNtr0pY_1 --remove

Hybrid + mask attack with rules:

./oclHashcat-plus64.bin -a 6 -1 ?l?d -m 0 ~/hashes/md5_1 ~/Wordlists/d3ad0ne.dic ?1?1 -r rules/best64.rule -o Ultimate_Crack/eNtr0pY_1 --remove

At this point, I did not use these last two methods as they are very time consuming. I rather found a better one using KoreLogic’s Rules for John the Ripper by piping the output of John the Ripper to oclHashcat-plus. As I mentioned, oclHashcat-plus is able to crack passwords up to 15 characters. For that reason, I had to define everytime, via the –stdout option, the length of the produced word. If you own a very fast GPU you do not have to use the following example.

./john --wordlist=~/Wordlists/all.lst -rules:KoreLogicRulesPrependYears --stdout=10 | ./oclHashcat-plus64.bin -m 0 ~/hashes/md5_1 -o Ultimate_Crack/eNtr0pY_1 --remove

Of course you can use other prepend rules created from Korelogic, like KoreLogicRulesPrependNumNum, or even better create your own rules! :-D

It was time to produce a wordlist from the cracked passwords and use it to crack the remaining hashes. From eNtr0pY_1, I removed the MD5 hashes with the following command.

cut -b34- eNtr0pY_1 > eNtr0pY_1.dic

By using the above produced wordlist, a big amount of MD5 hashes were cracked with the fingerprint attack. You can read more about this attack from Martin Bos @purehate and I guarantee that this technique is very successful!

Of course you can also use the binaries included into hashcat-utils and pipe the output of each util to oclHashcat-plus.

root@m3g9tr0n:~/oclHashcat-plus-0.08/hashcat-utils$ ls
combinator.bin  expander.bin  gate.bin  len.bin  mp32.bin  permute.bin  prepare.bin  req.bin  splitlen.bin

Cracking Passwords with John the Ripper

After testing all my wordlist collection and after several days, it was time to move to John the Ripper for cracking the rest of password hashes…

I used magnum-ripper compiled with OpenCL for ATI/AMD graphics card because I wanted to use the –format=raw-md5-opencl parameter. Compared to –format=raw-md5, it is way faster as it uses your CPU and GPU!

The Rules that were used with John the Ripper are wordlist, Single, NT, Extra, KoreLogicRulesAppendNumbers_and_Specials_Simple, KoreLogicRulesAppend6Num, KoreLogicRulesPrependAndAppendSpecial, KoreLogicRulesAppendNumNum_AddSpecialEverywhere, KoreLogicRulesAppendNumNumNum_AddSpecialEverywhere and KoreLogicRulesL33t.

Furthermore you can download these rules and add them to your john.conf file.

Let’s see now some examples with John the Ripper…

Using –rules=Single

./john --format=raw-md5-opencl --wordlist=../../Wordlists/all.lst --rules:Single ~/hashes/md5_1

The results of cracked hashes are stored in the john.pot file by default. You can examine its contents with cat, more, head and tail.

root@m3g9tr0n:~/Tools/Password_Cracking/magnum-jumbo-OpenCL/run$ tail -n 9 john.pot 
$MD5$0fad81e7a61b47d387dde893fcf8e88a:anacarolinagu
$MD5$0f82fc9a81f5db07eb9289767390fd2b:fabulousfoodsu
$MD5$0e22933267b2e7df062703c4e5842029:fabuloustravelu
$MD5$0d40086a54fefe993c9816d1441672ac:modularhomeu
$MD5$0ed8181fc4d18e260dd8e36633124bfd:greenshoppingu
$MD5$0d6e8da4017ec5c384ac5536087da44d:lawofattractionu
$MD5$0eb916d3c6a66a32cedd4acc6edb1dbb:hotreportu
$MD5$0e241f99b5c13d56686ec618ab54d5fa:flightsandholidaysu
$MD5$0f3c99478362aae389d2cbf716394269:stthomasmoresu

To produce a wordlist from the john.pot file, you can use the following command.

cut -d: -f 2- john.pot | sort -u > cracked.dic

The created wordlist can be used to crack more hashes when combined with the abovementioned rules.

When I was cracking MD5 hashes with oclHashcat-plus, I observed that some produced passwords were rejected. This is because oclHashcat-plus has a limitation about characters’ length. For that reason, I piped hashcat’s output to John the Ripper with the additional advantage of using hashcat rules with John the Ripper.

./hashcat-cli64.bin --stdout ~/Wordlists/d3ad0ne.dic -r rules/best64.rule | ./john --format=raw-md5-opencl --stdin ~/hashes/md5_1

After trying all the wordlists combined with the rules mentioned above, it was time to move to bruteforce attacks with John the Ripper. Unfortunately, John the Ripper does not use the mask attacks to produce passwords when implementing bruteforce attacks. We have to create our own charset based on cracked passwords contained in john.pot.

./john --make-charset=eNtr0pY.chr
Loaded 7948325 plaintexts
Generating charsets... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 DONE
Generating cracking order... DONE
Successfully written charset file: eNtr0pY.chr (95 characters)

Many of you will wonder about “31 DONE”… ^^ This is just because I compiled John the Ripper with 31 characters’ length. By default, John the Ripper is compliled with 8 characters’ length, so it is best to change it by modifying the following lines of the header file params.h located in the scr folder of John the Ripper.

#define CHARSET_MIN                     ' '
#define CHARSET_MAX                     0x7E
#define CHARSET_SIZE                    (CHARSET_MAX - CHARSET_MIN + 1)
#define CHARSET_LENGTH                  8 //Change that to 31 or whatever you wish

At last you have to include your created charset to john.conf as given in this example:

# Incremental modes
[Incremental:eNtr0pY]
File = $JOHN/eNtr0pY.chr
MinLen = 0
MaxLen = 31
CharCount = 95

Now it is time to use bruteforce attacks with our own charstet! :-D

./john --format=raw-md5-opencl --incremental=eNtr0pY ~/hashes/md5_1

If you look into john.conf you will see some bruteforce attack modes characterized as extrernals. These are Double, Strip, Keyboard (which uses neighbor combinations produced from keyboard characters), KnownForce, DateTime, Repeats, Sequence, Subsets and DumbForce for crazy password formats.

./john --format=raw-md5-opencl --external=DumbForce ~/hashes/md5_1

We would also like to crack double MD5 hashes with the help of dict2hash.pl script provided here.

perl dict2hash.pl < rockyou.txt | ./john --format=raw-md5-opencl --stdin ~/md5_1

Here you can see some samples of cracked md5s with John the Ripper:

Personally, I believe a password like “$MD5$0b26a0faf1344d6e772bf55628e10e29:n34=mn { .clipboard $me }” is impossible to crack with bruteforce attacks.

Note: All the abovementioned techniques can be used with oclHashcat-plus by defining -m 100 and with John the Ripper by defining –format=raw-sha1-opencl for SHA1 cracking with OpenCL!

Password Analysis

Finally, it worths to see an analysis using pipal (a password analyser) of a collected sample generated from cracking results.

root@m3g9tr0n:~/pipal$ ruby1.9.1 pipal.rb \
-o eNtr0pY_1 ~/Wordlists/Ultimate/Part1/eNtr0pY_5.dic
Total entries = 759103
Total unique entries = 758299

Top 10 passwords
niezgadniesz123 = 3 (0.0%)
ubqu = 3 (0.0%)
amonys = 3 (0.0%)
centralitie = 3 (0.0%)
bobydu = 3 (0.0%)
hanghuynh = 3 (0.0%)
hmadyousi = 3 (0.0%)
matthewperman = 3 (0.0%)
shadowninja2 = 3 (0.0%)
lhz4 = 3 (0.0%)

Top 10 base words
august = 219 (0.03%)
july = 205 (0.03%)
april = 199 (0.03%)
june = 195 (0.03%)
march = 165 (0.02%)
alex = 161 (0.02%)
love = 132 (0.02%)
chris = 130 (0.02%)
daniel = 128 (0.02%)
dragon = 122 (0.02%)

Password length (length ordered)
1 = 13 (0.0%)
2 = 103 (0.01%)
3 = 1332 (0.18%)
4 = 16781 (2.21%)
5 = 19831 (2.61%)
6 = 95800 (12.62%)
7 = 202414 (26.66%)
8 = 158562 (20.89%)
9 = 103855 (13.68%)
10 = 75652 (9.97%)
11 = 46023 (6.06%)
12 = 24997 (3.29%)
13 = 8423 (1.11%)
14 = 3772 (0.5%)
15 = 1560 (0.21%)

Password length (count ordered)
7 = 202414 (26.66%)
8 = 158562 (20.89%)
9 = 103855 (13.68%)
6 = 95800 (12.62%)
10 = 75652 (9.97%)
11 = 46023 (6.06%)
12 = 24997 (3.29%)
5 = 19831 (2.61%)
4 = 16781 (2.21%)
13 = 8423 (1.11%)
14 = 3772 (0.5%)
15 = 1560 (0.21%)
3 = 1332 (0.18%)
2 = 103 (0.01%)
1 = 13 (0.0%)

       |                                                                
       |                                                                
       |                                                                
       ||                                                               
       ||                                                               
       ||                                                               
       ||                                                               
       |||                                                              
      ||||                                                              
      ||||                                                              
      |||||                                                             
      |||||                                                             
      ||||||                                                            
      ||||||                                                            
    |||||||||                                                           
|||||||||||||||||                                                       
00000000001111111
01234567890123456

One to six characters = 133854 (17.63%)
One to eight characters = 494828 (65.19%)
More than eight characters = 264275 (34.81%)

Only lowercase alpha = 154996 (20.42%)
Only uppercase alpha = 14072 (1.85%)
Only alpha = 169068 (22.27%)
Only numeric = 119581 (15.75%)

First capital last symbol = 6088 (0.8%)
First capital last number = 73611 (9.7%)

Months
january = 109 (0.01%)
february = 45 (0.01%)
march = 247 (0.03%)
april = 251 (0.03%)
may = 850 (0.11%)
june = 246 (0.03%)
july = 223 (0.03%)
august = 300 (0.04%)
september = 80 (0.01%)
october = 134 (0.02%)
november = 113 (0.01%)
december = 115 (0.02%)

Days
monday = 59 (0.01%)
tuesday = 20 (0.0%)
wednesday = 7 (0.0%)
thursday = 38 (0.01%)
friday = 46 (0.01%)
saturday = 7 (0.0%)
sunday = 70 (0.01%)

Months (Abreviated)
jan = 1482 (0.2%)
feb = 249 (0.03%)
mar = 8397 (1.11%)
apr = 692 (0.09%)
may = 850 (0.11%)
jun = 889 (0.12%)
jul = 1051 (0.14%)
aug = 785 (0.1%)
sept = 215 (0.03%)
oct = 512 (0.07%)
nov = 821 (0.11%)
dec = 874 (0.12%)

Days (Abreviated)
mon = 4319 (0.57%)
tues = 28 (0.0%)
wed = 217 (0.03%)
thurs = 44 (0.01%)
fri = 758 (0.1%)
sat = 769 (0.1%)
sun = 1018 (0.13%)

Includes years
1975 = 411 (0.05%)
1976 = 388 (0.05%)
1977 = 446 (0.06%)
1978 = 432 (0.06%)
1979 = 441 (0.06%)
1980 = 541 (0.07%)
1981 = 453 (0.06%)
1982 = 519 (0.07%)
1983 = 533 (0.07%)
1984 = 603 (0.08%)
1985 = 585 (0.08%)
1986 = 616 (0.08%)
1987 = 710 (0.09%)
1988 = 641 (0.08%)
1989 = 941 (0.12%)
1990 = 931 (0.12%)
1991 = 995 (0.13%)
1992 = 935 (0.12%)
1993 = 905 (0.12%)
1994 = 907 (0.12%)
1995 = 4021 (0.53%)
1996 = 858 (0.11%)
1997 = 486 (0.06%)
1998 = 443 (0.06%)
1999 = 416 (0.05%)
2000 = 1024 (0.13%)
2001 = 643 (0.08%)
2002 = 586 (0.08%)
2003 = 1132 (0.15%)
2004 = 1254 (0.17%)
2005 = 796 (0.1%)
2006 = 818 (0.11%)
2007 = 1442 (0.19%)
2008 = 1019 (0.13%)
2009 = 742 (0.1%)
2010 = 767 (0.1%)
2011 = 516 (0.07%)
2012 = 925 (0.12%)
2013 = 165 (0.02%)
2014 = 142 (0.02%)
2015 = 146 (0.02%)
2016 = 118 (0.02%)
2017 = 139 (0.02%)
2018 = 131 (0.02%)
2019 = 172 (0.02%)
2020 = 179 (0.02%)
Years (Top 10)
1995 = 4021 (0.53%)
2007 = 1442 (0.19%)
2004 = 1254 (0.17%)
2003 = 1132 (0.15%)
2000 = 1024 (0.13%)
2008 = 1019 (0.13%)
1991 = 995 (0.13%)
1989 = 941 (0.12%)
1992 = 935 (0.12%)
1990 = 931 (0.12%)

Colours
black = 485 (0.06%)
blue = 549 (0.07%)
brown = 184 (0.02%)
gray = 89 (0.01%)
green = 348 (0.05%)
orange = 125 (0.02%)
pink = 262 (0.03%)
purple = 73 (0.01%)
red = 2974 (0.39%)
white = 179 (0.02%)
yellow = 85 (0.01%)
violet = 63 (0.01%)
indigo = 22 (0.0%)

Single digit on the end = 92080 (12.13%)
Two digits on the end = 87587 (11.54%)
Three digits on the end = 103715 (13.66%)

Last number
0 = 45407 (5.98%)
1 = 64764 (8.53%)
2 = 52570 (6.93%)
3 = 52890 (6.97%)
4 = 43719 (5.76%)
5 = 55185 (7.27%)
6 = 42826 (5.64%)
7 = 46169 (6.08%)
8 = 42475 (5.6%)
9 = 44930 (5.92%)

 |                                                                      
 |                                                                      
 | | |                                                                  
 ||| |                                                                  
|||| | | |                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
||||||||||                                                              
0123456789

Last digit
1 = 64764 (8.53%)
5 = 55185 (7.27%)
3 = 52890 (6.97%)
2 = 52570 (6.93%)
7 = 46169 (6.08%)
0 = 45407 (5.98%)
9 = 44930 (5.92%)
4 = 43719 (5.76%)
6 = 42826 (5.64%)
8 = 42475 (5.6%)

Last 2 digits (Top 10)
95 = 14675 (1.93%)
23 = 12192 (1.61%)
12 = 9230 (1.22%)
11 = 8214 (1.08%)
01 = 7606 (1.0%)
00 = 7131 (0.94%)
07 = 6295 (0.83%)
10 = 6182 (0.81%)
21 = 5881 (0.77%)
99 = 5868 (0.77%)

Last 3 digits (Top 10)
123 = 6857 (0.9%)
995 = 4122 (0.54%)
971 = 2916 (0.38%)
972 = 2850 (0.38%)
007 = 2514 (0.33%)
000 = 1868 (0.25%)
234 = 1725 (0.23%)
666 = 1465 (0.19%)
777 = 1389 (0.18%)
004 = 1347 (0.18%)

Last 4 digits (Top 10)
1995 = 3886 (0.51%)
1234 = 1379 (0.18%)
2007 = 1325 (0.17%)
2004 = 1121 (0.15%)
2003 = 1016 (0.13%)
2008 = 869 (0.11%)
2000 = 846 (0.11%)
1991 = 819 (0.11%)
2012 = 809 (0.11%)
1990 = 789 (0.1%)

Last 5 digits (Top 10)
12345 = 743 (0.1%)
23456 = 652 (0.09%)
54321 = 189 (0.02%)
23123 = 140 (0.02%)
56789 = 127 (0.02%)
34567 = 102 (0.01%)
11111 = 99 (0.01%)
45678 = 75 (0.01%)
00000 = 73 (0.01%)
88888 = 68 (0.01%)

US Area Codes
971 = Oregon:  Metropolitan Portland,
               Salem/Keizer area,
               incl Cricket Wireless (OR)
972 = Texas: Dallas Metro (TX)
234 = NE Ohio: Canton, Akron (OH)

Character sets
loweralphanum: 330937 (43.6%)
loweralpha: 154996 (20.42%)
numeric: 119581 (15.75%)
mixedalphanum: 41121 (5.42%)
upperalphanum: 41078 (5.41%)
mixedalpha: 28464 (3.75%)
upperalpha: 14072 (1.85%)
loweralphaspecial: 10222 (1.35%)
loweralphaspecialnum: 5735 (0.76%)
mixedalphaspecial: 4724 (0.62%)
upperalphaspecial: 2939 (0.39%)
mixedalphaspecialnum: 2247 (0.3%)
specialnum: 648 (0.09%)
upperalphaspecialnum: 374 (0.05%)
special: 47 (0.01%)

Character set ordering
stringdigit: 349534 (46.05%)
allstring: 197532 (26.02%)
alldigit: 119581 (15.75%)
digitstring: 28873 (3.8%)
othermask: 18649 (2.46%)
stringdigitstring: 14577 (1.92%)
stringspecial: 10441 (1.38%)
digitstringdigit: 9981 (1.31%)
stringspecialstring: 5469 (0.72%)
stringspecialdigit: 3075 (0.41%)
specialstring: 834 (0.11%)
specialstringspecial: 510 (0.07%)
allspecial: 47 (0.01%)

Hashcat masks (Top 10)
?d?d?d?d?d?d?d: 85053 (11.2%)
?l?l?l?l?l?l: 38400 (5.06%)
?l?l?l?l?l?l?l?l: 36217 (4.77%)
?l?l?l?l?l?l?l: 35468 (4.67%)
?l?l?l?l?l?l?d?d?d: 24051 (3.17%)
?l?l?l?l?l?l?d?d: 18591 (2.45%)
?l?l?l?l?l?d?d?d: 18047 (2.38%)
?d?d?d?d?d?d: 16048 (2.11%)
?l?l?l?l?l?l?l?l?l: 14236 (1.88%)
?l?l?l?l?d?d?d: 13802 (1.82%)

Conclusion

This was a very time consuming and hard job because I do not own the fastest card. The whole cracking process took about 5 months to accomplish because I had to finish my studies about CCNP certification. The lesson learned from this is that with a good and smart dictionary combined with handy rules either for hashcat or John the Ripper even strong passwords can be cracked. Based on the upon statement, admins should use a stronger hash algorithm (with salt) to store your passwords or even better from your side just change your passwords in a regular basis. ;-)

Thanks for reading. :-)
You can find me on twitter, @m3g9tr0n.

Downloads

You can download the results of cracked hashes:

m3g9tr0n_122Million_Passwords_WordLists.zip
m3g9tr0n_122Million_Passwords_WordLists.zip
m3g9tr0n_122Million_Passwords_WordLists.zip
Version: 1.0
721.9 MB
3379 Downloads
Details...

The provided KoreLogic torrent file contains various but unique password hashes. For that reason you may find duplicated passwords in these wordlists, as a single password can be hashed using various algorithmes! Meaning that 122 million unique hashes (MD5, SHA1, double MD5, etc.) were cracked and result in 83,6 million unique passwords.

You can download the “all in one” version, cleaned and sorted:

m3g9tr0n_Passwords_WordList_CLEANED.zip
m3g9tr0n_Passwords_WordList_CLEANED.zip
m3g9tr0n_Passwords_WordList_CLEANED.zip
Version: 1.0
270.2 MB
4390 Downloads
Details...

The command used to generate this “all in one” CLEANED wordlist was:

export LC_ALL='C' && cat * | sort | uniq > eNtr0pY_ALL_sort_uniq.dic

References

Incoming search terms:

:, , , , , , , , , , , , , , , , , , , , , , , , , , , ,

24 Comments for this entry

  • epixoip

    Nice writeup, and excellent job!

    One thing I will comment on though, for “Combination attack with rules” and “Permutation attack with rules” with oclhc-plus, those attack modes actually do not support using rules. It will not give an error if you try to use rules, and it will even tell you that it is using rules, but it doesn’t actually use the rules :-(

    So if you did want to do a combinator attack + rules, you would have to use combinator.bin from hashcat-utils and oclhc-plus in stdin mode with rules. Using your command line as an example:

    ~/hashcat-utils-0.9/combinator.bin ~/Wordlists/d3ad0ne.dic ~/Wordlists/list | ./oclHashcat-plus64.bin ~/hashes/md5_1 -r rules/passwordspro.rule -o Ultimate_Crack/eNtr0pY_1 –remove

    You can use cpu hashcat in stdout mode piped into oclhc-plus to do permutation attack or table attack with rules in a similar fashion. In that way the permutations would happen on the cpu, and the gpu would do all of the rule calculations. Slower, but very potent.

    Some other suggestions… I noticed you said you only used d3ad0ne.rule in only a few cases. I would encourage you to use it more, because it is the most effective ruleset that ships with hashcat. Also, I’d like to introduce you to rule chaining: you can specify multiple rule files with oclhc-plus. One of my favorite rule chains is -r rules/leetspeak.rule -r rules/toggles5.rule, which essentially emulates a table attack on the gpu.

    Keep up the good work! :-)

  • Fred

    Hi,

    Excelent paper, thank you for sharing !

    I have an question about your JTR compilation, i see you take the “MAgnum” Jumbo for take advantage of your GPU, can you please tell me what you modify exactly on the”Makefile”.

    Otherwize, the Parameter.h file, you just add “31″ & that’s all ? & rebuild ?

    Thanks again,

    Fred

  • IKONspirasi

    very good post here, thx for sharing with us, remind us to use salt + pepper lol

  • Redebidet

    That’s very impressive for a 3.2 MHz processor.

  • m3g9tr0n

    @epixoip

    Thank you very much for your kind words and of course for your details.
    As far as the d3ad0ne’s rule is concerned, with no doubt it is the best
    rule but when applying to big wordlists my not so fast GPU, ATI 5770,
    needed too much time to accomplish the cracking process. Also the
    temperature was going to high levels.

    @Fred

    Thank you for your kind words. So let’s get them from the beggining.

    Step 1

    Download Magnum Ripper via the following command:

    git clone https://github.com/magnumripper/magnum-jumbo.git
    Initialized empty Git repository in /home/m3g9tr0n/Desktop/magnum-jumbo/.git/
    remote: Counting objects: 7262, done.
    remote: Compressing objects: 100% (2325/2325), done.
    remote: Total 7262 (delta 5114), reused 7072 (delta 4932)
    Receiving objects: 100% (7262/7262), 3.27 MiB | 832 KiB/s, done.
    Resolving deltas: 100% (5114/5114), done.

    Step 2

    Navigate to src folder and type “make”. You will see something like
    the following output:

    make
    To build John the Ripper, type:
    make clean SYSTEM
    where SYSTEM can be one of the following:
    ([i] is an optional letter for pre-built intrinsics, eg. -sse2i vs -sse2):
    linux-x86-64-gpu Linux, x86-64 with SSE2, CUDA and OpenCL (experimental)
    linux-x86-64-opencl Linux, x86-64 with SSE2 and OpenCL (experimental)
    linux-x86-64-cuda Linux, x86-64 with SSE2 and CUDA (experimental)
    linux-x86-64-avx Linux, x86-64 with AVX (2011+ Intel CPUs)
    linux-x86-64-xop Linux, x86-64 with AVX and XOP (2011+ AMD CPUs)
    linux-x86-64[i] Linux, x86-64 with SSE2 (most common)
    linux-x86-64-icc Linux, x86-64 compiled with icc
    linux-x86-64-clang Linux, x86-64 compiled with clang
    linux-x86-gpu Linux, x86 32-bit with SSE2, CUDA and OpenCL (experimental)
    linux-x86-opencl Linux, x86 32-bit with SSE2 and OpenCL (experimental)
    linux-x86-cuda Linux, x86 32-bit with SSE2 and CUDA (experimental)
    linux-x86-sse2[i] Linux, x86 32-bit with SSE2 (most common, 32-bit)
    linux-x86-mmx Linux, x86 32-bit with MMX (for old computers)
    linux-x86-any Linux, x86 32-bit (for truly ancient computers)
    linux-alpha Linux, Alpha
    linux-sparc Linux, SPARC 32-bit
    linux-ppc32-altivec Linux, PowerPC w/AltiVec (best)
    linux-ppc32 Linux, PowerPC 32-bit
    linux-ppc64 Linux, PowerPC 64-bit
    linux-ia64 Linux, IA-64
    freebsd-x86-64[i] FreeBSD, x86-64 with SSE2 (best)
    freebsd-x86-sse2[i] FreeBSD, x86 with SSE2 (best if 32-bit)
    freebsd-x86-mmx FreeBSD, x86 with MMX
    freebsd-x86-any FreeBSD, x86
    freebsd-alpha FreeBSD, Alpha
    openbsd-x86-64[i] OpenBSD, x86-64 with SSE2 (best)
    openbsd-x86-sse2[i] OpenBSD, x86 with SSE2 (best if 32-bit)
    openbsd-x86-mmx OpenBSD, x86 with MMX
    openbsd-x86-any OpenBSD, x86
    openbsd-alpha OpenBSD, Alpha
    openbsd-sparc64 OpenBSD, SPARC 64-bit (best)
    openbsd-sparc OpenBSD, SPARC 32-bit
    openbsd-ppc32 OpenBSD, PowerPC 32-bit
    openbsd-ppc64 OpenBSD, PowerPC 64-bit
    openbsd-pa-risc OpenBSD, PA-RISC
    openbsd-vax OpenBSD, VAX
    netbsd-sparc64 NetBSD, SPARC 64-bit
    netbsd-vax NetBSD, VAX
    solaris-sparc64-cc Solaris, SPARC V9 64-bit, cc (best)
    solaris-sparc64-gcc Solaris, SPARC V9 64-bit, gcc
    solaris-sparcv9-cc Solaris, SPARC V9 32-bit, cc
    solaris-sparcv8-cc Solaris, SPARC V8 32-bit, cc
    solaris-sparc-gcc Solaris, SPARC 32-bit, gcc
    solaris-x86-64-cc Solaris, x86-64 with SSE2, cc
    solaris-x86-64[i]-gcc Solaris, x86-64 with SSE2, gcc
    solaris-x86-sse2-cc Solaris 9 4/04+, x86 with SSE2, cc
    solaris-x86-sse2[i]-gcc Solaris 9 4/04+, x86 with SSE2, gcc
    solaris-x86-mmx-cc Solaris, x86 with MMX, cc
    solaris-x86-mmx-gcc Solaris, x86 with MMX, gcc
    solaris-x86-any-cc Solaris, x86, cc
    solaris-x86-any-gcc Solaris, x86, gcc
    sco-x86-any-gcc SCO, x86, gcc
    sco-x86-any-cc SCO, x86, cc
    tru64-alpha Tru64 (Digital UNIX, OSF/1), Alpha
    aix-ppc32 AIX, PowerPC 32-bit
    macosx-x86-64 Mac OS X 10.5+, Xcode 3.0+, x86-64 with SSE2 (best)
    macosx-x86-64-opencl Mac OS X 10.5+, Xcode 3.0+, x86-64 with SSE2, OpenCL support
    macosx-x86-sse2 Mac OS X, x86 with SSE2
    macosx-ppc32-altivec Mac OS X, PowerPC w/AltiVec (best)
    macosx-ppc32 Mac OS X, PowerPC 32-bit
    macosx-ppc64 Mac OS X 10.4+, PowerPC 64-bit
    macosx-universal Mac OS X, Universal Binary (x86 + x86-64 + PPC)
    hpux-pa-risc-gcc HP-UX, PA-RISC, gcc
    hpux-pa-risc-cc HP-UX, PA-RISC, ANSI cc
    irix-mips64-r10k IRIX, MIPS 64-bit (R10K) (best)
    irix-mips64 IRIX, MIPS 64-bit
    irix-mips32 IRIX, MIPS 32-bit
    dos-djgpp-x86-mmx DOS, DJGPP, x86 with MMX
    dos-djgpp-x86-any DOS, DJGPP, x86
    win32-cygwin-x86-sse2[i] Win32, Cygwin, x86 with SSE2 (best)
    win32-cygwin-x86-mmx Win32, Cygwin, x86 with MMX
    win32-cygwin-x86-any Win32, Cygwin, x86
    win32-mingw-x86-sse2[i] Win32, MinGW, x86 with SSE2 (best)
    win32-mingw-x86-mmx Win32, MinGW, x86 with MMX
    win32-mingw-x86-any Win32, MinGW, x86
    beos-x86-sse2 BeOS, x86 with SSE2 (best)
    beos-x86-mmx BeOS, x86 with MMX
    beos-x86-any BeOS, x86
    generic Any other Unix-like system with gcc

    Depending on your GPU architecture try the appropriate command. For example, if
    you own an ATI GPU type “make linux-x86-opencl” for 32 bits architecture.
    Furthermore, do not forget that ATI owners should install ATI-STREAM-SDK
    in order to be able to compile JtR for GPU. You can download ATI SDK from
    that link http://developer.amd.com/sdks/AMDAPPSDK/downloads/Pages/default.aspx.

    Step 3

    The file is called “params.h” and is located to “src” folder. Yes, if you follow
    the instructions with success you will compile JtR for 31 characters password
    cracking.

    If you have any other question please do not hesitate to ask.

    @IKONspirasi

    Thank you my friend!Thireus also has done a great job about building and
    using JtR with MPI and OpenMP.

  • Solar Designer

    Thanks for the writeup!

    Just a couple of minor comments:

    I am surprised you found raw-md5-opencl significantly faster than raw-md5. At this time, it is not supposed to be; this is early development code only (it works, but its speed is CPU-like).

    Since you don’t mind trying out unreleased code, you could want to try the bleeding-jumbo branch of JtR (“git clone git://github.com/magnumripper/magnum-jumbo -b bleeding-jumbo bleeding-jumbo”). It uses a little bit less RAM for raw MD5 hashes (I’ve just measured 90 MB for 1 million hashes, vs. magnum-jumbo’s 95 MB), and it uses bitmaps in addition to hash tables for better performance at fast and saltless hashes like this when you load a lot of them at once, which you do. Overall, it’d probably let you load more hashes at once and it’d crack them faster.

    BTW, a 64-bit build of John the Ripper should in fact be able to load all of the 146 million hashes at once on a computer with enough RAM (tens of gigabytes). I understand that in practice (with smaller amounts of RAM) you have to split, though.

  • Wilbur W.

    A minor comment on the cut command. When you use:
    cut -d: -f2 john.pot | sort -u > cracked.dic
    if a password contains a colon, only the characters occuring before the colon will be written to your dictionary file.

    cut -d: -f2- john.pot | sort -u > cracked.dic
    This will write the entire password.

  • Solar Designer

    I’ve just confirmed that it is possible to load 146 million raw MD5 hashes at once. I took bleeding-jumbo, changed PASSWORD_HASH_SIZE_FOR_LDR in params.h from 4 to 6 to speedup loading of the huge file (I’d have to wait tens of minutes otherwise), and ran John on a file with exactly 146 million of raw MD5 hashes (32 hex chars per line, 4818000000 bytes file size, hashes are of strings “0″ through “145999999″), using the –external=DumbForce mode (could as well be incremental or wordlist, but not “single crack” or batch mode as these need login names or the like). The machine I ran this on is a 2xE5420 with 16 GB RAM, but only one CPU core was used and only 8 GB RAM ended up being used by the “john” process (meaning this would work on an 8 GB RAM machine with a little bit of swap as well, although performance could suffer then). The loading completed in under 3 minutes.

  • m3g9tr0n

    @Solar Designer

    Hello Alexander. It is really an honor for us to have a comment of you in our blog.
    When I was implementing the cracking process of these hashes, about 5 month ago,
    I had a machine with ubuntu 10.10 x86 installed under 4GB of RAM.
    Like I said, I started with oclHashcat because it has the –remove option.
    This was very convenient because was cleared many passwords up to 15 characters.
    The remaining hashes cracked via John the Ripper. In that point I would like to ask
    a favor from you about including something similar to JtR like –remove option. That
    would be very convenient. I will try to implement your instructions about Bleeding Jumbo
    under my new laptop which has 8GB of RAM and Intel i7 8 cpu cores processors.

    Again thanks a lot Solar Designer and hope to meet you some day and discuss
    about Password Cracking techniques :-)

    @Wilbur W.

    Thanks a lot my friend!!That was very clever and handy!!I appreciate your comment!

  • Solar Designer

    @m3g9tr0n – being unfamiliar with hashcat, I asked about its -remove option on #openwall (irc.freenode.net), and the consensus appears to be that there’s no point in adding that to JtR since it reads and excludes hashes found in john.pot on every load anyway. However, you may find –show=left useful for occasions when you need e.g. to run another tool on the uncracked hashes. I hope this helps.

    Oh, and let me invite you to the john-users mailing list. I think you should have announced your blog post in there. I’ll probably do that for you. ;-)

  • Solar Designer

    @m3g9tr0n – regarding your added double MD5 example and the dict2hash.pl script, this task is better approached by using JtR’s dynamic mode, and in particular the predefined –format=dynamic_2 for the double MD5 case.

  • m3g9tr0n

    @Solar Designer

    Thank you for quick feedback I really appreciate that!I will try that way!

    Greetings

  • fred

    Wow, i am impress & really appreciated your verry detailed explain, understand perfectly, thank’s a lot !

  • JimF

    I also worked this list pretty hard. Used only a 32 bit build of JtR, and some home built tools. I was able to crack quite a large part. I did download your eNtr0pY_ALL_sort_uniq.dic, and cracked about 450k more than what I had (thank you greatly).

    However, there appears to be a large amount of passwords in my list, that are not in yours. My cleanup is far from good, but I am pretty sure that it is pretty clean. In the file (eNtr0pY_ALL_sort_uniq.dic) there were only 83.6 million lines. I have 110.2 million unique words. After unique difference of the 2 data sets, I end up with about 32.6 million, not in your data set.

    If you want, I can get you these passwords.

    Also, I have quite a few NT’s, some OSC (dynamic_2), some PHPS, some LM, etc. The salted ones were found using JtR with a -regenerate-salt switch (finds hashes where the salt has been ‘lost’, also gets the proper salt). But in total, there less than a million or two of these (IIRC).

    If you would like the extra things I have found, shoot me an email off this list (I think you should have my email from this post), and we can come up a way for me to get the data to you.

    This ‘leak’ of these mostly fake hashes, was one thing which started some scalablity work within JtR, to try to get it to load more hashes, in less memory. It is better, but still could be improved.

    Jim.

  • m3g9tr0n

    @JimF

    Hello Jim. If I remember well from JtR mailing list you must be one of the developers of JtR. It is so nice that someone else did that too. The most important is that according to your words, you found about
    32 millions of additional passwords and that sounds great.

    I respect your choice about do not put them public. For that reason, I will send you an e-mail in order
    to give me your additional results.

    Thanks in advance my friend! :-)

  • JimF

    I have no problems if you place them out publicly. They are all (most) fabricated anyway, by Korelogic, at least it appears that is the case. Even if they were not fabricated, they are simply ‘matches’ for MD5, so there is little real information leakage, other than the passwords to some unknown account.

    I received your email, and will send you the data (by email). I will probably have to chop it up a bit, and send multiple files.

    Jim.

  • CTH

    Hi. I have been working also on the Korelogic dump and some I’m getting from pastebin or others. I haven’t done the exercise of comparing my results to yours to find out which ones we are missing from each other, but if you are interested in more feedback and discussions on how to handle the data and optimize the process drop me a line.

    I’m pretty satisfied with my results counting that I have quite limited CPU reources (1 CPU, 2 G RAM and no GPU’s) and I’ve reached over 100M only on the Korelogic md5 file

    Thanks for the article, it’s really interesting to see that I’m not the only mad arount this.

  • m3g9tr0n

    @CTH

    Thanks for your kind words. I am happy to that someone else also gave them a try. Furthermore, I am really interested for the techniques that you used in order to crack them. Especially, with less limited machine compared to mine.

    I will be waiting for your feedback and you can find me on twitter @m3g9tr0n ;-)

  • mICR0b

    Thank you for your amazing tutorials & I’ve some questions about JTR

    how can I add the 2 Rules on JTR (I think on john.ini) but where exactly ?! (OS Windows)

    How can I patch my JTR Jumbo for more speed ?! (OS Windows)

    and last Question is when I did this cmd –make-charset=eNtr0pY.chr I got this msg:

    Loaded 1 plaintext
    Generating charsets… 1 2 3 4 5 6 7 8 DONE
    Generating cracking order… DONE
    Successfully written charset file: eNtr0pY.chr (8 characters)

    Where is the Problem ?

    Thank you in advance @};-

  • m3g9tr0n

    @mICR0b

    Thanks for you kind words! So, to be honest I do not use JtR under windows. It is more
    convenient for me to complile it under Linux/Unix platforms. Actually I rarely use Windows
    because I am Linux/Unix passionate Lover!
    JtR Rules should be added to john.conf file. If you open with your favorite text editor
    the john.conf file, you will see the default rules like Single, Extra, Jumbo, nt and wordlist.
    I believe john.conf is operating system independend.
    As far your last question about charset is concerned, first you have to define the .pot file.
    Let’s explain that via an example:

    ./john –make-charset=mICR0b.chr –pot=mICR0b.pot

    If you do not define the .pot file then JtR will use john.pot by default.
    The problem of your example is you did not compile JtR for 31 characters.
    JtR by default is compiled for 8 charcters max.
    On every Linux distribution, to compile JtR for 31 characters maz, navigate to scr folder,
    open params.h header file and replace the following line as I explain:

    #define CHARSET_MIN ‘ ‘
    #define CHARSET_MAX 0x7E
    #define CHARSET_SIZE (CHARSET_MAX – CHARSET_MIN + 1)
    #define CHARSET_LENGTH 8 //Change that to 31 or whatever you wish

    I belive that the best option to compile JtR with OpenMP Libraries under Windows would be via
    Cygwin but as I said I am not familiar with that. May be this is a good motivation for you to
    explore the beautiful world of Linux :-)

  • db2

    Can you crack the following MD5 (salted) and if successful, can you please post the result.
    $1$vgFAuS5b$vyeWgoWb0I.xSp409sR1B0

  • Nithun

    Is there any way to use the older version of hashcat without using a gpu card, it was possible before and now it is shown as updated. Im using ubuntu 12.04 version

9 Trackbacks / Pingbacks for this entry

Thireus on Twitter