Thireus' Bl0g

John the Ripper into its latest community enhanced version (John the Ripper 1.7.9-jumbo-5) has many advanced features. Most of them are without any doubt very useful and appreciated such as MD5 hash cracking.

Four days to come before Hack In The Box Amsterdam 2012 security conferences. Excitement is at its top level, bags are already packed and iOS Hacker’s Handbook is left open on the beside table. But because it is always time for challenges, I decided to face one that I have in mind for years…

Cracking a custom hash algorithm and making your own password cracking cluster would be great huh… ? Well you know what? You can do it with John the Ripper jumbo version

Updates: (subscribe to my twitter to get notified)

• 11/16/2012 – Note about run/dynamic.conf file. No need to recompile, much more easier to edit!
• 11/16/2012 – Note dynamic function names up to 999 are reserved!
• 11/16/2012 – Added “–subformat=LIST” tip.

Prepare salt and pepper sauce… the French Cuisine

Most of the time, hashed passwords are salted and combined with different famous hash algorithms. For example, administrators who have a little sensibility with security will hash user passwords with different combinations, i.e. sha1(md5($salt.$password).”HelloWorld”). This kind of classic enhanced security to store hashed passwords makes the job harder for password crackers.

First of all the attacker needs to know how passwords were hashed. Reverse engineering is always a good start but the easiest way is to get the sources. The second point and the one I’ld like to talk in the first part of this article is to implement and use your own hash algorithm for cracking purpose.

• First go to http://www.openwall.com/john/, and download the latest jumbo ”community enhanced” version. When I write this article the latest stable release was 1.7.9-jumbo-5

$ wget http://www.openwall.com/john/g/john-1.7.9-jumbo-5.tar.gz
$ tar -xvzf john-1.7.9-jumbo-5.tar.gz

$ cd john-1.7.9-jumbo-5/src/

Before changing anything, we’ld like to check if it compiles well.

• The make command will list all available compilation modes. This time I’m gonna compile john on MacOS X Lion 10.7.3. Choose the one you prefer…

john-1.7.9-jumbo-5/src$ make
john-1.7.9-jumbo-5/src$ make macosx-x86-64

• If everything is ok you should see john binaries and configuration files into the run directory

john-1.7.9-jumbo-5/src$ cd ../run/
john-1.7.9-jumbo-5/run$ ./john --test

• Now go back to the src directory

john-1.7.9-jumbo-5/run$ cd ../src/

In the introduction I talked about a custom hash algorithm such as sha1(md5($salt.$password).”HelloWorld”). So let’s take this one as example.

Note: A similar procedure can also be applied directly to the run/dynamic.conf file, where you can add your own dynamic functions ([List.Generic:dynamic_XXXX]) without the need to recompile.

What we’ll need to modify is dynamic_preloads.c. This is where we can create our custom algorithm under the name of dynamic_1666. Names up to dynamic_999 are reserved, so make sure to use a number which is not already in use by another dynamic function. Use the command “./john –subformat=LIST” to check available numbers.

Additionally, you’ll find into this file many example of classic dynamic subformats such as md5(md5($password)). I advice you to understand by your own how things work before doing anything.

• When you are ready, open dynamic_preloads.c and add these new lines

//dynamic_1666 --> sha1(md5($s.$p)."HelloWorld") BY THIREUS
static DYNAMIC_primitive_funcp _Funcs_1666[] =
{
	DynamicFunc__clean_input,
	DynamicFunc__append_salt,
	DynamicFunc__append_keys,
	DynamicFunc__crypt,
	DynamicFunc__SSEtoX86_switch_output1,
	DynamicFunc__clean_input2,
	DynamicFunc__append_from_last_output_to_input2_as_base16,
	DynamicFunc__append_input2_from_CONST1,
	DynamicFunc__SHA1_crypt_input2_to_output1_FINAL,
	NULL
};
static struct fmt_tests _Preloads_1666[] =
{
	{"$dynamic_1666$e964aa651052d2bbd64aea60756d7705634187f6$admin","password"}, // salt=admin, password=password
	{"$dynamic_1666$4a573951007f7d23eb411c066e2cfb8a175a76d2$123456789","heydude"},
	{"$dynamic_1666$fee9c8708b2e1a177acd350513c14ce0e9900609$salted","test123"},
	{"$dynamic_1666$d8e18f5f1035ce486dd3a08911a4205d78fc7f49$bonjour","awesome"},
	{NULL}
};
static DYNAMIC_Constants _Const_1666[] =
{
	{"HelloWorld"},
	{NULL}
};

If you are curious about how to declare DynamicFunc__ actions and optimise your function, you’ll find all you need in dynamic_parser.c and dynamic_fmt.c 

• Finally at the end of the  file, we need to specify hashes format

{ "dynamic_1666: sha1($s.md5($p).\"HelloWorld\")", _Funcs_1666,_Preloads_1666,_Const_1666, MGF_SALTED|MGF_SHA1_40_BYTE_FINISH, MGF_NO_FLAG },

• Once everything is in place, we have to clean and compile again.

john-1.7.9-jumbo-5/src$ make clean
john-1.7.9-jumbo-5/src$ make macosx-x86-64

• You should see john binaries and configuration files into the run directory. And you can run the –test option of John the Ripper.

john-1.7.9-jumbo-5/src$ cd ../run/
john-1.7.9-jumbo-5/run$ ./john --test

New lines should appear to display benchmark scores for your function.

Benchmarking: dynamic_1666: sha1($s.md5($p)."HelloWorld") [SSE2i 10x4x3]... DONE
Many salts:	1855K c/s real, 1995K c/s virtual
Only one salt:	1741K c/s real, 1852K c/s virtual

Benchmarking: dynamic_1666: sha1($s.md5($p)."HelloWorld") [64x2 (MD5_Body)]... DONE
Many salts:	1373K c/s real, 1509K c/s virtual
Only one salt:	1283K c/s real, 1410K c/s virtual

• You can also verify that your dynamic function exists with the following command.

john-1.7.9-jumbo-5/run$ ./john --subformat=LIST

Tests fails?

There are many reasons why tests can fail. The main reasons are due to a bad use of DynamicFunc__ actions, bad order or bad implementation. This will result into a verbose fail of John before starting any tests.
Another common issue, could be that your fmt_tests are broken, meaning bad format for example, this results into a FAILED (valid) error during the tests.
And one last point, if you hash long strings using SSE mode your tests will automatically fail! That’s the reason why you have to switch between SSE and X86 mode using functions such as DynamicFunc__ToX86 or DynamicFunc__SSEtoX86_switch_output1.

• You should now be ready to crack these passwords

lydia:$dynamic_1666$72d4d61b4e5db9ef8704d1af81284c67eea640dd$skyrim
admin:$dynamic_1666$70ea6b7f633305f04521683226ecabd0537e90ec$example.com
user123:$dynamic_1666$907c7df1d7e349e98184d74fb7486c77eaf76d60$example.com
Thireus:$dynamic_1666$e41c041fda28b3615b63acddb6407cf74b354d66$CestLaFeteAlouette

• Put them into a hash.txt file, and crack them all

john-1.7.9-jumbo-5/run$ ./john hash.txt

Step by step instructions for grilling the perfect steak… with MPI enabled barbecue

Few months ago I wrote an article that explains how to compile John the Ripper with OpenMP enable to take advantage of Multiple Cores. Crack Passwords using John the Ripper with Multiple CPU Cores (OpenMP). OpenMP is good for algorithms such as DES which can be used by default with this awesome feature. The bad news is that not all algorithms are compatible with OpenMP, such as MD5 or SHA1. Fortunately there is one good news we can use the MPI (Message Passing Interface) feature of John the Ripper, to take advantage of all our CPU cores with any algorithm!

Before going any further, some packages are required. You have to install OpenMPI.

1. Under MacOS you can do it via MacPorts using the “sudo port install openmpi” command.
2. Under Linux you can get everything with “sudo apt-get install libopenmpi-dev openmpi-bin openmpi-doc“.

Make sure your have the mpirun command available.

• Now what you have to do is to open John’s Makefile and edit two lines

$ cd john-1.7.9-jumbo-5/src/
john-1.7.9-jumbo-5/src$ nano Makefile

• Locate the following lines

#CC = mpicc -DHAVE_MPI -DJOHN_MPI_BARRIER -DJOHN_MPI_ABORT
#MPIOBJ = john-mpi.o

• Uncomment MPI flags

CC = mpicc -DHAVE_MPI -DJOHN_MPI_BARRIER -DJOHN_MPI_ABORT
MPIOBJ = john-mpi.o

• Once everything is in place, we have to clean and compile again

john-1.7.9-jumbo-5/src$ make clean
john-1.7.9-jumbo-5/src$ make macosx-x86-64

Under Linux, compilation should work out of the box. Under MacOS users will face this issue:

john-mpi.c:6:10: fatal error: 'omp.h' file not found
#include <omp.h>
         ^
1 error generated.
make[1]: *** [john-mpi.o] Error 1
make: *** [macosx-x86-64] Error 2

To fix it, just open the john-mpi.c file and comment omp.h file inclusion (which is not needed and must not be used under MacOS X)

#include "john-mpi.h"
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
//#include <omp.h>

Now it should compile and run fine with mpirun

john-1.7.9-jumbo-5/run$ mpirun -n 8 ./john hash.txt

You need to adjust the number of cores depending on your CPU . With the previous command the work is now split in 8 sub-processes, one per core on my i7-8600K. Isn’t that great?

Warning: Once the number of cores has been fixed for a session, don’t change it unless you know what you are doing. Because for sure it can break your work

Note that you can use sessions, and similar options that can be associated with mpirun. For example, if you want to know the state of a session:

john-1.7.9-jumbo-5/run$ mpirun -n 8 ./john --status=mysavedsession

This will read for you all the “mysavedsession.%d.rec” where %d is a number between 0 and 7 in this case. One last thing, sessions are saved every 10 minutes, so don’t be scared if the status command displays null stats for the first 10 minutes

Cook some French fries for your steak

So you have many computers in your room, and want to take advantage of all CPUs? As promised, I’ll talk about clustering here for advanced users only

Before going any further, some packages are required. You have to install OpenMPI and mpich2.

1. Under MacOS you can do it via MacPorts using the “sudo port install openmpi mpich2” command.
2. Under Linux you can get everything with “sudo apt-get install libopenmpi-dev openmpi-bin openmpi-doc mpich2“

Make sure your have the mpirun command available and hydra_pmi_proxy.

hydra_pmi_proxy is the binary file which is used to talk between computers. It is located under “/opt/local/bin/hydra_pmi_proxy” on MacOS X and “/usr/local/bin/hydra_pmi_proxy” under Linux.

What you need to know now is that any systems must run the same John the Ripper version, in the same directory and use the same version of mpich2. If this is not the case you can manually compile and install mpich2 and also create symbolic links with “ln -s” command.

For example, to talk between MacOS and Linux I had to make sure hydra_pmi_proxy can be reached using the same path on both systems.

ln -s /usr/local/bin/hydra_pmi_proxy /opt/local/bin/hydra_pmi_proxy

Now that all your computers are ready, make sure you can reach them via ssh, because this is the way used by MPI messages. So I advice you to create SSH key pairs. Once done, create a nodes.txt file, containing ip addresses of the computers you want to use.

toto@192.168.1.145
localhost
localhost
localhost
localhost
mike@192.168.4.12
mike@192.168.4.12
mike@192.168.4.12
mike@192.168.4.12
192.168.5.5
192.168.5.5
paul@mydomain.com
paul@mydomain.com

You can now use this file to invoke commands on other systems. Let’s start with “john-1.7.9-jumbo-5/run/john –test”.

mpirun -f nodes.txt -n 18 john-1.7.9-jumbo-5/run/john --test

You may have noticed that I’m not using 18 processes (18 CPU cores). Because once the end of the nodes.txt file is reached, mpirun will start again at the beginning of the file, making loops. toto@192.168.1.145 will thus be used twice, as well as the 4 localhost. You should now be ready to play with your own password cracking cluster  

Ready to serve. Bon appétit !

Incoming search terms:

• hydra_pmi_proxy (21)
• john mpirun (7)
• openmp md5 (6)
• /usr/bin/hydra_pmi_proxy don\t exist mpi (5)
• hydra_pmi_proxy: no such file or directory (4)
• mpirun john (4)
• john the ripper cluster (4)
• internetcensus2012 wget (3)
• mpich2 hydra_pmi_proxy (3)
• John The Ripper MPI Patch (2)