John the Ripped – Steak and French Fries With Salt and Pepper Sauce for Hungry Password Crackers

John the Ripper into its latest community enhanced version (John the Ripper 1.7.9-jumbo-5) has many advanced features. Most of them are without any doubt very useful and appreciated such as MD5 hash cracking.

Four days to come before Hack In The Box Amsterdam 2012 security conferences. Excitement is at its top level, bags are already packed and the iOS Hacker’s Handbook is left open on the beside table. But, because it is always time for challenges I decided to face one that I had in mind for years…

Cracking a custom hash algorithm and making your own password cracking cluster would be great, wouldn't it? Well, you know what? You can make it with John the Ripper jumbo version.

Updates: (subscribe to my twitter to get notified)

11/16/2012 – Note about run/dynamic.conf file. No need to recompile, much easier to edit!
11/16/2012 – Note dynamic function names up to 999 are reserved!
11/16/2012 – Added “–subformat=LIST” tip.

Prepare salt and pepper sauce… the French Cuisine

Most of the time, hashed passwords are salted and combined with different famous hash algorithms. For example, developers who care a little about security will hash user passwords with different hash algorithm combinations, i.e. sha1(md5($salt.$password)."HelloWorld"). This kind of classic enhanced security to store hashed passwords makes the job harder for password crackers.

First of all the attacker needs to know how passwords were hashed. Reverse engineering is always a good start but the easiest way is to get the sources. The second step and the one I'd like to talk in the first part of this article is to implement and use your own hash algorithm for cracking purpose.

First go to http://www.openwall.com/john/, and download the latest jumbo “community enhanced” version. When I wrote this article the latest stable release was 1.7.9-jumbo-5.

$ wget http://www.openwall.com/john/g/john-1.7.9-jumbo-5.tar.gz

$ tar -xvzf john-1.7.9-jumbo-5.tar.gz
Let's see what we have here…

$ cd john-1.7.9-jumbo-5/src/

Before changing anything, we'd like to check if it compiles well.

The make command will list all available compilation modes. This time I'm gonna compile john on MacOS X Lion 10.7.3. Choose the one you prefer…

john-1.7.9-jumbo-5/src$ make

john-1.7.9-jumbo-5/src$ make macosx-x86-64
If everything is ok you should see john binaries and configuration files into the run directory.

john-1.7.9-jumbo-5/src$ cd ../run/

john-1.7.9-jumbo-5/run$ ./john --test
Now go back to the src directory

john-1.7.9-jumbo-5/run$ cd ../src/

In the introduction I talked about a custom hash algorithm such as sha1(md5($salt.$password)."HelloWorld"). So let's use this one as example.

Note: A similar procedure can also be applied directly to the run/dynamic.conf file, where you can add your own dynamic functions ([List.Generic:dynamic_XXXX]) without the need to recompile.

What we'll have to modify is dynamic_preloads.c. This is where we can create our custom algorithm under the name of dynamic_1666. Names up to dynamic_999 are reserved, so make sure to use a number which is not already in use by another dynamic function. Use the command ./john -subformat=LIST to check available numbers.

Additionally, you’ll find into this file many examples of classic dynamic subformats such as md5(md5($password)). I advise you to understand by your own how things work before doing anything.

When you are ready, open dynamic_preloads.c and add these new lines

  //dynamic_1666 --> sha1(md5($s.$p)."HelloWorld") BY THIREUS
  static DYNAMIC_primitive_funcp _Funcs_1666[] =
  {
      DynamicFunc__clean_input,
      DynamicFunc__append_salt,
      DynamicFunc__append_keys,
      DynamicFunc__crypt,
      DynamicFunc__SSEtoX86_switch_output1,
      DynamicFunc__clean_input2,
      DynamicFunc__append_from_last_output_to_input2_as_base16,
      DynamicFunc__append_input2_from_CONST1,
      DynamicFunc__SHA1_crypt_input2_to_output1_FINAL,
      NULL
  };
  static struct fmt_tests _Preloads_1666[] =
  {
      {"$dynamic_1666$e964aa651052d2bbd64aea60756d7705634187f6$admin","password"}, // salt=admin, password=password
      {"$dynamic_1666$4a573951007f7d23eb411c066e2cfb8a175a76d2$123456789","heydude"},
      {"$dynamic_1666$fee9c8708b2e1a177acd350513c14ce0e9900609$salted","test123"},
      {"$dynamic_1666$d8e18f5f1035ce486dd3a08911a4205d78fc7f49$bonjour","awesome"},
      {NULL}
  };
  static DYNAMIC_Constants _Const_1666[] =
  {
      {"HelloWorld"},
      {NULL}
  };

If you are curious about how to declare DynamicFunc__ actions and optimise your function, you’ll find all you need in dynamic_parser.c and dynamic_fmt.c files.

Finally at the end of the file, we need to specify hashes format

  { "dynamic_1666: sha1($s.md5($p).\"HelloWorld\")", _Funcs_1666,_Preloads_1666,_Const_1666, MGF_SALTED|MGF_SHA1_40_BYTE_FINISH, MGF_NO_FLAG },

Once everything is in place, we have to clean and compile again.

john-1.7.9-jumbo-5/src$ make clean

john-1.7.9-jumbo-5/src$ make macosx-x86-64
You should see john binaries and configuration files into the run directory. And you can run the -test option of John the Ripper.

john-1.7.9-jumbo-5/src$ cd ../run/

john-1.7.9-jumbo-5/run$ ./john --test

New lines should appear to display benchmark scores for your function.

    Benchmarking: dynamic_1666: sha1($s.md5($p)."HelloWorld") [SSE2i 10x4x3]... DONE
    Many salts: 1855K c/s real, 1995K c/s virtual
    Only one salt:  1741K c/s real, 1852K c/s virtual
     
    Benchmarking: dynamic_1666: sha1($s.md5($p)."HelloWorld") [64x2 (MD5_Body)]... DONE
    Many salts: 1373K c/s real, 1509K c/s virtual
    Only one salt:  1283K c/s real, 1410K c/s virtual

You can also verify that your dynamic function exists with the following command.

john-1.7.9-jumbo-5/run$ ./john --subformat=LIST

Test fails?

There are many reasons why tests can fail. The main reasons are due to a bad use of the DynamicFunc__ actions, bad order or bad implementation. This will result into a verbose fail of John before starting any tests.
Another common issue, could be that your fmt_tests are broken, meaning bad format for example, this results into a FAILED (valid) error during the tests.
And one last note, if you hash long strings using SSE mode your tests will automatically fail! That's the reason why you have to switch between SSE and X86 mode using functions such as DynamicFunc__ToX86 or DynamicFunc__SSEtoX86_switch_output1.

You should now be ready to crack these passwords.

  lydia:$dynamic_1666$72d4d61b4e5db9ef8704d1af81284c67eea640dd$skyrim
  admin:$dynamic_1666$70ea6b7f633305f04521683226ecabd0537e90ec$example.com
  user123:$dynamic_1666$907c7df1d7e349e98184d74fb7486c77eaf76d60$example.com
  Thireus:$dynamic_1666$e41c041fda28b3615b63acddb6407cf74b354d66$CestLaFeteAlouette

Put them into a hash.txt file, and crack them all.

john-1.7.9-jumbo-5/run$ ./john hash.txt

Step by step instructions for grilling the perfect steak… with MPI enabled barbecue

A few months ago I wrote an article that explains how to compile John the Ripper with OpenMP enabled to take advantage of multiple cores: Crack Passwords using John the Ripper with Multiple CPU Cores (OpenMP). OpenMP is good for algorithms such as DES which can be used by default with this awesome feature. The downside is that not all algorithms are compatible with OpenMP, such as MD5 or SHA1. Fortunately, we can use the MPI (Message Passing Interface) feature of John the Ripper to take advantage of all our CPU cores with any algorithm!

Before going any further, some packages are required. You have to install OpenMPI.

Under MacOS you can do it via MacPorts using the sudo port install openmpi command.
Under Linux you can get everything with sudo apt-get install libopenmpi-dev openmpi-bin openmpi-doc.

Make sure your have the mpirun command available.

Now what you have to do is to open John's Makefile and edit two lines.

$ cd john-1.7.9-jumbo-5/src/

john-1.7.9-jumbo-5/src$ nano Makefile

Locate the following lines.

  #CC = mpicc -DHAVE_MPI -DJOHN_MPI_BARRIER -DJOHN_MPI_ABORT
  #MPIOBJ = john-mpi.o

Uncomment MPI flags.

  CC = mpicc -DHAVE_MPI -DJOHN_MPI_BARRIER -DJOHN_MPI_ABORT
  MPIOBJ = john-mpi.o

Once everything is in place, we have to clean and compile again.

john-1.7.9-jumbo-5/src$ make clean

john-1.7.9-jumbo-5/src$ make macosx-x86-64

Under Linux, compilation should work out of the box. Under MacOS users will face this issue:

    john-mpi.c:6:10: fatal error: 'omp.h' file not found
    #include <omp.h>
             ^
    1 error generated.
    make[1]: *** [john-mpi.o] Error 1
    make: *** [macosx-x86-64] Error 2

To fix it, just open the john-mpi.c file and comment the omp.h file inclusion (which is not needed and must not be used under MacOS X).

    #include "john-mpi.h"
    #include <string.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    //#include <omp.h>

Now it should compile and run fine with mpirun.

john-1.7.9-jumbo-5/run$ mpirun -n 8 ./john hash.txt

You need to adjust the number of cores depending on your CPU. With the previous command the work is now split in 8 sub-processes, one per core on my i7-8600K. Isn’t that great?

Warning: Once the number of cores is fixed for a session, don’t change it unless you know what you are doing. Because for sure it can break your work.

Note that you can use sessions, and similar options that can be associated with mpirun. For example, if you want to know the state of a session:

john-1.7.9-jumbo-5/run$ mpirun -n 8 ./john --status=mysavedsession

The John instances will read all the mysavedsession.%d.rec, where %d is a number between 0 and 7 in our case. Also, sessions are saved every 10 minutes, so don't be scared if the status command displays null stats for the first 10 minutes.

Cook some French fries for your steak

So you have many computers in your room, and want to take advantage of all CPUs? As promised, I'll talk about clustering here for advanced users only.

Before going any further, some packages are required. You have to install OpenMPI and mpich2.

Under MacOS you can do it via MacPorts using the sudo port install openmpi mpich2 command.
Under Linux you can get everything with sudo apt-get install libopenmpi-dev openmpi-bin openmpi-doc mpich2.

Make sure your have the mpirun command available and hydra_pmi_proxy.

hydra_pmi_proxy is the binary file which is used to talk between computers. It is located under /opt/local/bin/hydra_pmi_proxy on MacOS X and /usr/local/bin/hydra_pmi_proxy under Linux.

What you need to know now is that any systems must run the same John the Ripper version, in the same directory and use the same version of mpich2. If this is not the case you can manually compile and install mpich2 and also create symbolic links with ln -s command.

For example, to talk between MacOS and Linux I had to make sure hydra_pmi_proxy can be reached via the same path on both systems.

ln -s /usr/local/bin/hydra_pmi_proxy /opt/local/bin/hydra_pmi_proxy

Now that all your computers are ready, make sure you can reach them using ssh, because this is how MPI messages are going to be transmitted. So, I advice you to create SSH key pairs. Once done, create a nodes.txt file, containing the list of ip addresses of the computers you want to use.

toto@192.168.1.145
localhost
localhost
localhost
localhost
mike@192.168.4.12
mike@192.168.4.12
mike@192.168.4.12
mike@192.168.4.12
192.168.5.5
192.168.5.5
paul@mydomain.com
paul@mydomain.com

You can now use this file to invoke commands on other systems. Let’s start with john-1.7.9-jumbo-5/run/john –test:

mpirun -f nodes.txt -n 18 john-1.7.9-jumbo-5/run/john --test

You may have noticed that I'm not using 18 instances (for 18 CPU cores). Because once the end of the nodes.txt file is reached, mpirun will start reading the list again from the beginning of the file, making a loop.
toto@192.168.1.145 will thus be used twice, as well as the four localhost entries.

You should now be ready to play with your own password cracking cluster!