John the Ripper GPU support

The content of this wiki page is currently mostly out of date, and should not be used. More up-to-date documentation can be found in the “doc” subdirectory in a JtR tree, and in particular in “doc/README-OPENCL”.

Historical content

John the Ripper 1.7.9-jumbo-6 supported GPUs for the following hashes and non-hashes: Reasonably efficient: phpass sha256crypt sha512crypt md5crypt bcrypt MSCash2 WPA-PSK RAR Password Safe CUDA Yes Yes Yes Yes Yes Yes Yes OpenCL Yes Yes Yes Yes Yes Yes Yes Yes Yes Notes: Latest release include more formats and latest git trees include even more.

CUDA significantly benefits from compile-time tuning of parameters for a given GPU, see doc/README-CUDA

sha512crypt is currently faster on NVIDIA than on AMD/ATI, and moreover the OpenCL code is currently faster than CUDA (unusual)

bcrypt is sort of supposed to be inefficient on GPU, so it achieves at best CPU-like performance currently (yet you do get the option to put your many GPUs to at least some reasonable use if you only have bcrypt hashes to attack)

WPA-PSK CUDA uses a little bit of CPU code as well, and benefits from OpenMP (so you'll get slightly faster speed if you enable OpenMP in the JtR build, although you'll load multiple CPU cores then). The OpenCL version has post-processing on GPU too so doesn't use much CPU resources and will not benefit from OpenMP.

RAR is a mix of CPU and GPU code, with the ratio varying widely between different RAR archives (type, size). Later versions has mature “early reject” code that minimizes this. Currently inefficient because the candidate passwords are generated on CPU only: NT raw-MD4 raw-MD5 raw-SHA1 raw-SHA256 raw-SHA224 raw-SHA512 XSHA512 MSCash NSLDAP MySQL 4.1 CUDA Yes Yes Soon Yes Yes Yes OpenCL Yes Soon Yes Yes Yes Yes Yes Yes Yes Notes: XSHA512 is actually pretty efficient in the “many salts” case and with few hashes per salt (normally there's just one hash per salt), achieving close to 70M c/s on GTX 580, which is reasonable for SHA-512 as it is not very GPU-friendly

raw-MD4 is inadvertently missing its high-level *_fmt.c file in 1.7.9-jumbo-6, although the OpenCL code is included. From Jumbo-7 on, this is corrected. Previously, some of these things were available as patches, but starting with 1.7.9-jumbo-6 they're integrated into the jumbo tree.

Usage

AMD

Install GPU driver (e.g., amd-driver-installer-12-4-x86.x86_64.run ) and AMD APP SDK (e.g., AMD-APP-SDK-v2.6-lnx64.tgz ) Build John make linux-x86-64-opencl for 64-bit systems

make linux-x86-opencl for 32-bit systems Replace the -opencl suffix with -gpu to build with CUDA support as well (if you have NVIDIA's CUDA SDK installed as well)

Test John Run ../run/john –test to make sure everything is OK Actually, some tests will likely fail because there are currently no “format destructors”; you'd need to test the individual formats on their own, not all at once, to really make sure that everything is OK (or to find out if not, for real). So run commands like e.g. ”../run/john –test –format=wpapsk-opencl”.

run directory and run opencl-test.sh (note: it tests a subset of OpenCL-using formats only) Optionally download john opencl tester , extract todirectory and run(note: it tests a subset of OpenCL-using formats only) Now you are ready to go

NVIDIA

Add OpenCL and CUDA support. For 64 bits Install GPU driver (e.g., NVIDIA-Linux-x86_64-340.32.run ) and CUDA SDK (e.g., cuda-repo-7.0-28_amd64.deb ) http://www.yourownlinux.com/2014/12/how-to-install-nvidia-340-65-stable-graphics-drivers-in-linux.html For Nvidia Driver Install http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/ For Cuda Install Exports variables to terminal or add to ~/.bashrc (If you add to bashrc file remember to reload terminal –> source ~/.bashrc) export PATH=$PATH:/usr/local/cuda-7.0/bin export LD_LIBRARY_PATH=:/usr/local/cuda-7.0/lib64 export PATH=$PATH:/usr/local/cuda-7.0/targets/x86_64-linux/include export LD_LIBRARY_PATH=:/usr/local/cuda-7.0/targets/x86_64-linux/lib Build John: 1.7.9-jumbo-7 make linux-x86-64-cuda for 64-bit systems make linux-x86-cuda for 32-bit systems Build John: 1.8.0-jumbo-1 ./configure && make clean && make -sj4 That should detect automatically Nvidia Opencl and CUDA Check the output in the terminal, you should see: .... Opencl .... yes CUDA ...... yes .... Check formats supported: ../run/john --list=formats --format=cuda #for CUDA ../run/john --list=formats --format=opencl #for Opencl Test John Run ../run/john --test to make sure everything is OK Actually, some tests will likely fail because there are currently no "format destructors". you'd need to test the individual formats on their own, not all at once, to really make sure that everything is OK (or to find out if not, for real). So run commands like e.g. "../run/john --test --format=wpapsk-opencl". Optionally download {{john:john_test_data:john-opencl-tester.tar.gz|john opencl tester}}, extract to run directory and run opencl-test.sh (note: it tests a subset of OpenCL-using formats only) Now you are ready to go For 32 bits Do the same as 64 bits but choose the correct files.

GPU selection

List available GPUs ../run/john --list=opencl-devices Select specific GPU (in this example device #3) ../run/john --devices=3 Select multiple GPUs (set fork to the number of GPUs selected) ../run/john --devices=0,1,4 --fork=3

Benchmarks

MSCash2

As of 1.7.9-jumbo-6: OpenCL version of MSCash2 supports cracking on multiple GPUs. For more detail see common_opencl_pbkdf2.h . Benchmarks: AMD Radeon HD 7970: 92.5K c/s real ATI Radeon HD 4890: 19.9K c/s real Nvidia GTX 570 : 26.7K c/s real AMD Radeon HD 7970 + Nvidia GTX 570: 113.5K c/s real

Old content