How to crack a Binary File Format By Frans Important notice Please do not send me binary files and expecting me to reverse engineer them for you for free. Request for cracking passwords and other forms of encryption are deleted immediately by me. At the moment, I am not available for any type of reverse engineering work.

Introduction

The most likely reason why people ask me for help is because I already have reversed engineerd some binary file formats. So far, I have worked on:

A good reason why you want to crack the format at all.

A lot of time.

A lot of preseverence.

A lot of creativity.

Willingness to do repeative tasks for hours and hours.

Good understanding of how numbers and strings are represented in binary manner.

Excellent command of the C programming language.

Besides this, you also need:

a computer (quite obvious),

a simple text editor capable of handling long lines,

(optional) a hex viewer or editor,

a calculator with hexadecimal-to-decimal conversion and vice-versa functionality,

a C or C++ compiler,

enough sample files about which you know what data they contain, and

(optional) the application which uses the files.

What is the fun?

Programming style

Some practical remarks:

Document your code (either through comments or, better, by giving meaningful names to variables and procedures).

Make regular back-ups. Although this should be a standerd procedure, with cracking a file format you will see yourself more often going back to an earlier version of the program whenever you get stuck in a certain area.

Build into your program switches (simple Boolean variables) which allow you to dump any intermediate data during any stage of the parsing. You will find that you often need to go back one step to make a step forward.

How to work

Where to start

printf("%02X", byte); printf("%02X ", byte); printf("%c%02X", (byte > ' ' && byte < 127)?byte:' ', byte); printf((byte > ' ' && byte < 127)?" %c":"%02X", byte);

byte

unsigned char

For a primary investigation of a format, the strings program might be useful. Sometimes binary format contain whole chunks of text.

Block structure

Number representation

Negative integers

An important advantage of the 2-compliments notation is that the arithmetic operations for the signed and unsigned representation are exactly the same. The only time the difference become obvious is when a number is cast to a larger representation form. For example the hexadecimal byte value 0xFF needs to be changed to 0xFFFF, when the signed number -1 is cast from a byte to a word, and has to be changed to 0x00FF, when the unsigned number 255 is cast from a byte to a word. In a binary file format, it can occur that a unsigned byte value is stored as a word with the signed expansion applied. As a result of this it can happen that 127 is stored hexadecimal as 0x007F, where 128 is stored hexadecimal as 0xFF80.

little and big endian

floating point formats

float

double

Strings

Packed format

More often it happens that a number of Boolean values are stored in a single byte or word. Sometimes it can be helpful to print numbers in their binary representation.

Rubbish

Reading files

Mmfile

CreateFileMapping

MapViewOfFile

The difference strategy

You can use a Binary File Compare utility to compare the files and discover how certain changes are reflected in the binary file, which gives you information about the where the information is stored.

Links

Tools

(Background image was taken from here)