By now, we’ve become quite familiar with using some debugging tools and have some sense of how to approach finding a file’s password. For this next challenge, we will use ltrace to intercept and record the library calls which are called by our executable file. We are attempting to find the password of this file, if you would like to follow along. First we run ltrace with our file to see if we can find any promising leads.

Using ltrace on our file.

Based on this output of ltrace we can see that the function strncmp() is being called repeatedly with successive environment variables being compared. Many of these are familiar and recognizable, for example I already know that my $USER environment variable is “vagrant”. Interestingly, though, this output leads us to explore a strange, unknown variable “Password” which currently has no value. Perhaps we could try giving it one and run ltrace again to see if it makes a difference.

Export our word “guess” to the environment variable “Passw0rd”.

Interestingly, when we run the password with that environment variable, the function strncmp() references that environment variable, and when it does, it calls another function MD5_Init(). We already know what strncmp() does (it compares two strings to see if they are the same), but we have never before encountered MD5_Init(), so let’s investigate.

A google search of that function reveals that it is a cryptographic hash function (a Message-digest algorithm) that uses a 128-bit encryption algorithm to generate a 32-character hexadecimal hash. That is quite a promising lead because now we can presume that our file is processing our “guess” as a 32-character hexadecimal hash using the MD5 algorithm. If we encode our string “guess” using this service, for example, we find a corresponding 32-character hash: 4142047431f5f974ef182c6f3a4982f6. Excitingly, we find that very same hash being broken into two-character intervals by the C library function sprintf(). Presumably, it is taking the string in the environment variable “Passw0rd” and sending the formatted output of MD5 to a string that is stored in memory for later comparison by the strncmp() function. Indeed, that is exactly what we see happening in the last line before our output tells us that the function puts() has informed the user: “Access Denied”. As we have encountered strncmp() before, and know that perhaps the other hash is an encrypted MD5 hash of the file’s password. If we run it through our decrypting service, we find that the word “qwerty” matches that hash in their database. Let’s change our environment variable to “qwerty” from “guess” and try again.

Export “qwerty” as our new password guess.

Execute the file once more, and BOOM, access to the file is granted.

Here’s a quick lesson, then. If you are using MD5 hashes to encode your file’s password, do not use common words or passwords that might already be in a decryption database. It is very easy to decrypt them and hack into that file.