When I play CTFs I tend to pick tasks that cover topics that I have no idea about. This makes an opportunity to learn something new. Those usually are web, pwn and forensics tasks. However, sometimes I just pick RE task to check if my skills haven’t rusted too much. Also reversing tasks are often well rewarded in ctf points.

Vancouver BSides CTF 2015 was online jeopardy style CTF hosted at https://yvrctf.ctfd.io/

Reversing task was called “detent” and it was worth 250 points.

Task file details:

Fine name: detent-88a5dabea6b4179f1e45aea44a34e0c761e2fdb6

File size: 26296 bytes

MD5 hash: 6d737220ba5c1673f080f393c3428dfb

SHA1 hash: 88a5dabea6b4179f1e45aea44a34e0c761e2fdb6

After quick reconnaissance I noticed a couple of things:

– binary is x64 linux executable file (ELF)

– code is neither obfuscated nor encrypted

– flag is kept on the server

– binary can be virtually divided into two parts:

* input preparation (transformation)

* input validation

Let’s take a look at application’s code. Reading commented parts of disassembled code is the best way to understand a problem to solve, but if you don’t want to bother with assembly you can skip it and read only descriptions. I decided not to change function names and names of local variables to keep disassembled code clean, so if you open it in IDA you will probably get the same names.

1) Input preparation

First thing I noticed there was the string: “please connect to detent.termsec.com:9182 and submit this to get your flag.” referenced by mov edi, offset aPleaseConnectT almost at the end of the function. This string is printed out when application flow is correct and all the checks are passed. Of course patching simply doesn’t make any sense here, as the binary is run server side.

Below code is the main function of the application. It performs series of checks on the input, to filter out unwanted characters, and transforms it into it’s own representation. Modified input and it’s length are passed as parameters to call sub_4002EB (input validation function).

sub_4003B8 proc near var_ 40 = qword ptr - 40h var_ 34 = dword ptr - 34h var_ 30 = byte ptr - 30h var_ 20 = byte ptr - 20h var_ 4 = dword ptr -4 push rbp mov rbp , rsp sub rsp , 40h mov [ rbp + var_ 34 ] , edi mov [ rbp + var_ 40 ] , rsi mov [ rbp + var_ 4 ] , 0 ; zero actual char index lea rax , [ rbp + var_ 20 ] mov rsi , rax mov edi , offset a16s ; "%16s" mov eax , 0 call sub_4006EF ; get input text from stdin jmp short label_loop_statement ; --------------------------------------------------------------------------- label_loop_body: mov eax , [ rbp + var_ 4 ] ; get index cdqe movzx eax , [ rbp + rax + var_ 20 ] ; get single char from input[index] cmp al , 40h jle short label_terminate_process ; check if input[index] is lower or equal 40h -> @ (in alphabet) ; jmp to terminate process mov eax , [ rbp + var_ 4 ] ; get index cdqe movzx eax , [ rbp + rax + var_ 20 ] ; get single char from input[index] cmp al , 60h jle short label_check_str_ending_0 ; check if input[index] is lower or equal 60h -> ' (in alphabet) ; jmp over terminate (all good) ; ; summary for this check ; if(input[index] <= 0x40 && input[index] > 60h) ; { ; terminateProcess() ; } label_terminate_process: mov edi , 1 call sub_400631 ; terminate process ; --------------------------------------------------------------------------- label_check_str_ending_0: mov eax , [ rbp + var_ 4 ] ; get index cdqe movzx eax , [ rbp + rax + var_ 20 ] ; get single char from input[index] test al , al ; check if it's \x00 (end of the input string) jnz short label_rebase_char jmp short label_main_checks ; --------------------------------------------------------------------------- label_rebase_char: mov eax , [ rbp + var_ 4 ] cdqe movzx eax , [ rbp + rax + var_ 20 ] movsx eax , al lea edx , [ rax - 41h ] ; convert char to new base starting from 0 ; so it's current_char - 65 mov eax , edx sar eax , 1Fh shr eax , 1Bh add edx , eax and edx , 1Fh sub edx , eax mov eax , edx mov edx , eax mov eax , [ rbp + var_ 4 ] cdqe mov [ rbp + rax + var_ 30 ] , dl add [ rbp + var_ 4 ] , 1 ; increment index label_loop_statement: cmp [ rbp + var_ 4 ] , 0Fh ; compare string length, it can't be more than 16 chars ; looks like while loop jle short label_loop_body label_main_checks: mov edx , [ rbp + var_ 4 ] lea rax , [ rbp + var_ 30 ] mov esi , edx mov rdi , rax call sub_4002EB ; main checks are in this call ; xor check and final outputs check test eax , eax jz short loc_400478 mov edi , offset aPleaseConnectT ; "please connect to detent.termsec.com:91"... call loc_40066A mov eax , 0 jmp short locret_400482 ; --------------------------------------------------------------------------- loc_400478: mov edi , 1 call sub_400631 ; terminate process ; --------------------------------------------------------------------------- locret_400482: leave retn sub_4003B8 endp

2) Input validation

This part operates on an already modified input. After an alphabet is rebased to start from 0 there are two checks that need to be passed in order to obtain the flag. Disassembled functions that contain those checks aren’t included as the codes of those are too simple. Instead of another assembly listings I wrote pseudocode interpretations of those functions in commentary to call instructions: call sub_4001E6 and call sub_400283.

– xor check – Algorithm that checks if xor operation between previous and current character in rebased alphabet equals one of the following values: 1, 2, 4, 8, 16.

Note: In the first iteration previous character is set to 0.

– output variables check – There are 3 variables that I called outputs. Those variables are modified in the loop with xor check and later compared with constant values at the end of the function. Output values depends on rebased alphabet character. Variable can be decremented by 3, incremented by 7 or set to either 0 or 1. There is also possibility that variable is left untouched.

Names used with descriptions:

rebased_alphabet – alphabet after rebasing from part one ( ‘A'(65) => 0 etc. )

alphabet_length – length of rebased alphabet

current_alphabet_ptr – pointer to current position in rebased alphabet

original_alphabet_ptr – pointer to beginning of rebased alphabet

output_1/_2/_3 – variable used for final check, it’s value depends on rebased alphabet character

previous_char – previous rebased letter value

current_char – current rebased letter value

sub_4002EB proc near var_2C = dword ptr - 2Ch var_ 28 = qword ptr - 28h var_ 18 = dword ptr - 18h var_ 14 = dword ptr - 14h var_ 10 = dword ptr - 10h var_ 9 = byte ptr -9 var_ 8 = qword ptr -8 push rbp mov rbp , rsp sub rsp , 30h mov [ rbp + var_ 28 ] , rdi ; copy rebased_alphabet ptr to local variable mov [ rbp + var_2C ] , esi ; copy rebased_alphabet length to local variable mov rax , [ rbp + var_ 28 ] mov [ rbp + var_ 8 ] , rax ; copy ptr - used as actual rebased_alphabet ptr mov [ rbp + var_ 9 ] , 0 ; zero local variable used in xor check as previous_char from rebased_alphabet mov [ rbp + var_ 10 ] , 0 ; zero output_1 local variable mov [ rbp + var_ 14 ] , 0 ; zero output_2 local variable mov [ rbp + var_ 18 ] , 0 ; zero output_3 local variable mov edi , 1 mov eax , 0 call sub_400795 jmp short label_start_loop ; --------------------------------------------------------------------------- label_loop_body: movsx edx , [ rbp + var_ 9 ] ; get previous_char mov rax , [ rbp + var_ 8 ] ; get ptr to rebased_alphabet movzx eax , byte ptr [ rax ] ; read rebased_alphabet char (current_char) movsx eax , al mov esi , edx mov edi , eax call sub_4001E6 ; bool xor check( current_char, previous_char ) - first pass previous_char is 0 ; return (previouse_alphabet_char ^ current_alphabet_char == 1 || ; previouse_alphabet_char ^ current_alphabet_char == 2 || ; previouse_alphabet_char ^ current_alphabet_char == 4 || ; previouse_alphabet_char ^ current_alphabet_char == 8 || ; previouse_alphabet_char ^ current_alphabet_char == 16 ) test eax , eax jnz short label_set_outputs mov edi , 1 call sub_400631 ; terminate process ; --------------------------------------------------------------------------- label_set_outputs: mov rax , [ rbp + var_ 8 ] ; get ptr to rebased_alphabet movzx eax , byte ptr [ rax ] ; read rebased_alphabet char movsx eax , al lea rcx , [ rbp + var_ 18 ] ; get output_3 local variable ptr lea rdx , [ rbp + var_ 14 ] ; get output_2 local variable ptr lea rsi , [ rbp + var_ 10 ] ; get output_1 local variable ptr mov edi , eax call sub_400283 ; call to set_outputs - this function set output variables needed to pass final checks ; ; in short pseudocode all this call does: ; input = current_alphabet_char & 3; // for output_1 ; input = (current_alphabet_char & 0xC) >> 2; // for output_2 ; input = (current_alphabet_char & 0x30 ) >> 4; // for output_3 ; ; /* this switch is being called 3 times for each output ; variable */ ; ; switch ( input ) ; { ; case 2: ; output/*_1 or _2 or _3*/ -= 3; ; break; ; case 3: ; output/*_1 or _2 or _3*/ = output/*_1 or _2 or _3*/ == 0; ; break; ; case 1: ; output/*_1 or _2 or _3*/ += 7; ; break; ; } ; ; ; in any other case output is left untouched mov rax , [ rbp + var_ 8 ] movzx eax , byte ptr [ rax ] mov [ rbp + var_ 9 ] , al ; move current_char to previous_char variable add [ rbp + var_ 8 ] , 1 ; increment rebased_alphabet pointer label_start_loop: mov rdx , [ rbp + var_ 8 ] mov rax , [ rbp + var_ 28 ] sub rdx , rax ; substitution of 2 pointers (current_alphabet_ptr - original_alphabet_ptr) ; to get actual position delta value mov eax , [ rbp + var_2C ] cdqe cmp rdx , rax ; compare pointer substitution with alphabet_length jl short label_loop_body mov eax , [ rbp + var_ 10 ] cmp eax , 1 ; compare output_1 with 1 jnz short label_return_false ; return false - bad boy mov eax , [ rbp + var_ 14 ] cmp eax , 0FFFFFFF8h ; compare output_2 with -8 jnz short label_return_false ; return false - bad boy mov eax , [ rbp + var_ 18 ] cmp eax , 7 ; compare output_3 with 7 jnz short label_return_false ; return false - bad boy mov eax , 1 ; if all checks are passed positive ; function returns true - good boy jmp short label_return ; --------------------------------------------------------------------------- label_return_false: mov eax , 0 ; return false - bad boy label_return: leave retn sub_4002EB endp

3) Solution

During in-depth analysis I thought I understood the problem, so that it could be solved analytically by finding a correlation between xor check and output variables being set proper values. After thinking a bit I found out that most likely a lot of strings that pass xor check will also pass output’s check. Knowing that I wrote a simple bruteforcer in python (which turned out to find valid solution within a seconds).

My bruteforce (ugly) solution in Python written during CTF in a rush for the points:

#/bin/python import sys,os from random import randint import random def alphabetBase( letter ): return ord(letter) - 65 alphabet = [ 'A', 'B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','R','S','T','U','V','W','X','Y','Z', '[', '\\', ']', '`', '_', '^' ] out_1 = 0 out_2 = 0 out_3 = 0 str_out = "" def xorCheck( ran, index ): xor_check = index ^ ran == 1 or index ^ ran == 2 or index ^ ran == 4 or index ^ ran == 8 or index ^ ran == 16 if xor_check == True: return True else: return False return False def setOutputs( ran ): global out_3, out_1, out_2 if ( ran & 3 ) == 1: out_1 = out_1 + 7 if ( ran & 0xC ) >> 2 == 1: out_2 = out_2 + 7 if ( ( ran & 0x30 ) >> 4 ) == 1: out_3 = out_3 + 7 if ( ran & 3 ) == 2: out_1 = out_1 - 3 if ( ran & 0xC ) >> 2 == 2: out_2 = out_2 - 3 if ( ( ran & 0x30 ) >> 4 ) == 2: out_3 = out_3 - 3 if ( ran & 3 ) == 3: if out_1 == 0: out_1 = 1 else: out_1 = 0 if ( ran & 0xC ) >> 2 == 3: if out_2 == 0: out_2 = 1 else: out_2 = 0 if ( ( ran & 0x30 ) >> 4 ) == 3: if out_3 == 0: out_3 = 1 else: out_3 = 0 foo = True while foo: out_1 = 0 out_2 = 0 out_3 = 0 str_out = "" index = 0 found = [] for x in range( 16 ): ran = alphabetBase( ( random.choice( alphabet ) ) ) xor_check = xorCheck( ran, index ) if xor_check == False: while xor_check == False: ran = alphabetBase( ( random.choice( alphabet ) ) ) xor_check = xorCheck( ran, index ) if xor_check == True: setOutputs( ran ) str_out = str_out + chr( ran + 65 ) found.append( ran ) else: break index = found[x] foo = ( out_1 != 1 ) or ( out_2 != -8 ) or ( out_3 != 7 ) print "out_1: ", out_1, "out_2: ", out_2, "out_3: ", out_3 print str_out

After connecting to ctf server with this task and pasting generated key, application printed flag to standard output.

flag{th3_fly3st_c4v3-4g3d_ch33s3}