This article was published on the 22nd of September 2019. This article was updated on the 30th of April 2020.

In this article, the fifth challenge of the sixth Flare-On series will be analysed. At first, observations based on the challenge will be given, after which the challenge will be solved in three ways.

Table of contents

One can download the challenge here. The challenge’s copyright belongs to FireEye.

This article contains methods and tips that were shared with me. For that, I’d like thank Itay ‘Megabeets’ Cohen, Pham Duy Phuc, Salim ‘SolidSnake’ Bitam, and Nikhil Hegde.

Virtual machines tend to have limited support for DirectX. To be able to run this program within a virtual machine, multiple changes need to be made to the default set-up. Firstly, 3D acceleration needs to be enabled.

In Virtual Box, one can enable this by going into the settings of a virtual machine, clicking the Display item in the list view and checking the Enable 3D acceleration checkbox. Note that this acceleration requires more video memory, which should be assigned.

Within the virtual machine, the Guest Additions application should be installed, which can be done when the virtual machine is running. In some cases, the virtual operating system needs to be started in safe mode, otherwise the Direct3D Support cannot be installed.

Select Devices in the menu items on the top of the virtual machine and click on Insert Guest Additions CD image. A virtual CD will then be inserted in the virtual machine, from which the installation wizard is launched. During the installation, make sure to check Direct3D Support (Experimental) when choosing the components that need to be installed.

At last, the correct version of DirectX needs to be installed. Install DirectX 9 to run this challenge smoothly.

Each challenge consists of a little introduction, together with an archive that contains the files for the challenge. Below, the introduction is given.

Someone on the Flare team tried to impress us with their demoscene skills. It seems blank. See if you can figure it out or maybe we will have to fire them. No pressure. 7Zip password: flare ** You will need DirectX 9

Based on this, the binary will show a (seemingly) blank screen, but the introduction hints that there is more to this challenge than meets the eye.

Note that the demo scene refers to a scene where people show their programming skills by creating small programs (called demos) that contain spectacular visuals. In some cases, audio is also present. A site which that is well known within the demo scene, is scene.org.

Upon executing the binary, a single window is shown. In this window, the logo of Flare-On is continuously turning. The only user input that alters the behaviour of the program is the escape key. This causes the program to exit.

Based on the observations above, one can guess that the flag is hidden in plain sight. Because the all flags in Flare-On have a similar lay-out, an e-mail address that ends on @flare-on.com, one can assume that the flag cannot be hidden behind the spinning logo due to its size. As such, it will be present somewhere else.

Another assumption is the fact that the flag is likely to be present in the form of a DirectX texture, as the challenge does not seem to be based upon user input.

The first solution is based on the assumption that the flag is present in the form of a texture. One can extract and view textures, making it trivial to obtain the flag.

The second solution is based on the assumption that the logo and the flag are both objects. If the flag is created instead of the logo, the flag will be visible to the viewer in the middle of the screen.

The third solution is based on the assumption that the flag is not within the boundaries of the screen. Changing the location of the flag object to a visible location, will provide the flag to the viewer.

DirectX is designed to make life easier when working with graphics on the Windows platform, similar to the platform independent OpenGL. The framework exposes APIs that can be used to handle textures.

To extract the textures, a program called NinjaRipper is used, together with tools that are given in this how-to guide. To start NinjaRipper, one has to extract the downloaded archive and navigate to the following location:

[ ninjaripper_folder ] / [ architecture ] / NinjaRipper.exe.

Note that [ninjaripper_folder] is the root of the extracted folder, and [architecture] is either x86 (32-bits) or x64 (64-bits), depending on the operating system which will execute NinjaRipper.

First, select the binary that was given in the challenge as the executable to use. Since the DirectX version is given in the challenge description, two options can be used when ripping the textures: the version specific D3D9 Wrapper or the more general Intruder inject.

The output directory specifies the location where the ripped textures will be stored. The program requires no additional parameters, so this field can be left empty. Upon starting the program, one should wait a few seconds and then press F10. This is the hotkey to extract all textures, as defined in NinjaRipper’s settings.

Noesis, the tool to view the textures, requires a plug-in to support the texture format that NinjaRipper extracted. Before starting Noesis, the plug-in should be placed in the following directory:

[ noesis_folder ] / plugins / python

Note that [noesis_folder] is the root of the extracted folder.

Execute [noesis_folder]/Noesis.exe and navigate to the folder which contains the ripped textures. There are two different textures in the folder, although they may be present multiple times, depending on the duration that the demo binary was running.

The two files differ in size. The flag is a bigger texture, hence the bigger size: 11 kilobytes instead of 2 kilobytes. Click the texture in the Noesis file explorer and press enter to view it in the right pane.

The flag is then visible: moar_pouetry@flare-on.com.

Before diving into the other two solutions, one has to understand the binary. The binary is packed, meaning it will be unpacked during runtime. After the unpacking routine has finished, the used libraries are loaded dynamicaly. These two topics, along with the general flow of the binary, will be covered before the second and third solutions are presented.

To debug the application, the x32dbg debugger will be used. Whilst it can feel logical to step into every function that is encountered, this method will result in a slow, yet thorough, analysis process.

The difference between a breadth first or depth first approach, is to either step over or step into unknown functions. The breadth first approach is most useful in this challenge, because the code base contains quite a lot of unknown functions, whilst it is likely that the solution can be found in a single function.

By narrowing down the amount of possibilities first, the in-depth search for the solution is only needed in a few functions.

The unpacking routine

When debugging 4k.exe with x32dbg, it automatically places a breakpoint to stop at. Pressing ALT + F9, the program will execute until the user defined code is reached. The unpacking routine is the first user defined code that is reached, which can be seen in the code below.

0040005C | push ebx 0040005D | xor ebp , ebp 0040005F | mov ebx , 2 00400064 | nop 00400065 | mov esi , 4k . 400144 0040006A | push 1 0040006C | pop eax 0040006D | mov edi , 4k . 420000 00400072 | mov cl , 0 00400074 | nop 00400075 | push edi 00400076 | jmp 4k . 40008A ;downwards jump 00400078 | add byte ptr ds : [ eax ] , al 0040007A | add byte ptr ds : [ eax ] , al 0040007C | add byte ptr ds : [ eax ] , al 0040007E | add byte ptr ds : [ eax ] , al 00400080 | pop edx 00400081 | jb 4k . 40008A ;downwards jump 00400083 | xchg edx , eax 00400084 | sub ecx , edx 00400086 | add al , 0 00400088 | sub eax , edx 0040008A | pushad 0040008B | lodsd 0040008C | add eax , edi 0040008E | je 4k . 4000BC ;downwards jump 00400090 | push A 00400092 | pop edx 00400093 | mov dword ptr ss : [ esp + edx * 2 ] , edx 00400096 | mov dword ptr ss : [ esp + 10 ] , edx 0040009A | lodsd 0040009B | xor ebp , ebp 0040009D | dec ebp 0040009E | inc ebp 0040009F | add eax , eax 004000A1 | jb 4k . 40009E ;upwards jump 004000A3 | je 4k . 400054 ;upwards 004000A5 | pushad 004000A6 | lodsb 004000A7 | mov dl , al 004000A9 | xor al , byte ptr ds : [ edi ] 004000AB | imul eax , eax , 6F 004000AE | add al , byte ptr ds : [ edi ] 004000B4 | dec eax 004000B5 | dec edi 004000B6 | add dl , dl 004000B8 | jb 4k . 4000A9 ;upwards jump 004000BA | jne 4k . 4000B5 ;upwards jump 004000BC | mov edi , 4k . 421D70 004000C1 | mov ecx , 4k . F9FF81D 004000C6 | jae 4k . 4000D4 ;downwards jump 004000C8 | rep stosw 004000CB | or al , byte ptr ds : [ esi ] 004000CD | popad 004000CE | lea esi , dword ptr ds : [ esi + 14 ] 004000D1 | jnp 4k . 40008A ;upwards jump 004000D3 | ret 004000D4 | div ecx 004000D6 | lea edi , dword ptr ds : [ edi + edx * 2 ] 004000D9 | mov ecx , ebp 004000DB | xor eax , eax 004000DD | scasb 004000DE | je 4k . 4000E4 ;downwards jump 004000E0 | add byte ptr ds : [ edi ] , al 004000E2 | jne 4k . 4000E6 ;downwards jump 004000E4 | inc ecx 004000E5 | inc ecx 004000E6 | movzx edx , byte ptr ds : [ edi + eax ] 004000EA | shl edx , cl 004000EC | add dword ptr ss : [ esp + eax * 4 + 34 ] , edx 004000F0 | dec eax 004000F1 | jp 4k . 4000E6 ;upwards jump 004000F3 | test ebx , ebx 004000F5 | jg 4k . 400104 ;downwards jump 004000F7 | shr byte ptr ds : [ edi + ebx ] , 1 004000FA | jne 4k . 4000FF ;downwards jump 004000FC | rcl byte ptr ds : [ edi + ebx ] , 1 004000FF | not ebx 00400101 | inc byte ptr ds : [ edi + ebx ] 00400104 | popad 00400105 | inc esi 00400106 | jmp 4k . 40009D ;upwards jump

The first instruction, located at 0040005C, is the entry point. When looking through this function in x32dbg, multiple upwards and downwards jumps can be observed. This is an indication that loops are present in the code.

The presence of the pushad instruction, is also a sign that this function is used to unpack the binary. This instruction stores EAX, ECX, EDX, EBX, ESP, EBP, ESI, and EDI on the stack. The popad instruction, which is also present in the unpacking routine, restores the previously stored values from the stack into the respective registers.

The end of the unpacking routine (which does not need to be at the end of the function) is marked with the popad instruction and possibly a ret instruction to return, although this isn’t required. In this challenge, both are used, as can be seen below.

004000CD | popad 004000CE | lea esi , dword ptr ds : [ esi + 14 ] 004000D1 | jnp 4k . 40008A 004000D3 | ret

Note that in some cases, the jmp instruction is used instead of the ret, which can either function as an unconditional jump towards the unpacked code, or it can function as a ret instruction by jumping to the return address. In this program, this technique is not used.

The pushad instruction occurs twice, which is why the jnp (Jump No Parity) instruction is used. When the instruction is first reached, the jump is taken. The second time, it is not taken, and the return instruction is reached. By putting a breakpoint on the return instruction (004000D3), one can skip the unpacking routine. Simply step into (or over) the return instruction to go to the unpacked code that is located at 00420000.

Note that breakpoints that reside in the unpacked address space, are turned off when restarting the binary within x32dbg, as the code is invalid before it is unpacked. Leaving the breakpoint on the return instruction at 004000D3 is a quick way to get into the unpacked binary upon restarting: pressing F9 twice. The first time, the code before the user defined code section is executed. The second time, the unpacking loop is done executing. Then one simply has to step into (or over) the return to end up in the unpacked code.

Dynamic function loading

There are are only a few labeled function calls in the binary, which is odd, given that the DirectX framework is used. In this segment, the required function calls are loaded dynamically.

00420000 | mov ebx , 4k . 400108 00420005 | mov esi , 4k . 420556 0042000A | mov edi , 4k . 430000 0042000F | pop eax 00420010 | mov eax , dword ptr ds : [ eax + C ] 00420013 | mov eax , dword ptr ds : [ eax + C ] 00420016 | mov eax , dword ptr ds : [ eax ] 00420018 | mov eax , dword ptr ds : [ eax ] 0042001A | mov ebp , dword ptr ds : [ eax + 18 ] 0042001D | test ebp , ebp 0042001F | jne 4k . 42002F 00420021 | push 0 00420023 | push 0 00420025 | push edx 00420026 | push 0 00420028 | call dword ptr ds : [ 430014 ] 0042002E | ret 0042002F | xor eax , eax 00420031 | lodsb 00420032 | xchg ecx , eax 00420033 | pushad 00420034 | mov eax , dword ptr ss : [ ebp + 3C ] 00420037 | add eax , ebp 00420039 | mov edx , dword ptr ds : [ eax + 78 ] 0042003C | add edx , ebp 0042003E | mov ecx , dword ptr ds : [ edx + 18 ] 00420041 | mov eax , dword ptr ds : [ edx + 20 ] 00420044 | add eax , ebp 00420046 | mov esi , dword ptr ds : [ eax + ecx * 4 - 4 ] 0042004A | add esi , ebp 0042004C | xor edi , edi 0042004E | rol edi , 6 00420051 | xor eax , eax 00420053 | lodsb 00420054 | xor edi , eax 00420056 | dec eax 00420057 | jge 4k . 42004E 00420059 | cmp edi , dword ptr ds : [ ebx ] 0042005B | loopne 4k . 420041 0042005D | mov eax , dword ptr ds : [ edx + 24 ] 00420060 | add eax , ebp 00420062 | mov cx , word ptr ds : [ eax + ecx * 2 ] 00420066 | mov eax , dword ptr ds : [ edx + 1C ] 00420069 | add eax , ebp 0042006B | mov eax , dword ptr ds : [ eax + ecx * 4 ] 0042006E | mov dword ptr ss : [ esp + 1C ] , eax 00420072 | popad 00420073 | add eax , ebp 00420075 | stosd 00420076 | add ebx , 4 00420079 | loop 4k . 420033 0042007B | push esi 0042007C | call dword ptr ds : [ 430000 ] 00420082 | xchg ebp , eax 00420083 | mov edx , esi 00420085 | lodsb 00420086 | dec al 00420088 | jns 4k . 420085 0042008A | inc al 0042008C | je 4k . 42001D 0042008E | call 4k . 4201CA

When looking at the instructions above, one can also see the pushad and popad instructions. This is an indication that some unpacking or loading takes place in this segment of the code. When loading a library, the registers are used. When the loading has finished, the original values are restored into the registers. This way, the execution will continue as if nothing happened in between. Note that the upward jumps indicate the presence of loops.

When stepping through the code, one can see the names of several dynamic link libraries pass by, as well as functions that are within them. Continuing the execution up until the call 4k.4201CA instruction that is located at 0042008E, shows what the given code segment is doing exactly.

Before the execution, the call dword ptr ds:[430000] instruction at 0042007C does not provide any clue on its content. After the loading routine, the value at the adress of 430000 is changed into something understandable: call dword ptr ds:[]. Based on these observations, one can conclude that this code segment loads the libraries dynamically.

To verify this, one can look at the instruction that is located at 00420028. Below, the instruction is given before and after the execution of the code segment.

;Before the execution of the code segment 00420028 | call dword ptr ds : [ 430014 ] ;After the execution of the code segment 00420028 | call dword ptr ds : [ <&MessageBoxA> ]

A pointer towards the MessageBoxA function is stored at 00430014, meaning that the function has been loaded dynamically. This verifies the conclusion that has been made above.

DirectX functionality

The function call at 0042008E is the first function that is encountered after the dynamic function loading routine has finished. Below, the function is given.

004201CA | push ebp 004201CB | mov ebp , esp 004201CD | sub esp , C 004201D0 | movaps xmm0 , xmmword ptr ds : [ 421D60 ] 004201D7 | movups xmmword ptr ds : [ 421CF4 ] , xmm0 004201DE | mov dword ptr ss : [ ebp - 4 ] , BF800000 004201E5 | mov eax , dword ptr ss : [ ebp - 4 ] 004201E8 | xorps xmm0 , xmm0 004201EB | unpcklps xmm0 , xmm0 004201EE | movq qword ptr ds : [ 421D30 ] , xmm0 004201F6 | mov dword ptr ds : [ 421D38 ] , eax 004201FB | mov esp , ebp 004201FD | pop ebp 004201FE | ret

This function performs some calculations based on floating points, as can bee seen by the usage of the xmm0 register. There is no clue that this is relevant for now, thus a detailed analysis will be skipped until it is required.

The next part of the program is given below.

00420093 | push ebp 00420094 | mov ebp , esp 00420096 | sub esp , 14 00420099 | push esi 0042009A | push 20 0042009C | call < JMP . &Direct3DCreate9> 004200A1 | mov dword ptr ss : [ ebp - 4 ] , eax 004200A4 | test eax , eax 004200A6 | je 4k . 42016B 004200AC | push ebx 004200AD | push edi 004200AE | push 0 004200B0 | push 0 004200B2 | push 0 004200B4 | push 0 004200B6 | push 258 004200BB | push 320 004200C0 | push 0 004200C2 | push 0 004200C4 | push 4k . 10CF0000 004200C9 | push 4k . 420574 004200CE | push 4k . 420580 004200D3 | push 0 004200D5 | call dword ptr ds : [ <&CreateWindowExA> ] 004200DB | mov ebx , eax 004200DD | lea eax , dword ptr ss : [ ebp - 14 ] 004200E0 | push eax 004200E1 | push ebx 004200E2 | call dword ptr ds : [ <&GetWindowRect> ] 004200E8 | mov esi , dword ptr ds : [ <&GetSystemMetrics> ]

In the code that is given above, the Direct3DCreate9 function is called, which returns a pointer towards an IDirect3D9 object. The instance of this object is based on the given SDK version. In this case, the SDK version (which is passed via the stack at 0042009A) equals 20. The result is stored at the address that EBP-4 points to.

After that, the CreateWindowExA function is called, as well as the GetWindowRect and GetSystemMetrics functions. These three functions define the size and the location of the window.

Below, the next part of the code is given.

004200EE | push 0 004200F0 | call esi ;user32.GetSystemMetrics 004200F2 | sub eax , dword ptr ss : [ ebp - C ] 004200F5 | cdq 004200F6 | sub eax , edx 004200F8 | mov edi , eax 004200FA | push 1 004200FC | sar edi , 1 004200FE | call esi ;user32.GetSystemMetrics 00420100 | sub eax , dword ptr ss : [ ebp - 8 ] 00420103 | push 5 00420105 | push 0 00420107 | cdq 00420108 | push 0 0042010A | sub eax , edx 0042010C | sar eax , 1 0042010E | push eax 0042010F | push edi 00420110 | push 0 00420112 | push ebx 00420113 | call dword ptr ds : [ <&SetWindowPos> ] 00420119 | mov ecx , dword ptr ss : [ ebp - 4 ] 0042011C | push 4k . 43003C 00420121 | push 4k . 420588 00420126 | push 40 00420128 | push ebx 00420129 | push 1 0042012B | mov dword ptr ds : [ 4205A4 ] , ebx 00420131 | mov eax , dword ptr ds : [ ecx ] 00420133 | push 0 00420135 | push ecx 00420136 | call dword ptr ds : [ eax + 40 ] ;d3d9.6CCFC8B8 + 0x40 00420139 | pop edi 0042013A | pop ebx 0042013B | test eax , eax 0042013D | js 4k . 42016B ;code that is executed if the sign flag is not set, is omitted here 0042016B | xor eax , eax 0042016D | pop esi 0042016E | mov esp , ebp 00420170 | pop ebp 00420171 | ret

The position of the window is set via the SetWindowPos function.

Note that the instructions which calls the ESI register, actually calls user32.GetSystemMetrics, since ESI contains a pointer towards the GetSystemMetrics function.

Also note that the function call that resides at 00420136 points towards d3d9.6CCFC8B8 + 0x40, which is a function that is not within the user defined code. When stepping over it, a black square becomes visible on the screen. The size of it, is equal to the size of the window where the Flare-On logo is spinning in.

As such, this function provides a key piece of information that is related to the assumption that was made earlier. If the flag is a texture, then the texture is likely to be loaded after the window has been created. As such, all unknown code prior to this function, can be ignored until this hypothesis has been tested.

If the sign flag is set, the jump is taken. This results in the termniation of the program. When executing the program, this jump is not taken.

The code that is executed if the jump is not taken, is given below.

0042013F | call 4k . 4201FF 00420144 | call 4k . 4201A2 00420149 | mov esi , dword ptr ds : [ <&GetAsyncKeyState> ] 0042014F | nop dword ptr ds : [ eax ] , eax 00420153 | push 0 00420155 | call 4k . 42038A 0042015A | add esp , 4 0042015D | push 1B 0042015F | call esi 00420161 | test ax , ax 00420164 | je 4k . 420153 00420166 | pop esi 00420167 | mov esp , ebp 00420169 | pop ebp 0042016A | ret

At first, two unknown functions are called, after which the GetAsynckeyState function pointer is moved into ESI. This function determines if a specific key (which is given as an argument to the function), is currently pressed or has been pressed since the last time the function was called.

The nop instruction is executed, but does nothing. This is the expected result, since nop stands for no operation.

Then, the value 0 is pushed onto the stack and a unknown function is called. After the function call, four bytes are added to the stack pointer to restore the stack as it was before the function call. The amount of 4 bytes is chosen because a single parameter is pushed onto the stack. The size of this in a 32-bit program is 32-bits, which equals 4 bytes.

The value 1B is passed as an argument to the GetAsyncKeyState function. Per the Microsoft documentation regarding Virtual Key Codes, the value 0x1B equals VK_ESCAPE, which is the escape key. As long as the escape key is not pressed, the loop continues. Upon pressing it, the jump is not taken and the program closes. This is in line with the observations that were made in the beginning.

A short recap

In short, the binary unpacks itself and dynamically loads all required functions. After which the window’s size and location are calculated. Based on these values, the window is created. The current assumption is that the textures are loaded after the window has been created. This assumption is likely to be true due to the optimisation that is present in the code: if the textures are created before the window, but the window cannot be created, the program consumed CPU cycles without reason. The other way around, each step is only started if the previous one has completed.

The current code base that still needs to be analysed, consists of only three functions. The first two (located at 004201FF and 004201A2) are executed only once, whereas the function at 0042038A is executed during every iteration of the loop, until the escape key is pressed.

Analysing function 004201FF

This function consists of several function calls. The first two calls refer to the same function, but with different arguments. The return value of both calls is stored and the stack is freed up again. The code is given below.

004201FF | push 4993F8C4 00420204 | push 68A0D4D3 00420209 | push E816F5EC 0042020E | push 38 00420210 | push 4k . 420778 00420215 | push 1E 00420217 | push 4k . 420610 0042021C | call 4k . 4202A8 00420221 | push 84AF72C3 00420226 | push 867B81F0 0042022B | push 4k . CB343C8 00420230 | push 10A 00420235 | push 4k . 4216A8 0042023A | push 128 0042023F | push 4k . 4208C8 00420244 | mov dword ptr ds : [ 430050 ] , eax 00420249 | call 4k . 4202A8 0042024E | add esp , 38 00420251 | mov dword ptr ds : [ 430054 ] , eax

This function is within the user defined code base and is called twice. This might be important later on.

All other function calls refer to the same DirectX base address, where the value of 0xE4 is added before the call instruction is executed. When stepping into it, or when looking at the value of ECX, one can see the name of the dynamic link library, followed by the function address within the library: vboxd3d9.2178EB30 + 0xE4. One can see that there are three push instructions before each function call.

Below, the pattern is given in pseudo code together with comments.

;store the pointer of the function pointer to the function in EAX mov eax , dword ptr ds : [ ptr_ptr_vboxd3d9 ] ;push argument 1 on the stack push arg1 ;store the function pointer towards the vboxd3d9 function in ECX mov ecx , dword ptr ds : [ eax ] ;push argument 2 on the stack push arg2 ;push argument 3 on the stack push arg3 ;call the vboxd3d9 function call dword ptr ds : [ ecx + E4 ]

Below, the rest of this function’s assembly code is given. The pattern that is given above, can be seen, although the order in which some instructions are placed, might vary.

00420256 | mov eax , dword ptr ds : [ 43003C ] 0042025B | push 1 0042025D | mov ecx , dword ptr ds : [ eax ] 0042025F | push 89 00420264 | push eax 00420265 | call dword ptr ds : [ ecx + E4 ] ;vboxd3d9.2178EB30 + 0xE4 0042026B | mov eax , dword ptr ds : [ 43003C ] 00420270 | push FF323232 00420275 | push 8B 0042027A | push eax 0042027B | mov ecx , dword ptr ds : [ eax ] 0042027D | call dword ptr ds : [ ecx + E4 ] ;vboxd3d9.2178EB30 + 0xE4 00420283 | mov eax , dword ptr ds : [ 43003C ] 00420288 | push 1 0042028A | push 16 0042028C | push eax 0042028D | mov ecx , dword ptr ds : [ eax ] 0042028F | call dword ptr ds : [ ecx + E4 ] ;vboxd3d9.2178EB30 + 0xE4 00420295 | mov eax , dword ptr ds : [ 43003C ] 0042029A | push 1 0042029C | push 7 0042029E | push eax 0042029F | mov ecx , dword ptr ds : [ eax ] 004202A1 | call dword ptr ds : [ ecx + E4 ] ;vboxd3d9.2178EB30 + 0xE4 004202A7 | ret

Because the second part of the function does not seem too interesting, this part can be excluded from a possible in-depth analysis later on. This decreases the amount of time that needs to be spend on a part of the program, making the analysis more efficient.

Analysing function 004201A2

The second function that is called before the loop, is much shorter and contains only two functions. The code is given below.

004201A2 | mov eax , dword ptr ds : [ 43003C ] 004201A7 | push 4k . 421CF0 004201AC | push 0 004201AE | push eax 004201AF | mov ecx , dword ptr ds : [ eax ] 004201B1 | call dword ptr ds : [ ecx + CC ] ;vboxd3d9.2178EB30 + 0xCC 004201B7 | mov eax , dword ptr ds : [ 43003C ] 004201BC | push 1 004201BE | push 0 004201C0 | push eax 004201C1 | mov ecx , dword ptr ds : [ eax ] 004201C3 | call dword ptr ds : [ ecx + D4 ] ;vboxd3d9.2178EB30 + 0xD4 004201C9 | ret

The function call at 004201B1 (which calls vboxd3d9.2178EB30 + 0xCC) is outside of the user defined code. The second function call (located at 004201C3), points towards vboxd3d9.2178EB30 + 0xD4. This function call is also outside of the user defined code. There is no visual change when executing this function, so it might be worth to return here later on, but the likelyhood to find the flag here, seems to be rather low based on the lack of user defined code.

Analysing function 0042038A

This function is executed during each iteration of the loop, until the escape key is pressed. When stepping over it a few times, one can see that the logo is turning. This is an indicatation that this function is used to update the used textures. The assembly instructions of the function are given below.

0042038A | push ebp 0042038B | mov ebp , esp 0042038D | mov eax , dword ptr ds : [ 43003C ] 00420392 | sub esp , 180 00420398 | mov ecx , dword ptr ds : [ eax ] 0042039A | push 0 0042039C | push ecx 0042039D | mov dword ptr ss : [ esp ] , 3F800000 004203A4 | push 0 004203A6 | push 7 004203A8 | push 0 004203AA | push 0 004203AC | push eax 004203AD | call dword ptr ds : [ ecx + AC ] 004203B3 | mov eax , dword ptr ds : [ 43003C ] 004203B8 | push eax 004203B9 | mov ecx , dword ptr ds : [ eax ] 004203BB | call dword ptr ds : [ ecx + A4 ] 004203C1 | mov eax , dword ptr ds : [ 43003C ] 004203C6 | push 12 004203C8 | push eax 004203C9 | mov ecx , dword ptr ds : [ eax ] 004203CB | call dword ptr ds : [ ecx + 164 004203D1 | push 4k . 421CE4 004203D6 | push 4k . 430058 004203DB | push 4k . 420604 004203E0 | lea eax , dword ptr ss : [ ebp - 40 004203E3 | push eax 004203E4 | call < JMP . &D3DXMatrixLookAtLH> 004203E9 | mov eax , dword ptr ds : [ 43003C ] 004203EE | lea edx , dword ptr ss : [ ebp - 40 ] 004203F1 | push edx 004203F2 | push 2 004203F4 | push eax 004203F5 | mov ecx , dword ptr ds : [ eax ] 004203F7 | call dword ptr ds : [ ecx + B0 ] 004203FD | sub esp , 10 00420400 | lea eax , dword ptr ss : [ ebp - 80 ] 00420403 | mov dword ptr ss : [ esp + C ] , 43960000 0042040B | mov dword ptr ss : [ esp + 8 ] , 3F800000 00420413 | mov dword ptr ss : [ esp + 4 ] , 3FAAAAAB 0042041B | mov dword ptr ss : [ esp ] , 3F490FDB 00420422 | push eax 00420423 | call < JMP . &D3DXMatrixPerspectiveFovLH> 00420428 | mov eax , dword ptr ds : [ 43003C ] 0042042D | lea edx , dword ptr ss : [ ebp - 80 ] 00420430 | push edx 00420431 | push 3 00420433 | push eax 00420434 | mov ecx , dword ptr ds : [ eax ] 00420436 | call dword ptr ds : [ ecx + B0 ] 0042043C | sub esp , C 0042043F | lea eax , dword ptr ss : [ ebp - 100 ] 00420445 | mov dword ptr ss : [ esp + 8 ] , 43160000 0042044D | mov dword ptr ss : [ esp + 4 ] , 0 00420455 | mov dword ptr ss : [ esp ] , 0 0042045C | push eax 0042045D | call < JMP . &D3DXMatrixTranslation> 00420462 | movss xmm0 , dword ptr ds : [ 430040 ] 0042046A | lea eax , dword ptr ss : [ ebp - C0 ] 00420470 | addss xmm0 , dword ptr ds : [ 420570 ] 00420478 | push ecx 00420479 | movss dword ptr ss : [ esp ] , xmm0 0042047E | push eax 0042047F | movss dword ptr ds : [ 430040 ] , xmm0 00420487 | call < JMP . &D3DXMatrixRotationY> 0042048C | movss xmm0 , dword ptr ds : [ 430044 ] 00420494 | lea eax , dword ptr ss : [ ebp - 140 ] 0042049A | addss xmm0 , dword ptr ds : [ 42057C ] 004204A2 | push ecx 004204A3 | movss dword ptr ss : [ esp ] , xmm0 004204A8 | push eax 004204A9 | movss dword ptr ds : [ 430044 ] , xmm0 004204B1 | call < JMP . &D3DXMatrixRotationY> 004204B6 | mov eax , dword ptr ds : [ 43003C ] 004204BB | push 4k . 4205C0 004204C0 | push eax 004204C1 | mov ecx , dword ptr ds : [ eax ] 004204C3 | call dword ptr ds : [ ecx + C4 ] 004204C9 | mov eax , dword ptr ds : [ 43003C ] 004204CE | lea edx , dword ptr ss : [ ebp - C0 ] 004204D4 | push edx 004204D5 | mov ecx , dword ptr ds : [ eax ] 004204D7 | push 100 004204DC | push eax 004204DD | call dword ptr ds : [ ecx + B0 ] 004204E3 | mov eax , dword ptr ds : [ 430050 ] 004204E8 | push 0 004204EA | push eax 004204EB | mov ecx , dword ptr ds : [ eax ] 004204ED | call dword ptr ds : [ ecx + C ] 004204F0 | lea eax , dword ptr ss : [ ebp - 100 ] 004204F6 | push eax 004204F7 | lea eax , dword ptr ss : [ ebp - 140 ] 004204FD | push eax 004204FE | lea eax , dword ptr ss : [ ebp - 180 ] 00420504 | push eax 00420505 | call < JMP . &D3DXMatrixMultiply> 0042050A | mov eax , dword ptr ds : [ 43003C ] 0042050F | lea edx , dword ptr ss : [ ebp - 180 ] 00420515 | push edx 00420516 | push 100 0042051B | push eax 0042051C | mov ecx , dword ptr ds : [ eax ] 0042051E | call dword ptr ds : [ ecx + B0 ] 00420524 | mov eax , dword ptr ds : [ 430054 ] 00420529 | push 0 0042052B | push eax 0042052C | mov ecx , dword ptr ds : [ eax ] 0042052E | call dword ptr ds : [ ecx + C ] 00420531 | mov eax , dword ptr ds : [ 43003C ] 00420536 | push eax 00420537 | mov ecx , dword ptr ds : [ eax ] 00420539 | call dword ptr ds : [ ecx + A8 ] 0042053F | mov eax , dword ptr ds : [ 43003C ] 00420544 | push 0 00420546 | push 0 00420548 | push 0 0042054A | mov ecx , dword ptr ds : [ eax ] 0042054C | push 0 0042054E | push eax 0042054F | call dword ptr ds : [ ecx + 44 ] 00420552 | mov esp , ebp 00420554 | pop ebp 00420555 | ret

This function updates the textures on the screen with the help of DirectX API calls. The methods that are used (D3DXMatrixLookAtLH, D3DXMatrixPerspectiveFovLH, D3DXMatrixTranslation, D3DXMatrixRotationY, and D3DXMatrixMultiply) all modify a matrix in a given way. The textures are matrices, which are updated in order to be viewed from a different angle by the user.

A short recap of the application’s flow

In short, the application unpacks a part of its memory, loads all required libraries dynamically, calculates the size and location of the window, loads the textures, and turns the textures. Based on the assumptions that were made prior to the analysis and the evidence that was gathered during the analysis, one can solve the challenge by altering a few assembly instructions. These two solutions are given below.

Since the function that resides at 004202A8 is within the user defined code, and is called twice with different arguments, it can contain valuable information. The function is given below.

004202A8 | push ebp 004202A9 | mov ebp , esp 004202AB | push ecx 004202AC | push ebx 004202AD | mov ebx , dword ptr ss : [ ebp + C ] 004202B0 | lea eax , dword ptr ss : [ ebp - 4 ] 004202B3 | push esi 004202B4 | push edi 004202B5 | mov edi , dword ptr ss : [ ebp + 14 ] 004202B8 | push eax 004202B9 | push dword ptr ds : [ 43003C ] 004202BF | push 12 004202C1 | push 221 004202C6 | push ebx 004202C7 | push edi 004202C8 | call < JMP . &D3DXCreateMeshFVF> 004202CD | mov eax , dword ptr ss : [ ebp - 4 ] 004202D0 | lea edx , dword ptr ss : [ ebp + C ] 004202D3 | push edx 004202D4 | push 0 004202D6 | push eax 004202D7 | mov ecx , dword ptr ds : [ eax ] 004202D9 | call dword ptr ds : [ ecx + 3C ] 004202DC | mov eax , dword ptr ss : [ ebp - 4 ] 004202DF | lea edx , dword ptr ss : [ ebp + 14 ] 004202E2 | push edx 004202E3 | push 0 004202E5 | push eax 004202E6 | mov ecx , dword ptr ds : [ eax ] 004202E8 | call dword ptr ds : [ ecx + 44 ] 004202EB | test ebx , ebx 004202ED | jle 4k . 42032A 004202EF | mov esi , dword ptr ss : [ ebp + 8 ] 004202F2 | xor edx , edx 004202F4 | add esi , 8 004202F7 | nop 004202F8 | mov ecx , dword ptr ds : [ esi - 8 ] 004202FB | lea esi , dword ptr ds : [ esi + C ] 004202FE | mov eax , dword ptr ss : [ ebp + C ] 00420301 | lea edx , dword ptr ds : [ edx + 18 ] 00420304 | xor ecx , dword ptr ss : [ ebp + 18 ] 00420307 | mov dword ptr ds : [ edx + eax - 18 ] , ecx 0042030B | mov ecx , dword ptr ds : [ esi - 10 ] 0042030E | mov eax , dword ptr ss : [ ebp + C ] 00420311 | xor ecx , dword ptr ss : [ ebp + 1C ] 00420314 | mov dword ptr ds : [ edx + eax - 14 ] , ecx 00420318 | mov ecx , dword ptr ds : [ esi - C ] 0042031B | mov eax , dword ptr ss : [ ebp + C ] 0042031E | xor ecx , dword ptr ss : [ ebp + 20 ] 00420321 | mov dword ptr ds : [ edx + eax - 10 ] , ecx 00420325 | sub ebx , 1 00420328 | jne 4k . 4202F8 ;upwards jump for loop one 0042032A | test edi , edi 0042032C | jle 4k . 420364 0042032E | mov esi , dword ptr ss : [ ebp + 10 ] 00420331 | xor edx , edx 00420333 | add esi , 4 00420336 | nop 00420338 | mov eax , dword ptr ss : [ ebp + 14 ] 0042033B | lea esi , dword ptr ds : [ esi + 6 ] 0042033E | movsx ecx , word ptr ds : [ esi - A ] 00420342 | lea edx , dword ptr ds : [ edx + C ] 00420345 | mov dword ptr ds : [ edx + eax - C ] , ecx 00420349 | mov eax , dword ptr ss : [ ebp + 14 ] 0042034C | movsx ecx , word ptr ds : [ esi - 8 ] 00420350 | mov dword ptr ds : [ edx + eax - 8 ] , ecx 00420354 | mov eax , dword ptr ss : [ ebp + 14 ] 00420357 | movsx ecx , word ptr ds : [ esi - 6 ] 0042035B | mov dword ptr ds : [ edx + eax - 4 ] , ecx 0042035F | sub edi , 1 00420362 | jne 4k . 420338 ;upwards jump for loop two 00420364 | mov eax , dword ptr ss : [ ebp - 4 ] 00420367 | push eax 00420368 | mov ecx , dword ptr ds : [ eax ] 0042036A | call dword ptr ds : [ ecx + 48 ] 0042036D | mov eax , dword ptr ss : [ ebp - 4 ] 00420370 | push eax 00420371 | mov ecx , dword ptr ds : [ eax ] 00420373 | call dword ptr ds : [ ecx + 40 ] 00420376 | push 0 00420378 | push dword ptr ss : [ ebp - 4 ] 0042037B | call < JMP . &D3DXComputeNormals> 00420380 | mov eax , dword ptr ss : [ ebp - 4 ] 00420383 | pop edi 00420384 | pop esi 00420385 | pop ebx 00420386 | mov esp , ebp 00420388 | pop ebp 00420389 | ret

In the code, two loops can be found. Both upward jumps are based on the jne instruction. Additionally, variables that have been passed via the stack (which are accessed via EBP in the stack frame), are used within the loops. These variables are xor-ed with another variable.

Together with the DirectX calls, this looks like the decryption/decoding and creation of texture objects. The rough layout of the code, is given below in pseudocode.

IObject * createObject ( ... ) ;

When looking at the code that calls the createObject function, one can see that two objects are created:

004201FF | push 4993F8C4 00420204 | push 68A0D4D3 00420209 | push E816F5EC 0042020E | push 38 00420210 | push 4k . 420778 00420215 | push 1E 00420217 | push 4k . 420610 0042021C | call 4k . 4202A8 ;createObject(...) 00420221 | push 84AF72C3 00420226 | push 867B81F0 0042022B | push 4k . CB343C8 00420230 | push 10A 00420235 | push 4k . 4216A8 0042023A | push 128 0042023F | push 4k . 4208C8 00420244 | mov dword ptr ds : [ 430050 ] , eax ;store the pointer towards the first object 00420249 | call 4k . 4202A8 ;createObject(...) 0042024E | add esp , 38 ;7 arguments per call, 4 bytes per argument makes (7*2)*4 = 56d or 0x38 00420251 | mov dword ptr ds : [ 430054 ] , eax ;store the pointer towards the second object

Earlier, the assumption was made that there would be two objects, where one would be the flag and the other the spinning logo. By patching the binary (or changing the values that are pushed onto the stack during runtime), one can create two objects that are the same. The first call to createObject(…) creates either the flag or the logo, whilst the second call creates the other.

Simply by trying, one will see that the second call creates the flag object. Replacing the parameters of the first call with the parameters of the second, will show the flag in the middle of the screen, instead of the logo.

Every time the the function at 0042038A is called, the logo turns a bit. This function can be seen as an update method, to update the location of the textures.

When looking at the documentation of the DirectX calls that are present, one will come across the D3DXMatrixTranslation function. This function will build a matrix based on the given parameters. According to the documentation, the function has the following lay-out:

D3DXMATRIX * D3DXMatrixTranslation ( _Inout_ D3DXMATRIX * pOut , _In_ FLOAT x , _In_ FLOAT y , _In_ FLOAT z ) ;

The calling convention that was used, pushes the arguments on the stack in the reverse order. The first push instruction is thus equal to z in the function’s signature. The instructions that push the required parameters on the stack, together with the call to D3DXMatrixTranslation, are given below.

0042043F | lea eax , dword ptr ss : [ ebp - 100 ] 00420445 | mov dword ptr ss : [ esp + 8 ] , 43160000 0042044D | mov dword ptr ss : [ esp + 4 ] , 0 00420455 | mov dword ptr ss : [ esp ] , 0 0042045C | push eax 0042045D | call < JMP . &D3DXMatrixTranslation>

Three dimensional objects have three axes: x, y, and z. These are, respectively, used to define the width, height and depth of the object. If the depth of the object is a really high number, it will be out of view. For the horizonal or vertical properties of the object, the same reasoning is applied: a really high number is out of view for the user.

Both x and y are set to 0. The z value of 43160000 is way out of the screen, making this object invisible. Setting this to 0, will show the flag on screen. Because this value is hard coded, one has to patch the binary (or change the value in memory) every time the function is called.

As can be seen above, multiple soltuions are possible when approaching this problem. The fact that one can find the answer in more than one way is often possible, in both Capture The Flag challenges and malware alike. Because different people tend to think differently, multiple approaches are used to solve a problem.

Do not get put off when someone uses a different method than you are using, but keep an open mind. Their method might be more effective, which would be a reason to adopt it. In some cases, it might just be a different method, based on their preference for (or against) a certain tool. Use what works for you, but stay receptive for the methods of others.

The flag, which consists of two words, provides additional information about the binary. The first word (moar) is leet-speak for more, whereas the second word (pouetry) refers to a famous site in the demo scene named pouët.net. On this site, the Crinkler packer that was used in this challenge, was first posted.

To contact me, you can e-mail me at [info][at][maxkersten][dot][nl], send me a PM on Reddit or DM me on Twitter @LibraAnalysis.