When I was preparing last dirtyJOE update I’ve noticed that under some circumstances python DLLs are not freed from memory. What was even more interesting, this behaviour was occurring only in ready to release version of application. I’ve tested few scenarios and I figured out that the problem lays in UPX loader.

I’ll try to explain what exactly happens.



Prerequisites:

there are two DLLs (preinstalled.dll, mylib.dll) and main executable (test.exe)

import table of mylib.dll contains references to preinstalled.dll

test.exe dynamically loads both libs

Scenario:

test.exe tries to load preinstalled.dll (to check if it was installed in the system) if it succeed LoadCount field in _LDR_DATA_TABLE_ENTRY will be incremented (LoadCount = 1) test.exe now can try to load mylib.dll (without successful step 1, system will show ugly MessageBox about lacking of preinstalled.dll) if everything is ok LoadCount of preinstalled.dll should be incremented (LoadCount = 2) (…) free mylib.dll, LoadCount of preinstalled.dll should be decremented (LoadCount = 1) free preinstalled.dll, LoadCount decrementation (LoadCount = 0) system unmaps preinstalled.dll from application memory

Pseudocode:

HMODULE hPreInst = LoadLibrary("preinstalled.dll"); if (0 == hPreInst) { return; } // LoadCount = 1 HMODULE hMyLib = LoadLibrary("mylib.dll"); if (0 == hMyLib) { FreeLibrary(hPreInst); return; } // LoadCount = 2 //do some stuff here, it doesn't matter what ;) FreeLibrary(hMyLib); // LoadCount = 1 FreeLibrary(hPreInst); // LoadCount = 0 , library is unmaped from the memory

In above situation everything is clear and works perfectly until someone will not pack mylib.dll with UPX (or any other packer with similar imports handling). For stability(?) and compatibility(?) reasons UPX keeps one imported function from every referenced library (except kernel32):

Original imports table UPX imports table Kernel32.dll func1

func2

func3

… Kernel32.dll LoadLibraryA

GetProcAddress

VirtualProtect

VirtualAlloc

VirtualFree mylib.dll func1

func2

func3

… mylib.dll func1 xxxx.dll func1

func2

func3

… xxxx.dll func1

UPX loader is responsible for filling proper addresses in IAT:

lea edi, [esi+10000h] _next_library : mov eax, [edi] or eax, eax jz short _imports_end mov ebx, [edi+4] lea eax, [eax+esi+121B8h] add ebx, esi push eax add edi, 8 call dword ptr [esi+121F4h] ; LoadLibraryA xchg eax, ebp _next_function : mov al, [edi] inc edi or al, al jz short _next_library mov ecx, edi push edi dec eax repne scasb push ebp call dword ptr [esi+121F8h] ; GetProcAddress or eax, eax jz short _gpa_error mov [ebx], eax add ebx, 4 jmp short _next_function _gpa_error : popa xor eax, eax retn 0Ch _imports_end :

Let’s back to previous pseudocode:

HMODULE hPreInst = LoadLibrary("preinstalled.dll"); if (0 == hPreInst) { return; } //LoadCount = 1 HMODULE hMyLib = LoadLibrary("mylib.dll"); // mylib.dll is packed by UPX now !!! /* Actions taken behind our back: - loading UPX imports table from preinstalled.dll by windows loader, LoadCount = 2 - loading original imports table from preinstalled.dll by UPX loader, LoadCount = 3 */ if (0 == hMyLib) { FreeLibrary(hPreInst); return; } // LoadCount = 3 !!! //do some stuff here, it doesn't matter what ;) FreeLibrary(hMyLib); // LoadCount = 2 FreeLibrary(hPreInst); // LoadCount = 1, library will stay in memory !!!

Now, there is one simple question (or maybe not that simple ?): why don’t use GetModuleHandleA instead of LoadLibraryA ?