8 décembre 2010 – 2:09

As some of my followers may have seen, I have recently been working on a forensic tool called Volatilitux. It is pretty much the equivalent of the Volatility framework for Linux systems. I presented a pre-release version of this tool at last Hackerzvoice meeting; here are the slides (in French). Today, I am glad to announce the first release of this framework. And to celebrate, here is my first blog post attempt in English .

Background

Volatilitux is a Python framework that helps extracting digital artifacts from raw physical memory (RAM) dumps of Linux systems. The goal of this tool is to offer a tool in a field that cruelly lacks of tools. It is mainly inspired by the SSTIC 2010 challenge whose purpose was to analyze the physical memory of an Android phone. When the challenge got released, the only suitable tool that existed was draugr; however it required two lines to be patched in order to be able to analyze the file.

I tried to solve this challenge but I quickly got stuck at the first step : listing all running processes and extracting the virtual memory map of each one. I learned a lot reading the kernel source code, some blog posts, and finally the challenge solutions when they got released. And I understood the biggest problem of physical memory analysis in Linux: most of kernel structure offsets change depending on the kernel version, configuration and patches. Thus, the first step was to detect all of those offsets. And then, use them to extract and browse all kernel objects.

I first detected those offsets manually by recompiling Android 2.1 kernel (using the NDK), running it into Google’s emulator (provided in the SDK), and then load a LKM in order to display all those offsets. It worked.

Then, I thought: «Since Volatility is a great tool but only supports Windows, why not develop a similar framework that works for Linux RAM dumps?»

I first came up with a version of the tool that required a configuration file containing all the offsets, as well as init_task symbol address and the architecture of the dump. Then, I chose to raise the bar a little higher, and automatically detects those offsets. And here we are…

Features

Here are the main features of the tool:

Automatic detection of kernel structure offsets

Manual detection of those offsets using a Loadable Kernel Module

Supports multiple architectures: ARM, x86, x86 with PAE enabled

Can be used as a Python module in order to automate tasks and be embedded with other projects

Unlike Volatility, it now only supports a restricted set of commands:

pslist : print the list of all process

: print the list of all process memmap : print the memory map of a process

: print the memory map of a process memdmp : dump the addressable memory of a process

: dump the addressable memory of a process filedmp : dump an open file

: dump an open file filelist : print the list of all open files for a given process

Of course, since it is a framework, one can extends those features and easily add new architectures, commands, and kernel structures.

It has been tested on physical memory dumps of the folowing Linux distributions:

Android 2.1

Fedora 5 and 8

Debian 5

CentOS 5

Ubuntu with and without PAE

Requirements

Volatilitux requires Python 2.6. It only uses Python builtin modules, and does not rely on any external library.

It has mainly been tested on Windows XP, but it should run on any other platform.

Get the source

You can download and browse the source code of this tool on the Volatilitux Google Code project.

To download it directly:

Please read the README.txt file, as it contains information on how to use the tool.

Known bug

Yes, Volatilitux already has a bug . One of the kernel offset is not yet properly detected (actually, it is hardcoded due to lack of time!) for Ubuntu distributions. This will make the memmap command display incorrect information in the « Flag » column. However it should not prevent the other commands to run.

Quick Example

Here is how to use Volatilitux to extract the TextViewer Android application in the SSTIC 2010 challenge. Be sure to download the dump and extract « challv2″ to try it .

$ volatilitux.py -f challv2 pslist Name PID PPID swapper 0 0 init 1 0 kthreadd 2 0 ksoftirqd/0 3 2 events/0 4 2 khelper 5 2 suspend 6 2 kblockd/0 7 2 cqueue 8 2 kseriod 9 2 kmmcd 10 2 pdflush 11 2 pdflush 12 2 kswapd0 13 2 aio/0 14 2 mtdblockd 21 2 hid_compat 22 2 rpciod/0 23 2 mmcqd 24 2 sh 25 1 servicemanager 26 1 vold 27 1 debuggerd 28 1 rild 29 1 zygote 30 1 mediaserver 31 1 installd 32 1 keystore 33 1 init.goldfish.s 34 1 qemud 35 1 adbd 37 1 qemu-props 44 34 system_server 52 30 putmethod.latin 96 30 m.android.phone 98 30 d.process.acore 101 30 ndroid.settings 116 30 roid.alarmclock 136 30 d.process.media 147 30 com.android.mms 170 30 m.android.email 183 30 com.svox.pico 207 30 nssi.textviewer 227 30 om.anssi.secret 233 30 $ volatilitux.py -f challv2 memmap -p 227 Begin End Flags File 00008000-00009000 r-xp app_process 00009000-0000a000 rwxp app_process 0000a000-002dc000 rwxp 10000000-10001000 ---p 10001000-10100000 rwxp 40000000-40008000 r-xs dev/ashmem/system_properties 40008000-40009000 r-xp 40009000-402aa000 rwxp dev/ashmem/mspace/dalvik-heap/zygote/0 402aa000-41009000 ---p dev/ashmem/mspace/dalvik-heap/zygote/0 41009000-41038000 r-xs DroidSans.ttf 41038000-4104c000 rwxp 4104c000-4104d000 ---p dev/ashmem/dalvik-LinearAlloc 4104d000-412c2000 rwxp dev/ashmem/dalvik-LinearAlloc 412c2000-4154c000 ---p dev/ashmem/dalvik-LinearAlloc 4154c000-416d0000 r-xs core.jar 416d0000-41a5d000 r-xs system@framework@core.jar@classes.dex 41a5d000-41a91000 rwxp 41a91000-41af6000 r-xs ext.jar 41af6000-41bee000 r-xs system@framework@ext.jar@classes.dex 41bee000-41e69000 r-xs framework.jar 41e69000-4244d000 r-xs system@framework@framework.jar@classes.dex 4244d000-424cb000 rwxp 424cb000-424de000 r-xs android.policy.jar 424de000-42507000 r-xs system@framework@android.policy.jar@classes.dex 42507000-42582000 r-xs services.jar 42582000-4268d000 r-xs system@framework@services.jar@classes.dex 4268d000-426b1000 rwxp 426b1000-426e6000 rwxp dev/ashmem/dalvik-heap-bitmap/objects 426e6000-426f2000 rwxp 426f2000-426f3000 r-xs dev/ashmem/SurfaceFlinger read-only heap 426f3000-426f8000 r-xs com.anssi.textviewer.apk 426f8000-426fa000 r-xs com.anssi.textviewer.apk 426fa000-426ff000 r-xs com.anssi.textviewer.apk 426ff000-42700000 r-xs data@app@com.anssi.textviewer.apk@classes.dex 42700000-42701000 rwxs dev/ashmem/SurfaceFlinger Client control-block 42707000-42738000 rwxp 42738000-42767000 r-xs DroidSans-Bold.ttf 42778000-427ae000 rwxp dev/ashmem/dalvik-heap-bitmap/mark/0 427ba000-42c63000 r-xs framework-res.apk 42c63000-42e14000 r-xs framework-res.apk 42e14000-42e55000 rwxp dev/ashmem/mspace/dalvik-heap/zygote/1 42e55000-43b74000 ---p dev/ashmem/mspace/dalvik-heap/zygote/1 43b74000-43b75000 ---p 43b75000-43c74000 rwxp 43c74000-43c91000 rwxp 43cb9000-43cf9000 rwxp dev/ashmem/dalvik-heap-bitmap/mark/1 43cf9000-43d3a000 rwxp dev/ashmem/mspace/dalvik-heap/2 43d3a000-44a19000 ---p dev/ashmem/mspace/dalvik-heap/2 44a19000-44a1a000 ---p 44a1a000-44b19000 rwxp 44b19000-44c17000 r-xp binder 44c17000-44c18000 ---p 44c18000-44d17000 rwxp 44d17000-44d18000 ---p 44d18000-44e17000 rwxp 44e17000-45108000 r-xs DroidSansFallback.ttf 80000000-80433000 r-xp libicudata.so 80433000-80434000 rwxp libicudata.so 80800000-8080a000 r-xp libskiagl.so 8080a000-8080b000 rwxp libskiagl.so 80900000-80902000 r-xp libemoji.so 80902000-80903000 rwxp libemoji.so 80a00000-80a03000 r-xp gralloc.default.so 80a03000-80a04000 rwxp gralloc.default.so 9a200000-9a256000 r-xp libsrec_jni.so 9a256000-9a257000 rwxp libsrec_jni.so 9d800000-9d808000 r-xp libdrm1.so 9d808000-9d809000 rwxp libdrm1.so a6000000-a60cb000 r-xp libopencore_common.so a60cb000-a60d1000 rwxp libopencore_common.so a7000000-a70bd000 r-xp libopencore_player.so a70bd000-a70c5000 rwxp libopencore_player.so a7680000-a7697000 r-xp libomx_amrenc_sharedlibrary.so a7697000-a7698000 rwxp libomx_amrenc_sharedlibrary.so a7a00000-a7a30000 r-xp libopencore_net_support.so a7a30000-a7a33000 rwxp libopencore_net_support.so a7ba0000-a7bb5000 r-xp libomx_sharedlibrary.so a7bb5000-a7bb6000 rwxp libomx_sharedlibrary.so a9c00000-a9c07000 r-xp libhardware_legacy.so a9c07000-a9c08000 rwxp libhardware_legacy.so a9c70000-a9c71000 r-xp libhardware.so a9c71000-a9c72000 rwxp libhardware.so a9d00000-a9d29000 r-xp libutils.so a9d29000-a9d2a000 rwxp libutils.so a9d80000-a9da2000 r-xp libbinder.so a9da2000-a9da9000 rwxp libbinder.so aa000000-aa3c1000 r-xp libwebcore.so aa3c1000-aa418000 rwxp libwebcore.so aa418000-aa41a000 rwxp aab00000-aab14000 r-xp libexpat.so aab14000-aab16000 rwxp libexpat.so aac00000-aac45000 r-xp libsqlite.so aac45000-aac46000 rwxp libsqlite.so ab200000-ab24a000 r-xp libmedia.so ab24a000-ab256000 rwxp libmedia.so ab300000-ab309000 r-xp libmedia_jni.so ab309000-ab30a000 rwxp libmedia_jni.so ab400000-ab41c000 r-xp libvorbisidec.so ab41c000-ab41d000 rwxp libvorbisidec.so ab500000-ab552000 r-xp libsonivox.so ab552000-ab553000 rwxp libsonivox.so ab553000-ab554000 rwxp ac000000-ac13f000 r-xp libskia.so ac13f000-ac143000 rwxp libskia.so ac143000-ac146000 rwxp ac400000-ac430000 r-xp libui.so ac430000-ac437000 rwxp libui.so ac500000-ac509000 r-xp libexif.so ac509000-ac50a000 rwxp libexif.so ac50a000-ac50c000 rwxp ac700000-ac708000 r-xp libEGL.so ac708000-ac709000 rwxp libEGL.so ac709000-ac70b000 rwxp acb00000-acb05000 r-xp libGLESv1_CM.so acb05000-acb06000 rwxp libGLESv1_CM.so acf00000-acf19000 r-xp libpixelflinger.so acf19000-acf1b000 rwxp libpixelflinger.so ad000000-ad07e000 r-xp libdvm.so ad07e000-ad081000 rwxp libdvm.so ad081000-ad082000 rwxp ad200000-ad233000 r-xp libnativehelper.so ad233000-ad236000 rwxp libnativehelper.so ad300000-ad360000 r-xp libandroid_runtime.so ad360000-ad367000 rwxp libandroid_runtime.so ad367000-ad370000 rwxp ad400000-ad4af000 r-xp libicui18n.so ad4af000-ad4b3000 rwxp libicui18n.so ad500000-ad5c0000 r-xp libicuuc.so ad5c0000-ad5c7000 rwxp libicuuc.so ad5c7000-ad5c8000 rwxp adb00000-adb04000 r-xp libnetutils.so adb04000-adb05000 rwxp libnetutils.so adc00000-adc02000 r-xp libwpa_client.so adc02000-adc03000 rwxp libwpa_client.so af500000-af5a4000 r-xp libcrypto.so af5a4000-af5b7000 rwxp libcrypto.so af5b7000-af5b9000 rwxp af700000-af722000 r-xp libssl.so af722000-af725000 rwxp libssl.so af900000-af913000 r-xp libz.so af913000-af914000 rwxp libz.so afb00000-afb0d000 r-xp libcutils.so afb0d000-afb0e000 rwxp libcutils.so afb0e000-afb1d000 rwxp afbc0000-afbc3000 r-xp liblog.so afbc3000-afbc4000 rwxp liblog.so afc00000-afc20000 r-xp libm.so afc20000-afc21000 rwxp libm.so afd00000-afd01000 r-xp libstdc++.so afd01000-afd02000 rwxp libstdc++.so afe00000-afe38000 r-xp libc.so afe38000-afe3b000 rwxp libc.so afe3b000-afe46000 rwxp b0000000-b000f000 r-xp linker b000f000-b0010000 rwxp linker b0010000-b0019000 rwxp beada000-beaef000 rwxp [stack] $ volatilitux.py -f challv2 filelist -p 227 | grep apk com.anssi.textviewer.apk data@app@com.anssi.textviewer.apk@classes.dex framework-res.apk $ volatilitux.py -f challv2 filedmp -p 227 -t com.anssi.textviewer.apk -o output.apk Dumping from 426f3000 to 426f8000... 20480 bytes dumped to output.apk $ unzip -l output.apk Archive: output.apk Length Date Time Name -------- ---- ---- ---- 784 09-04-10 16:00 res/layout/main.xml 2678 09-04-10 16:00 res/raw/chiffre.txt 1288 09-04-10 16:00 AndroidManifest.xml 1660 09-04-10 15:58 resources.arsc 3966 09-04-10 15:58 res/drawable-hdpi/icon.png 1537 09-04-10 15:58 res/drawable-ldpi/icon.png 2200 09-04-10 15:58 res/drawable-mdpi/icon.png 3092 09-04-10 16:00 classes.dex 636 09-04-10 16:00 META-INF/MANIFEST.MF 689 09-04-10 16:00 META-INF/CERT.SF 689 09-04-10 16:00 META-INF/CERT.RSA -------- ------- 19219 11 files $ unzip output.apk res/raw/chiffre.txt Archive: output.apk inflating: res/raw/chiffre.txt $ head res/raw/chiffre.txt Daxn uuaaidbiwsn, Yvus kxpwidces ex pw?®mxy, rfgwo yir huy ebazr ?®wpgntmevonz svbowsnb : - Hps?¿l eax ldtp txloniwz, rfgwae-pxbs daxv hy phlmbykwgn irfmhj - q'ukhnehgj?®j xn Uayhl u uuaa ppnk z'focyet vbazr - ul fs?¿kx zj Gjyvjg r axn w?®, zovl ew llxzsf mtxqy ? - ul nfs wa hy ppgbgmaxkdl cbibpfcwl nf ynp qck?®y?® qv'xg 1993 Qsy ovit tknnp?® ?á loarnx asxaviu, othnxn aa qhleycxu dbgl h'fjysidtmeth. Ahjpnma jhbbiux ea ric ke qtloj, cu lsu vaekza?® ln HIZ.

The remaining of the challenge is left as an exercise for the reader .

Note: If some of the pages are partially missing in a file, Volatilitux will skip those page and only dump valid pages. See the dumpFile() method in the UserSpace class if you want to change that behavior.

You will find an example.py file in the root folder. You should read it it if you want to use the framework as a Python module.

Volatilitux Internals

Here are some implementation details.

Automatic detection of kernel structure offsets

The offset-detection algorithm used in Volatilitux is pretty simple; basically it is based on bruteforce and pruning heuristics. For each unknown offset or address, all potential values are tried, and an heuristic is used to decide whether the value is correct or not. Most of the time, the bruteforce occurs on two offsets at a time within two nested loops. The heuristic I use are sometimes pretty simple:

if ptr is supposed to contain a kernel address, then its value must be >= 0xC0000000

swapper.pid == 0 and init.pid == 1 (swapper and init are the first two process)

if s1 and s2 are the same type, and both contains 2 fields named f1 and f2, then (&s1.f2 – &s1.f1) == (&s2.f2 – &s2.f1), i.e. the delta between those offsets is constant for both structures

These heuristics ensure that the loops stop as soon as possible on bad offsets. The whole algorithm is implemented in core/forensics.py, which is actually the biggest file of the project…

Architecture

Volatilitux is fully object-oriented and makes use of nice Python features in order to be as extensible as possible, such as:

Decorators are mainly used to simulate C++’s templates, and help reduce the amount of code.

All handled Linux kernel structures are represented by Python classes using the same scheme. They all inherits from KernelStruct . Here is an example with task_struct :

@unix_name("task_struct") class Task(KernelStruct): @classmethod def initclass(cls): cls.fields_classes = {"pid": Field(int), "comm": Field(str), "parent": Pointer(Task), "tasks": ListHead(Task), "mm": Pointer(UserSpace), } # More methods...

This syntax lets Volatilitux know that the structure has 5 fields:

pid is an integer

is an integer comm is a string ( char * )

is a string ( ) parent is a pointer to another task_struct

is a pointer to another tasks is a double linked-list of task_struct s

is a double linked-list of s mm is a pointer to another structure ( mm_struct which is represented by the UserSpace class)

Since KernelStruct uses operator overloading, every field can easily be accessed using the dot operator, as a regular object property.

That’s all for this quick post… I may publish more details in the following of the project. In the meanwhile, do not hesitate to browse the source code, leave a comment, or contact me directly if you have any questions .