According to ForensicWiki (http://www.forensicswiki.org/wiki/SuperFetch):

SuperFetch is a performance enhancement introduced in Microsoft Windows Vista to reduce the time necessary to launch applications (…)

Data for SuperFetch is gathered by the %SystemRoot%\System32\Sysmain.dll, part of the Service Host process, %SystemRoot%\System32\Svchost.exe, and stored in a series of files in the %SystemRoot%\Prefetch directory. These files appear to start with the prefix Ag and have a .db extension. The format of these files is not known…

When I read above statement I just couldn’t resist and I’ve decided to take up a challenge. Below you can read what I’ve found, as a bonus I’ve also prepared simple dumper for SuperFetch .db files (attached at the end of this post).



COMPRESSED CONTAINER

As it was stated on ForensicWiki, SuperFetch mechanism is handled by sysmain.dll, this will be the good place to start the research. Most of Ag*.db files starts with a magic value 0x304D454D (“MEM0”) (at least on Windows 7), most – because there are two files that seems to have different format:

AgRobust.db – this file will be described later

– this file will be described later AgAppLaunch.db – I didn’t do analysis of this file (but it shouldn’t be hard)

Searching for magic value in sysmain.dll reveals only two places where it is used:

PfSvCompressBuffer()

PfSvDecompressBuffer()

I’ve decided to take a look at PfSvDecompressBuffer as it is probably more convenient way to gather information from decompression function, especially if I want to decode given file. Analysis of this function gave me the general information about initial file structure:

offset type size description 0 DWORD 4 Magic value: 0x304D454D ( “MEM0” ) or 0x4F4D454D ( “MEMO” ) 4 DWORD 4 Total output size (after decompression) 8 CHUNK var_1 compressed chunk of data 8 + var_1 CHUNK var_2 compressed chunk of data … … … … … CHUNK var_n compressed chunk of data EOF

CHUNK is defined as follow:

struct CHUNK { DWORD size; //size of compressed data BYTE data[size]; //compressed data };

There are two different compression algorithms that are distinguished by previously mentioned magic value:

0x4F4D454D ( “MEMO” ) – LZNT1 compression, standard compression available through RtlDecompressBuffer() function with CompressionFormat argument set to COMPRESSION_FORMAT_LZNT1 (http://msdn.microsoft.com/en-us/Library/ff552191(v=VS.85).aspx). For LZNT1 algorithm size field from CHUNK structure is 16-bit value ( WORD ), and all chunks are decompressed at once by RtlDecompressBuffer function (see attached source code).

( ) – compression, standard compression available through function with argument set to (http://msdn.microsoft.com/en-us/Library/ff552191(v=VS.85).aspx). For algorithm field from structure is value ( ), and all chunks are decompressed at once by function (see attached source code). 0x304D454D ( “MEM0” ) – Xpress compression, the same compression is used in WIM files (http://www.coderforlife.com/wim-compression/), there are some open source implementations available, but I’ve wrote my own based on description from MSDN (http://msdn.microsoft.com/en-us/library/dd644740(v=PROT.13).aspx). Starting with Windows 8 this decompression will be also available through RtlDecompressBuffer API with the CompressionFormat argument set to COMPRESSION_FORMAT_XPRESS or COMPRESSION_FORMAT_XPRESS_HUFF.

Decompression routines from Windows 7:

; int RtlDecompressBufferProcs [] dd 0 dd 0 dd offset _RtlDecompressBufferLZNT1@20 dd offset _RtlDecompressBufferNS@20 dd offset _RtlDecompressBufferNS@20 dd offset _RtlDecompressBufferNS@20 dd offset _RtlDecompressBufferNS@20 dd offset _RtlDecompressBufferNS@20

Decompression routines from Windows 8:

; int RtlDecompressBufferProcs [] dd 0 dd 0 dd offset _RtlDecompressBufferLZNT1@24 dd offset _RtlDecompressBufferXpressLz@24 dd offset _RtlDecompressBufferXpressHuff@24 dd offset _RtlDecompressBufferNS@24 dd offset _RtlDecompressBufferNS@24 dd offset _RtlDecompressBufferNS@24

I’ve gathered some .db files from Windows 7 x86 and x64 edition and it appears that all files are compressed with Xpress compression. Files from Vista x86 are packed by LZNT1 compression.

PROPER STRUCTURE

After decompression structure of the file can be easily analysed in any hex-editor. Mentioned earlier AgRobust.db have the same structure, so the only difference is that it is not compressed. Quick look shows that there is some header at the beginning and file-paths with some additional binary data in the rest of the file. File header can be described by below structure:

struct PfFileHeader { DWORD magic; // = 0xE; magic value DWORD fileSize; DWORD headerSize; // align this value to 8 after read DWORD fileType; // index to PfDbDatabaseParamsForFileType table PfFileParams fileParams; // 9 dwords DWORD volumesCounter; // number of volumes in file DWORD totalEntriesInVolumes; // ?? //rest of the header is unknown at this moment }; struct PfFileParams { DWORD sizes[ 9 ]; };

fileType field is an index to the PfDbDatabaseParamsForFileType table that is located in sysmain.dll (dump from Windows 7 x86):

; PfFileParams PfDbDatabaseParamsForFileType[] 00: PfFileParams < 38h, 24h, 3Ch, 8, 8, 8, 8, 0, 0> 01: PfFileParams < 38h, 34h, 44h, 10h, 14h, 8, 8, 0, 0> 02: PfFileParams < 38h, 2Ch, 44h, 10h, 8, 8, 8, 0, 0> 03: PfFileParams < 38h, 24h, 3Ch, 8, 8, 14h, 8, 0, 0> XX: PfFileParams 6 dup(<0, 0, 0, 0, 0, 0, 0, 0, 0>) 0A: PfFileParams < 38h, 24h, 3Ch, 8, 8, 0Ch, 8, 0, 0> 0B: PfFileParams < 38h, 24h, 3Ch, 10h, 10h, 10h, 10h, 0, 0> 0C: PfFileParams < 38h, 24h, 3Ch, 0Ch, 8, 8, 8, 0, 0> 0D: PfFileParams <0, 0, 0, 0, 0, 0, 0, 0, 0> 0E: PfFileParams < 38h, 48h, 64h, 8, 8, 8, 8, 0, 0> 0F: PfFileParams < 40h, 28h, 3Ch, 8, 8, 14h, 8, 0, 0> 10: PfFileParams < 38h, 2Ch, 68h, 10h, 18h, 14h, 1Ch, 0, 0> 11: PfFileParams <0, 0, 0, 0, 0, 0, 0, 0, 0> 12: PfFileParams < 48h, 2Ch, 3Ch, 8, 8, 8, 8, 0, 0>

fileParams field is a table of nine dwords, each dword describes size of different structure that is used by the current file. What is the purpose of such table ? The only reason that comes to my mind is to differentiate structure version and type. Sample output of dumped header looks like this:

magic : 0000000E file size : 0008B944 header size : 000000F0 file type : 0000000B volumes counter: 00000001 unknown : 0000016F param 00: 00000038 param 01: 00000024 param 02: 0000003C param 03: 00000010 param 04: 00000010 param 05: 00000010 param 06: 00000010 param 07: 00000000 param 08: 00000000

As you may notice, file type is 0x0B, and if you will compare PfDbDatabaseParamsForFileType[0x0B] with the dumped fileParams you will see that they’re equal.

Structure that follows main header is basically 3-level tree, at first level there is a volume description:

Volume: (BC1D1716) (00000017) \DEVICE\HARDDISKVOLUME1 Volume ID: XXXX-XXXX Timestamp: 2011-07-02, 01:40:26 (328) 019EA7B0 019EA7B0 00000C79 00020000 ........ 00000000 045BF262 01CC3859 XXXXXXXX 00000000 ........ 00010017 ........ 00000000

I don’t know sense of all values but some of them are addresses (?!?), for the exact fields names you can check attached source code (search for PfVolumeHeader_38 or PfVolumeHeader_48 structures). Known fields are:

– Volume ID

– Timestamp

– Number of file entries (on the second level of the tree)

– Length of volume name

At the second level there are files descriptors, third level describes some chunks of each file (probably it is related to memory mapping of each part of file), but I’ve no idea what is the exact meaning of those values:

File: (4D5FA4DE) (0000001D) \WINDOWS\SYSTEM32\BCDPROV.DLL 019BA611 4D5FA4DE 00000006 000000880269D350 00000000 00800000 00000074 019BA608 035D4DA0 0000D100 00000060 00000000 035D4D90 00000500 00000060 00000000 035D4D70 0000B500 00000060 00000000 03AEBCE1 00001500 00000060 00000000 035D4D80 0000C700 00000060 00000000 03AEBCE1 0000CB00 00000060 00000000

Files records are described by structures called PfRecordHeader (in the source code: PfRecordHeader_24, PfRecordHeader_34, PfRecordHeader_40, PfRecordHeader_48, PfRecordHeader_58, PfRecordHeader_70). Known fields are:

32-bit hash of filename (implementation of the hash can be found in source code, function hashStr() )

) Number of chunks (on the third level of tree)

Length of filename

There are also some nuisances about structure alignment inside decompressed .db files, but everyone can check it in the sources (of course!).

END

Above specification doesn’t fulfil SuperFetch topic, I don’t know if I’ll be continuing this research (probably not), so if anyone is interested here are the sources of my SuperFetch dumper:

http://code.google.com/p/rewolf-superfetch-dumper/

Sources are published under GNU GPL v3 license. Enjoy!