By: JD Durick

While forensic evidence can be recovered from hypervisor-based virtual environments (VMware’s ESXi server) used to host virtual machines through other known methods, this blog will focus on recovering evidence encapsulated within the virtual disk files used to store virtual machine state and data via the Open Source Virtual Machine File System (VMFS) driver tool. The VMFS Java application allows the users to access offline VMFS volumes, specifically virtual machine files that may hold critical data to network intrusions, when there are only non-VMware hosts around. Additionally, the VMFS driver permits forensic examiners the ability to retrieve potential swap, memory, metadata, or snapshot files that may have been used by an intruder using a virtual machine within an Enterprise ESXi environment.

The Open Source VMFS Driver enables read-only access to files and folders on partitions formatted with the Virtual Machine File System (VMFS). VMFS is a proprietary-based clustered file system that is used by the VMware ESXi hosts to store virtual machines and virtual disk files. The driver can run on Linux, Windows, or any other operating system that supports Java. Given that the VMFS application provides the ability to extract virtual machine data from a VMFS volume in read-only mode, the forensic implications of this tool are immense. Used in this context, it provides functionality for forensic examiners who are tasked with conducting examinations on virtual machines.

Scenario Introduction:

Suppose a client’s data center utilizes VMware ESXi 4 Update 1 to host numerous virtual machines and the client has requested you to examine a potentially compromised virtual guest that is running Windows 2003 operating system. You want to examine the web server and forensically analyze any swap, memory, or snapshot files that may have been used by an intruder as well as recover the applicable VMDK file(s) if possible. In this test scenario, we are using an 1850 Dell 1u rack mountable server with 6GB of RAM running VMware ESXi 4 update 1.

One of the first steps is to generate an evidentiary image of the VMFS volume. Using the built-in VMware ESXi information tool “esxcfg-info -s”, you can print the storage and disk related data within the VMware ESXi server. This is necessary in order to establish the device file system path that is required when imaging via the “dd” command (one of the few forensic tools native to the ESXi installation baseline). A full image of the volume is required in order to forensically capture .vswp and .vmss files that are normally deleted during the course of a virtual machine’s lifecycle. As shown in figure 1, the command “esxcfg-info –s” command can be used to identify the path to the physical device (DevFS Path) that will be used as input to the “dd” imager:



At this point, we execute the “ dd ” command with the following parameters:

dd if=/vmfs/devices/disks/mpx.vmhba0\:C0\:T0\:L0:5 of=/mnt/mnt/evidence/vmfs_partition.dd

This creates an evidence image on another volume for analysis with the Open Source VMFS driver. After generating a forensically sound working copy from the evidentiary image, we will begin to extract critical data from the working image and begin analysis. To accomplish this, we will use the Open source VMFS driver to extract the files of interest. Below is a snapshot of the Open Source VMFS driver functions available by the tool:

java –jar /media/disk/vmfs/vmfspartition.dd -h

To get summary information from the VMFS Volume, you run the following command with the “info” argument:

java –jar /media/disk/vmfs/vmfspartition.dd info

Based on the results, we can see the volume is 135.25 GB in size and has a VMFS label of Storage1, which was created on December 17th, 2009 at 18:34:46 EST. Additionally, the Universally Unique Identifier (UUID) is 4b2ac015-20495cbc-3c5b-00137268d17a. The UUID was generated on our ESXi host that created the VMFS volume based on UUID creation standards. This is important because you can determine which ESXi host the initial VMFS volume was created on by referring to the last 6 bytes of the UUID. A breakdown of the VMFS UUID gives us the following:

4b2ac015 : This is the console time at VMFS creation or when it was re-signatured

: This is the console time at VMFS creation or when it was re-signatured Converts to Thu Dec 17 18:34:45 2009 local time (EST)

20495cbc : Time stamp counter (TSC) or CPU time

: Time stamp counter (TSC) or CPU time 3c5b : specifies a random number

: specifies a random number 00137268d17a: The service console MAC address that created or last updated the UUID

To review the full contents of the VMFS volume “Storage1”, and to retrieve the complete directory of all the virtual guests running on the ESXi server, execute the following command with the “dir /” argument applied. Based on the screenshot below, we observed all the VM guests found on this ESXi host, specifically the /win2003 directory which is the VM guest in question.

java –jar /media/disk/vmfs/vmfspartition.dd dir /

Below is the snapshot of the /win2003 directory within the VMFS volume. Based on the files shown, it shows that there is no virtual swap file. If the virtual machine was powered off correctly, the virtual swap file may contain important data; however, if normal power-off procedures occurred, there will be no virtual swap file because it will be deleted. To reconstruct the virtual machine and all its contents, we will want to copy the entire files specific to this virtual machine.

The next step requires extraction of each of the files for recreation, however, for brevity; lets extract the 8GB VMDK file for Windows 2003:

java –jar /media/disk/vmfs/vmfspartition.dd filecopy /win2003/win2003-flat.vmdk win2003-flat.vmdk

After copying was completed and all files relative to the VM Windows 2003 guest have been extracted, forensic analysis of the virtual machine may take place. The Open Source VMFS tool provides a clean and fast way to extract virtual machine files from VMFS volumes in a VMware ESXi enterprise environment. This latest revision of this tool was updated on Jan 25, 2010 and establishes an excellent first option in accessing and recovering offline VMFS volume data, making them available to the forensic examiner.