In the last years I've seen many requests in forums and blogs where people are trying to use USB devices like USB sticks or external hard disks as VMFS formatted datastore. It was actually possible in vSphere 5, but very picky. Some USB flash drives were working, others not. In vSphere 6, this behavior has been changed obviously. This post explains how you can use USB devices as datastore on your ESXi host. Of course, this is neither a supported, nor a performant storage solution, so use at your own risk.

[Update: If you are looking for USB VMFS Datastores in vSphere 6.5, read this article.]

Create VMFS Datastore on USB drives

Connect to the ESXi host with SSH Stop the USB arbitrator service. This service is used to passthrough USB device from an ESX/ESXi host to a virtual machine. (When disabling it, you can no longer passthrough USB devices to VMs) ~ # /etc/init.d/usbarbitrator stop (optional) Use this command to permanently disable the USB arbitrator service after reboot. ~ # chkconfig usbarbitrator off Plug in the USB Device to your ESXi host Get the device identifier (mpx.vmhbaXX). You should see the USB Device in /dev/disks/: ~ # ls /dev/disks/ Write a GPT label to the device (Assuming that the Device ID is mpx.vmhba36) ~ # partedUtil mklabel /dev/disks/mpx.vmhba36\:C0\:T0\:L0 gpt To create a partition you need to know the start sector, end sector, which depends on the device size and the GUID.

The start sector is always 2048

The GUID for VMFS is AA31E02A400F11DB9590000C2911D1B8

The end sector can be calculated with the following formula (Use the numbers from getptbl): ~ # partedUtil getptbl /dev/disks/mpx.vmhba36\:C0\:T0\:L0 gpt 1947 255 63 31293440 1947 * 255 * 63 - 1 = 31278554 You can also calculate the endsector with the following command: ~ # eval expr $(partedUtil getptbl /dev/disks/mpx.vmhba36\:C0\:T0\:L0 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1 31278554 Create the VMFS partition (Replace with your endsector) ~ # partedUtil setptbl /dev/disks/mpx.vmhba36\:C0\:T0\:L0 gpt "1 2048 31278554 AA31E02A400F11DB9590000C2911D1B8 0" Format the partition with VMFS5 ~ # vmkfstools -C vmfs5 -S USB-Stick /dev/disks/mpx.vmhba36\:C0\:T0\:L0:1

The USB-Stick should now appear in your datastores view.



And the final proof is a virtual machine running on it:



This is how your command output should look like:

~ # partedUtil mklabel /dev/disks/mpx.vmhba43\:C0\:T0\:L0 gpt ~ # eval expr $(partedUtil getptbl /dev/disks/mpx.vmhba43\:C0\:T0\:L0 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1 31278554 ~ # partedUtil setptbl /dev/disks/mpx.vmhba43\:C0\:T0\:L0 gpt "1 2048 31278554 AA31E02A400F11DB9590000C2911D1B8 0" gpt 0 0 0 0 1 2048 31278554 AA31E02A400F11DB9590000C2911D1B8 0 ~ # vmkfstools -C vmfs5 -S USB-Stick /dev/disks/mpx.vmhba43\:C0\:T0\:L0:1 create fs deviceName:'/dev/disks/mpx.vmhba43:C0:T0:L0:1', fsShortName:'vmfs5', fsName:'USB-Stick' deviceFullPath:/dev/disks/mpx.vmhba43:C0:T0:L0:1 deviceFile:mpx.vmhba43:C0:T0:L0:1 ATS on device /dev/disks/mpx.vmhba43:C0:T0:L0:1: not supported . Checking if remote hosts are using this device as a valid file system. This may take a few seconds... Creating vmfs5 file system on "mpx.vmhba43:C0:T0:L0:1" with blockSize 1048576 and volume label "USB-Stick". Successfully created new volume: 56226b60-118f2e3f-04ba-001b2193b3b0

Performance

You cant expect much performance out of usb flash drives. To see what's possible I've connected a Samsung mSATA SSD (840 EVO 250GB) to my Intel NUCs (NUC5i5MYHE) USB3 port and created a VMFS datastore on it.



Unfortunatelly it's detected as USB 2 device. There is a USB 3.0 hub, but it is not utilized. That seems to be a common problem, I don't know if it's actually possible at the moment.

[Update: October 19. 2015 - Figured out why devices are detected as USB 2. Devices now with full USB 3.0 performance. Performance test results updated.]

Max Write Throughput: 301.91 MB/s

Max Read Throughput: 308.78 MB/s

Max Write IOPS: 8159.31 CMDS/s

Max Write IOPS: 8228.49 CMDS/s

Throughput Test:





IOPS Test:



Mixed IO Test (I/O Analyzer Exchange 2007):



Is it possible with ESXi 5.x?

USB Devices in ESXi 5.x are a little bit picky. Some devices work, others not. I couldn't figure out why, but it is definitely related to ESXi 5.x. I've use the same procedure on the same hardware with the same flash drives. ESXi 6.0 takes all devices without any Issues, ESXi 5.5 takes some, others not.

In ESXi 5.5 the output for non working devices looks like this:

/dev/disks # partedUtil mklabel mpx.vmhba38\:C0\:T0\:L0 gpt /dev/disks # eval expr $(partedUtil getptbl mpx.vmhba38\:C0\:T0\:L0 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1 31278554 /dev/disks # partedUtil setptbl mpx.vmhba38\:C0\:T0\:L0 gpt "1 2048 31278554 AA31E02A400F11DB9590000C2911D1B8 0" gpt 0 0 0 0 1 2048 31278554 AA31E02A400F11DB9590000C2911D1B8 0 /dev/disks # vmkfstools -C vmfs5 -S terra mpx.vmhba38\:C0\:T0\:L0:1 create fs deviceName:'mpx.vmhba38:C0:T0:L0:1', fsShortName:'vmfs5', fsName:'terra' deviceFullPath:/dev/disks/mpx.vmhba38:C0:T0:L0:1 deviceFile:mpx.vmhba38:C0:T0:L0:1 Checking if remote hosts are using this device as a valid file system. This may take a few seconds... Creating vmfs5 file system on "mpx.vmhba38:C0:T0:L0:1" with blockSize 1048576 and volume label "terra". Usage: vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/vml... or, vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/naa... or, vmkfstools -C [vmfs3|vmfs5] /vmfs/devices/disks/mpx.vmhbaA:T:L:P Error: vmkfstools failed: vmkernel is not loaded or call not implemented.

To workaround that, I've tried to create the datastore with ESXi 6, and plugged it into an ESXi 5.5 host. No success. The datastore is recognized but appears as "inactive/unmounted".



vmkernel.log:

2015-10-16T19:44:08.410Z cpu3:33315)WARNING: LinScsiLLD: scsi_add_host:573: vmkAdapter (usb-storage) sgMaxEntries rounded to 255. Reported size was 65535 2015-10-16T19:44:08.410Z cpu3:33315)DMA: 612: DMA Engine 'vmhba38' created using mapper 'DMANull'. 2015-10-16T19:44:08.411Z cpu3:33315)<6>usb-storage 1-1.2:1.0: interface is claimed by usb-storage 2015-10-16T19:44:08.411Z cpu3:33315)<6>usb 1-1.2: device is not available for passthrough 2015-10-16T19:44:08.411Z cpu3:33315)<6>usb 1-1.2: usbfs: registered usb0107 2015-10-16T19:44:08.422Z cpu2:32809)ScsiNpiv: 1504: GetInfo for adapter vmhba38, [0x410897b96400], max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, sts=bad0020 2015-10-16T19:44:08.422Z cpu2:32809)ScsiNpiv: 1504: GetInfo for adapter vmhba32, [0x410897bda600], max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, sts=bad0020 2015-10-16T19:44:09.436Z cpu0:37002)usb-storage: detected SCSI revision number 0 on vmhba38 2015-10-16T19:44:09.436Z cpu0:37002)usb-storage: patching inquiry data to change SCSI revision number from 0 to 2 on vmhba38 2015-10-16T19:44:09.436Z cpu2:37003)ScsiScan: 976: Path 'vmhba38:C0:T0:L0': Vendor: ' ' Model: 'Patriot Memory ' Rev: 'PMAP' 2015-10-16T19:44:09.436Z cpu2:37003)ScsiScan: 979: Path 'vmhba38:C0:T0:L0': Type: 0x0, ANSI rev: 2, TPGS: 0 (none) 2015-10-16T19:44:09.436Z cpu2:37003)ScsiUid: 273: Path 'vmhba38:C0:T0:L0' does not support VPD Device Id page. 2015-10-16T19:44:09.436Z cpu2:37003)ScsiScan: 1105: Path 'vmhba38:C0:T0:L0' : No standard UID: Failure. ANSI version 'SCSI-2' (0x2). 2015-10-16T19:44:09.437Z cpu1:37003)VMWARE SCSI Id: Could not get disk id for vmhba38:C0:T0:L0 2015-10-16T19:44:09.437Z cpu1:37003)ScsiScan: 1503: Add path: vmhba38:C0:T0:L0 2015-10-16T19:44:09.437Z cpu1:37003)ScsiPath: 4695: Plugin 'NMP' claimed path 'vmhba38:C0:T0:L0' 2015-10-16T19:44:09.981Z cpu0:37002)usb-storage: detected SCSI revision number 0 on vmhba38 2015-10-16T19:44:09.981Z cpu0:37002)usb-storage: patching inquiry data to change SCSI revision number from 0 to 2 on vmhba38 2015-10-16T19:44:09.981Z cpu3:37003)ScsiUid: 273: Path 'vmhba38:C0:T0:L0' does not support VPD Device Id page. 2015-10-16T19:44:09.981Z cpu3:37003)VMWARE SCSI Id: Could not get disk id for vmhba38:C0:T0:L0 2015-10-16T19:44:09.981Z cpu3:37003)vmw_psp_fixed: psp_fixedSelectPathToActivateInt:479: Changing active path from NONE to vmhba38:C0:T0:L0 for device "Unregistered Device". 2015-10-16T19:44:09.981Z cpu3:37003)StorageApdHandler: 698: APD Handle Created with lock[StorageApd0x410897] 2015-10-16T19:44:09.981Z cpu3:37003)ScsiEvents: 501: Event Subsystem: Device Events, Created! 2015-10-16T19:44:09.982Z cpu3:37003)VMWARE SCSI Id: Could not get disk id for vmhba38:C0:T0:L0 2015-10-16T19:44:09.982Z cpu3:37003)ScsiDeviceIO: 7410: Get VPD 80 Inquiry for device "mpx.vmhba38:C0:T0:L0" from Plugin "NMP" failed. Not supported 2015-10-16T19:44:09.982Z cpu3:37003)ScsiDeviceIO: 7420: Get VPD 83 Inquiry for device "mpx.vmhba38:C0:T0:L0" from Plugin "NMP" failed. Not supported 2015-10-16T19:44:09.982Z cpu3:37003)ScsiDeviceIO: 7456: Get VPD 86 Inquiry for device "mpx.vmhba38:C0:T0:L0" from Plugin "NMP" failed. Not supported 2015-10-16T19:44:09.983Z cpu3:37003)ScsiDeviceIO: 6170: Could not detect setting of QErr for device mpx.vmhba38:C0:T0:L0. Error Failure. 2015-10-16T19:44:09.983Z cpu2:32787)NMP: nmp_ThrottleLogForDevice:2322: Cmd 0x1a (0x412e80846d40, 0) to dev "mpx.vmhba38:C0:T0:L0" on path "vmhba38:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0. Act:NONE 2015-10-16T19:44:09.983Z cpu2:32787)ScsiDeviceIO: 2338: Cmd(0x412e80846d40) 0x1a, CmdSN 0x89a from world 0 to dev "mpx.vmhba38:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0. 2015-10-16T19:44:09.984Z cpu3:37003)ScsiDeviceIO: 6684: Could not detect setting of sitpua for device mpx.vmhba38:C0:T0:L0. Error Failure. 2015-10-16T19:44:10.004Z cpu3:37003)ScsiDevice: 3443: Successfully registered device "mpx.vmhba38:C0:T0:L0" from plugin "NMP" of type 0