USB stick RAID in ACTION!

Warning. Attempting similar projects at home may cause feelings of grandiosity.

Introduction

RAIDs (Redundant Arrays of Inexpensive [sic] Disks) are considered pretty handy for a number of things. This is an example of productive and practical use of a RAID. Granted, this project does not have the archaic grandeur of a Floppy Disk RAID, but then again, the capacity and performance of this system are utterly superior to those of a Floppy Disk RAID. The following is meant as an instruction sheet of how to build a rock-hard USB stick RAID system and simultaneously transform from an ordinary nerd to a SUPER LINUX GURU.

Ingredients

There is no unequivocal agreement between chefs de computing cuisine about the ingredients required for a USB stick RAID, but with this kind of a recipe one can easily make good use of a half an hour of time and about fifty hard-earned euros.

A computer with a USB port (no monitor or other sissy stuff needed)

Gentoo Linux with some nerdtweaks

A USB HUB

Three (3) USB sticks: the sticks used in this utility are 128M sticks of different brands

Single lifestyle

Set-up

Well, the hardware set-up is quick but you can still get your hands dirty: plug-in the HUB and the sticks and take a sip of Coke. However, because you're running GNU/Linux and not Windows, prepare to spend two days in figuring out how to make ALL the different brands of USB sticks work. Oh yeah, don't forget to compile the RAID modules to the kernel while doing your daily tweak.

Figure 1: Stick'em in

When you can make all the USB sticks flash their tiny led-eyes, you know you're ready to go.

First, tell the ordinary USB sticks to transform into mighty Linux Raid Autodetect sticks (notice the author's deep thought in naming his computer):

root@dragon mmeri # fdisk /dev/ubb Command (m for help): p Disk /dev/ubb: 131 MB, 131072000 bytes 32 heads, 32 sectors/track, 250 cylinders Units = cylinders of 1024 * 512 = 524288 bytes Device Boot Start End Blocks Id System /dev/ubb1 1 250 127984 fd Linux raid autodetect Command (m for help): q root@dragon mmeri #

Then, command your sticks to regroup into a RAID device. IMHO there's no difference between the coolness of different RAID levels: no matter which one you choose for your server, it tells equally good a story about the extent of your social life. So in the following we create ( -C ) verbosely ( -v ) a device ( /dev/md0 ) with RAID level 5 ( -l=5 ) out of three ( -n=3 ) devices uba1 , ubb1 , and ubc1 . Tell yourself, THIS ROCKS!

root@dragon mmeri # mknod /dev/md0 b 9 0 root@dragon mmeri # mdadm -C -v /dev/md0 -l=5 -n=3 /dev/uba1 /dev/ubb1 /dev/ubc1 mdadm: layout defaults to left-symmetric mdadm: chunk size defaults to 64K mdadm: size set to 125824K mdadm: largest drive (/dev/ubb1) exceed size (125824K) by more than 1% Continue creating array? Yes mdadm: array /dev/md0 started. root@dragon mmeri #

Now you've actually turned your three slack USB sticks into a SUPER COOL RAID device! A trick that leaves Copperfield dumbstruck and wins you the chicks. Trust me, I'm almost a doctor. Then, grab another 1.5L Coke (I know you've got one if you've read it this far), and format and mount your new beauty:

root@dragon mmeri # mke2fs /dev/md0 mke2fs 1.35 (28-Feb-2004) Filesystem label= OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) 62992 inodes, 251648 blocks 12582 blocks (5.00%) reserved for the super user First data block=1 31 block groups 8192 blocks per group, 8192 fragments per group 2032 inodes per group Superblock backups stored on blocks: 8193, 24577, 40961, 57345, 73729, 204801, 221185 Writing inode tables: done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 30 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. root@dragon mmeri # mount /dev/md0 /mnt/usbraid/ root@dragon mmeri #

Drink the coke and relax reading hex dumps while the USB sticks on the HUB blink like a computer geek's star constellation (see the stunning movie1 or movie2). This'll take a while. When the formatting is finally done, observe the tremendous capacity that your new fault-tolerant device has (even more than this if you don't use RAID5) -- 231M should be enough for anybody! Also notice how the performance of your system KICKS ASS!

root@dragon mmeri # df Filesystem 1K-blocks Used Available Use% Mounted on ... /dev/md0 243695 13 231100 1% /mnt/usbraid ... root@dragon mmeri # hdparm -tT /dev/md0 /dev/md0: Timing cached reads: 744 MB in 2.00 seconds = 371.50 MB/sec Timing buffered disk reads: 4 MB in 4.08 seconds = 1004.07 kB/sec root@dragon mmeri #

Sit back, take a sip of coke, and congratulate yourself. You are COOL DUDE.

Performance

This sorta record-cracking system surely cries to be tested for performance. The tools used in this test are special UNIX tools developed for system testing: date, cp, and umount. (Wizards time less, date more. And what's a hdparm?) First, we'll copy a file of size 221M on the RAID apparatus (write performance), while measuring the time, and then copy it back (read performance):

root@dragon mmeri # ls -l total 2540661 ... -rwxr-xr-x 1 mmeri users 221832772 Dec 10 2004 VCDLuku1.mpg ... root@dragon mmeri #date && cp VCDLuku1.mpg /mnt/usbraid/ && umount /mnt/usbraid && date Sat Aug 13 14:35:20 EEST 2005 Sat Aug 13 14:50:43 EEST 2005 root@dragon mmeri # date && cp /mnt/usbraid/VCDLuku1.mpg /tmp/ && umount /mnt/usbraid && date Sat Aug 13 15:27:57 EEST 2005 Sat Aug 13 15:31:51 EEST 2005 root@dragon mmeri #

Conclusion

As RAID technology is meant only for serious computer-wizards who know stuff, the USB stick RAID will, unfortunately, remain a curiosity in the field of computing. Pity, especially if one considers the almost endless productive uses that this face-meltingly-fast technology could be harnessed for. Pack in the almost endless capacity, and you have a cool fault-tolerant data storage device.

Next in series: 5.25" Floppy disk RAID: With the inexpensive technology, is this a respectable competitor for USB stick RAID?

--By Daddy Kewl 13-Aug-2005

Appendix: Serious-looking stuff