This article was inspired by a post from Steve Foskett on Dell's The Future of Storage site. In his post, Foskett tries to make the case for the "Home SAN." While I'm not convinced that the answer to all my home storage problems is a "SAN," like Foskett proposes, I do agree that something has to be done. Check out my proposed solution, and sound off in The Server Room.

I have a lot of networked storage in my house. Various iPhones and iPods, a Mac Mini, three laptops, an AppleTV, a NAS device, multiple Firewire drives attached to the laptops, and the list goes on. I haven't done the math to total it all up, but by the time I upgrade my NAS this summer I'll have at least 5TB total of storage scattered throughout my home, tucked away in pockets, purses, briefcases, entertainment centers, closets, and other nooks and crannies. Living amidst this much hardware might sound great, but parts of the experience get old. In particular, I'm really, really tired of managing all these "disks." I'm tired of manually syncing and copying and things between one disk or partition and another, and I wish every gadget in my house was part of one big virtual filesystem—a "storage cloud," if you will—that would liberate me from the curse of constantly syncing.

Why I hate syncing

I spend way too much time and effort trying to keep different file collections in sync among different devices, so that I can make sure that I have the right version of the right file in the right place when I need it. In short, file syncing is the bane of my existence, and managing multiple versions of individual files and databases (both structured and unstructured) is a constant battle.

For instance, I recently spent an entire day developing an elaborate method for keeping the iTunes libraries on two different Macs totally in sync, so that adding a file to one would propagate the changes to the other without my having to manually add any files via iTunes' interface. (For the curious, the solution involved putting my entire iTunes directory on one of Leopard's new sparse bundle images, placing that image on each machine, and then using rsync to sync that image between my MacBook Pro and my Mini, so that an update to one library would appear in the other library simply by relaunching iTunes.)

My solution worked, but then I decided to scrap it all when it occurred to me that I may want to sync the AppleTV to the Mini and my iPhone to the MacBook, which would mean a separate library for each file... but then what happens if I buy a new song from the iTunes Music Store on my AppleTV? How does it get to the iPhone? Or what if I buy a new song on the iPhone? In short, adding music to an iTunes-based digital music collection is a mess, if you're trying to keep multiple iTunes libraries on multiple machines in sync.

Then there are the problems that I have syncing my MacBook Air with my MacBook Pro. Some applications that I use, like Together, now have .Mac sync services built in. But others, like Journler, do not. And that still doesn't change the fact that if I move to the living room to work on a set of files on the Air, I have to manually sync those files back to the MacBook that's wired to my desktop setup before switching computers again. (I use Chronosync for this.)

Then there's the NAS, which I formerly used for backup before I started using Time Machine. Also, I'm currently looking into a replacement for Strongspace for offsite backup... The syncing never ends.

I'm not sure what the answer is, but I know that some kind of more automated or "invisible" set of sync services isn't it. Because with all the syncing that I've done, I've found the following axiom to be true: the only thing worse than a sync that requires manual intervention is an automated sync that has unwanted results that you don't learn about until it's too late.

Bring on the cloud

So I'm ready for some cloud storage. But I don't want all my gadgets to connect to some distant cloud. Rather, I want them to be the cloud, so that my data surrounds me like some mist with a life of its own, instead of sitting in these little isolated balls that I have to juggle.

It's not that I don't ever want to sync again. No, syncing should be like rebooting—something you do once in a blue moon, when you absolutely have to. In fact, the most I should ever have to sync a laptop is when I walk back into my house after having been away working on something.

When I come home from a trip, I want my network to say, "hey, it's the MacBook Air. Let's see what's new on its hard disk, and what changes has Jon saved to different files." And then I want my network to start copying the new and altered files off my Air, but not onto a specific machine or server on my network—I want it copied into the cloud. And when I'm sitting in my chair and I change a file on the Air, I want that change propagated back out to the cloud instantaneously, the way that a change to an e-mail draft in Gmail is saved back to Google's servers, so that when I switch machines the changes are there automatically.

And I don't want to sync my iPhone to my laptop. Rather, I want my iPhone to move files to and from my iTunes library—but the one that lives in the cloud that encompasses my entire house, and not some version of a file collection plus metadata that lives on a particular machine.

And when I add songs to my AppleTV, those songs are in the cloud and any (authorized) machine in my house can instantly see them and play them by either caching local copies or streaming... I don't care which, and I don't want to know.

Above all, I don't want to really manage disk space, and file versions, and timestamps, and sync scheduling, and all that other stuff that my computers should be managing for me with minimal intervention. I just want to be able to live and work and play, all without the hassle of manually shuffling files and preferences and other types of information around from one machine to the next. Is that too much to ask?

Syncing vs. "saving"

I can hear at least one objection already: "Isn't this 'cloud' that you're describing just a fancy way of talking about a lot of automated, invisible sync, which you've said you don't like?"

The answer is, "no... and yes."

A "disk" is already a storage abstraction that hides all sorts of sync activity from you, the user. So as you use a computer, the different levels of the storage hierarchy (registers, cache, main memory, hard disk) are all "syncing" different copies of the data on which you're working, but your exposure to the nuts and bolts of that sync is largely limited to the "Save" command in whatever application you're using.

When you "save a file," what you're really doing is syncing different storage pools, all of which are hidden from you by the abstraction that is the "disk drive."

So what I'm proposing is that we hide all of the separate "disk drives" in an average home behind an even larger abstraction: "the cloud." And then we let software do what it's good at, i.e., managing the read and write operations among the various storage pools so that instead of "syncing files" you merely "save changes," and those changes that you've saved appear everywhere on your network.

Steve, Bill, Linus... any takers? The world is waiting for someone to build this mousetrap. Or, at least, I am.