Posted on by

The 2011 SNIA SDC: Storage Developers Circus

The annual Storage Developers Conference is kind of like a five-ring circus. There are way too many tracks to follow, all going on at once and all interesting to varying degrees to varying people. It was a big show again this year, but this year it didn’t matter what else was going on. The center of attention was Microsoft’s center-ring act: SMB2.2.



I have been going to the SNIA SDC every year since before it was the SDC. I started going in 1997, when it was still “The CIFS Conference”. Even before that, I went to lots of different computer conferences and shows. Remember DECUS and DEXPO? Amiga DevCon? LISA-NT? I do.

In all of those years, and all of those many conferences, this is the first time I have ever said “Wow, that’s cool” at a Microsoft presentation. So let me say it again: SMB2.2—Wow, that’s cool.

It’s cool for a couple of big reasons and several smaller reasons. The big reasons are these:

This wasn’t just an overview and demonstration. They were giving us advanced access to the technology. There are already draft specifications on Microsoft’s website. What isn’t yet documented was covered in enough detail to give us a head-start, and they are working with the SMB/CIFS community on anything that is baked enough to be shared.

It was actually cool technology.

On that second point, the more correct way to put it is to say that SMB2.2 is a cool implementation of cool technology. There is very little that is truly new in SMB2.2. It’s all cool stuff that has been fluttering around for a while, and has finally been caught and pinned down and labeled and organized into a coherent system on a platform with enough market share to make a difference. Both the client and server-sides are in sync, so we won’t have to wait for the other half to be put together before the first half is useful.

The Butterfly Collection

Butterflies generally don’t buzz, but we’ll list a few buzzwords anyway.

Clusters

A while back, Samba introduced the CTDB clustered database system, which allowed Samba servers to run on top of clustered file systems in active/active mode. Microsoft may have taken this as a bit of a challenge. SMB2.2 supports both file server clusters and application server (client) clusters. These can be combined in [m × n] ways to scale with the workload.

Multipath

If wire speed is a bottleneck, add more wires. SMB2.2 dynamically adapts itself to use any or all connection paths. This was demoed, using pre-alpha code, during Microsoft’s keynote presentation. …and it worked. …and it scaled almost linearly.

RDMA

SMB2.2 can be run over RDMA (Remote Direct Memory Access). This is particularly useful when building clusters, but it has other implications. Most of all, RDMA is fast. It can run on top of a variety of high-speed interconnects including InfiniBand and iWarp.

Distributed Caching

The previous SMB2 version, known as SMB2.1, included support for BranchCache, a distributed caching system that leverages content-addressable data (data blocks indexed by hash values). SMB2.2 adds support for a new version of BranchCache that allows for different content hashing algorithms. In BranchCache V2 the hash type does not need to be specified, which provides opportunities for vendors to create and utilize their own algorithms.

These features are clearly game-changers, and they represent a serious threat to NFSv4 and v4.1, which have yet to gain much in the way of traction. NFSv4 also supports transfers over RDMA and other cool features, but people are not using them.

Geek Gods

From the very start of this year’s conference, it was clear that the market for CIFS talent was hot. There were a lot of companies hiring. Microsoft’s drive, over the past few years, to deliver and improve upon the SMB2 suite has caused several storage vendors and startups to invest in SMB/CIFS and SMB2 engineering. The new SMB2.2 version will only amplify this effect.

The problem, though, is that there aren’t enough CIFS geeks to go around, and those that are available are typically very high-level engineers; CIFS Geek gods.

In the NFS market, a company in Silicon Valley can put a sign in the window that says “NFS Help Wanted”, and a line will form. Employers are used to being able to hire developers with a recent CS degree who know how to mess around with NFS, because NFS is taught in Colleges and Universities. That’s what happens with open standards.

Such is not the case in the CIFS world.

CIFS isn’t taught anywhere, there is only one book on CIFS protocol internals available, and until a couple of years ago there weren’t any protocol specifications at all. As a result, the current pool of CIFS geeks generally had to come up through the ranks in one of two ways: Either they were already high-level engineers who had a compelling need to enter the murky swamp, or they were newbies who got thrown in at the deep end and managed to swim—and avoid getting eaten by the AndX monsters. Chomp.

Now that Microsoft has published their official specifications (see [MS-CIFS], [MS-SMB], and [MS-SMB2] as starting points), entry into the CIFS world should be a little bit easier. The availability of specifications also means that more companies are trying to implement these protocols in their products, so more developers with SMB/CIFS/SMB2 know-how are needed.

That should translate into good news for the few CIFS geeks who don’t currently have a day job. The only problem is that HR departments have not yet come to grips with the fact that the available talent is all senior level, and isn’t all located in California. Recruiters have confirmed that this is the situation, and that it’s a problem. There are people available, but their skills are way beyond what managers are expecting and costs more than they are ready to pay.

Whither Samba?

It is very difficult to innovate in the SMB/CIFS space. Microsoft’s market share grants de facto standard status to whatever they do, but nothing to whatever they don’t. Windows systems, for example, do not support the so-called “CIFS Unix Extensions” that were designed and implemented by a loose consortium of third-party vendors that included the Samba Team. The Unix Extensions survive in obscurity because there are a handful of products that support them. Many other cool ideas have withered completely due to lack of support in Windows. You have to be very clever and very compelling to innovate in this space.

Samba has a long history of being that kind of clever, but many Samba innovations are hidden from the user and some, such as improvements made to the now-outdated Browser Service, are old enough that they are no longer of interest. Newer innovations, such as Samba/CTDB clusters and Samba’s Active Directory implementation, are not as important to every day users as basic file services. Users sometimes forget that Samba’s even there.

Which is why Samba is more relevant than ever.

Samba has become part of the infrastructure, like roads or telephone poles. It needs to be maintained and updated and improved or it really will get noticed, in the same way that a bridge gets noticed when a car falls through it. So the Samba Team have been doing a lot of infrastructure work over the past several years, improving the internal structure of Samba and setting the groundwork for the next major set of new features.

Samba 3.6.0, released last August, is a big step forward. It contains SMB2.0 support. This support is relatively new, even though bits of the code have been around for quite a while, so this feature is disabled by default. Enable it, test it, and send in bug reports. Community involvement is the only way to ensure that it gets the extra polish that it needs.

Implementation of SMB2.1—particularly BranchCache support—has already started, but the pace is likely to be accelerated now. Following Microsoft’s presentations at the SDC, SMB2.2 has been added to the Samba roadmap.

A large contingent of the Samba Team attended the SDC this year, and spent a lot of time testing the latest code in the SMB/CIFS/SMB2 Plugfest. The new stuff is on its way.