The Most Revolutionary Microsoft Technology You’ve Never Heard Of 12 Apr 2007

Nope, it’s not from Microsoft Research. It doesn’t hail from Cambridge or Mountainview, nor is it some underbelly technology in Microsoft Vista that only Mark Russinovich and the responsible SDE is aware of. Rather, it’s a new service-oriented application model built on two overlapping technologies: Decentralized Software Services (DSS) and the Concurrency and Coordination Runtime (CCR). It is currently shipping as part of the Microsoft Robotics Studio (more on that later) and is poised to disrupt the way we think about the Windows Communication Framework (WCF) and the way we design, architect, and implement distributed applications. It is a work of genius.

CCR/DSS (which clearly needs a more catchy moniker, such as Erlang# or Microsoft Nano) is the brainchild of Henrik F Nielsen and George Chrysanthakopoulos, a Lennon-McCartney, Morrissey-Marr pairing of gearheads that isn’t often seen in software since Carmack met Romero and formed id Software. Henrik was co-author of the HTTP protocol and Microsoft’s lead envoy to the W3C SOAP committee, and George was one of the wizards that got all the bits and bare metal talking to each other in the original Xbox project. I spoke with them on a recent visit to Redmond and heard first-hand about the things they’re working on.

The story about how these two met and came to work with each other is unfortunately unsatisfying and arbitrary. Plucked by whim from Microsoft’s technologista, Henrik, a fair bespectacled rail, and George, a raucous Dale Carnegie of the software world, were thrust together in an office to have a think about how complex distributed software ought to be written. As luck would have it, they hit it off, and they have been vibrating with technological creativity ever since.

An early prototype demonstrated a bank of computers, each outfitted with mouse, keyboard, audio, and graphics. In their demo, they mixed and matched the peripherals in real-time through a web-browser, routing the keyboard of one to the other, the graphics from one to the other, and so forth for all of the peripherals. It was a perfect, seamless, high-performance Frankenstein, and it challenged many ingrained assumptions at Microsoft about how operating systems should be written. They have since generalized their work and developed something that should scare the bejeezus out of the .NET Framework guys, if only they were paying enough attention to it.

Imagine an application model composed of ubiquitous state-driven services. Each service has a little bulletin board of richly-typed key/value pairs and can trigger events whenever a value is changed. Services can be instantiated anywhere where there is a lightweight service host (such as a PC, microprocessor, browser, or smartphone) and can register themselves as visible to other services, either by kind or identity. Services easily detect, address, and send messages to each other, requesting changes to each other’s state values and registering for another’s change notifications. Running services can migrate from machine to machine. Service state is completely versioned and serializable and can be backed up, restored, and rolled forward and back in time. Rich exception information flows between deep trees of services along lines of causality, regardless of whether the services are on the same machine or across the world. Each service and its state can be trivially viewed and modified from a web browser with no extra work, but can also be locked down using several different layers of security. Now imagine that that application model supported a level of high-performance concurrency usually relegated to specialized language systems like Erlang.

If you were to imagine all that as a fully-managed .NET system, you’ve got a decent picture of what CCR/DSS is all about. Working with the framework offers many “duh/aha/of course” moments; it quickly becomes clear that the way they did it is exactly the way it should be done. The idea of a constellation of communicating, state-driven services isn’t necessarily revolutionary, but their conception and execution of that idea is. The framework was conceived not just as a library, but as a full-blown application model. It’s an elegant work, esoteric to a point, but transformative and crystalline in purpose.

It is also yet another reminder that the top one percent of engineers are of an entirely different ilk than the rest of us. It’s not that the quality or quantity of their work is untouchable (though it comes close); rather, it comes down to the completeness of their work and the ease with which they do it, like they were jumping over a puddle. Capable of glimpsing a sublime solution to a problem and sketching it up without syntax errors in the back pages of a Moleskine notebook, the top one percent of engineers are the technological equivalent of a three-year old solving a Rubik’s cube or a twelve-year old scoring symphonies for the New Haven Symphony Orchestra. They can bind creativity to purpose and realize it with great fidelity. Henrik and George are in that top one percent.

So why then, as previously mentioned, is this incredible Leninist technology squirreled away in the oddball Microsoft Robotics Studio? It’s not that CCR/DSS has any deep relationship to robots or robotics.

There are at least two answers to that question.

First, it is true that robotics marry well with a state-driven application model. Servos have speed and position values, on/off flags, and service records. Sensors have sensor data. Robots can contain an arbitrary number of services that need to route information and events to and from each other. In a sense, a robot is kind of distributed system on a chassis. Though robotics aren’t the fat part of the market for a technology like CCR/DSS, it certainly is an interesting playground for it.

Second, Microsoft’s internal version of Pin the Tail on the Donkey is called Pin the Product on the Ship Vehicle. For various reasons, Microsoft can’t simply ship a couple assemblies and a lone executable all by itself. It didn’t fit temporally or politically in the Microsoft .NET Framework 3.0 effort, and it wasn’t large enough to stand on its own. So, for better or worse, it became the backbone of Microsoft Robotics Studio (along with the very interesting Visual Programming Language or VPL).

I’ll write some more about CCR/DSS in the near future, but in the meantime, check out the wiki, watch a video or two and install Microsoft Robotics Studio. Right now it’s a bit like a 1966 VW Bug fitted with a turbocharged Porsche engine. And, yes, the documentation is meager. But that’s okay. We know how to pop the hood.