In 1995 I was working for an AT&T spinoff, Unix System Labs, in the kernel group, but USL wasn't doing too well. It looked like Windows was winning the OS wars (remember those?) and was going to scrape Unix from the face of the earth. My buddy Tom had been trying to get me to come out to the west coast to work for Sun.

He was working in a group that did network security. He said that it didn't matter what OS won, we'd always have network security to worry about. That made sense to me. He set up the interviews, and since I didn't know anything about security, told me to buy a copy of Applied Cryptography and read it before I got there.

So I got hired into Sun's group developing firewalls and IP-level encryption. This was great work. Security is really intellectually challenging and rewarding. Little unseen errors in apparently simple code or protocols can lead to collapse of the entire system. Rigor and thought count.

But commercially our group was a bust. In addition to our firewall product, the Sun sales force was reselling a third-party product too. And they seemed to like to sell the third party product better than Sun's homegrown one.

We were also trying to sell an add-on security workstation package. The sales force didn't seem to think much of the commission on our $99 software bundle when they were pushing their multi-million dollar hardware orders.

Furthermore, the US government didn't seem to want to let us sell our stuff. International sales of cryptographic products are regulated, since they're classified as "munitions". We had Swiss banks that wanted to buy our encrypting firewalls. We'd have to meet with the NSA to get export approval, but were never able to sell the full strength versions. There was a dark story about a rebuffed offer to include a backdoor in the crypto for the powers-that-be, which lead them to subsequently look on us unfavorably. We tried all sorts of shenanigans to loophole around these restrictions but it was an uphill battle.

The final nail in the coffin came when Sun's crypto protocol lost out to another faction's in the IETF.

I was just an engineer in this group, but the reality of what was happening in the market to our product line started to seep in. Here I was putting all of this effort into stuff that never would be used by anyone. It was still intellectually challenging...like doing crossword puzzles or something. But it had no utility to the world.

I started to look around and I saw many other examples of groups working on stuff that no one would ever use or care about. Mobile IP initiatives, endless work around standards that nobody cared about, research from the labs that would never be applied or even cited.

Yikes.

I had written stuff that people actually used, before. It felt good. I had written a usenet newsreader that was used by hundreds of thousands of people. I was running an online game, as a commercial hobby on the side, which had several hundred paying customers. Sheesh, I thought. My side projects have more customers than my day job.

So I made a simple resolution. I wanted to work on stuff that people would actually use.

This sounds simple. But if you walk the halls of Sun, AOL, HP, IBM, AOL, Cisco, Siebel, Oracle, any university, many startups, and even Google and Yahoo, you'll find people working on stuff that isn't going to ship. Or that if it does ship, it won't be noticed, or won't move the needle.

That's tragic. It's like writing a blog that nobody reads. :-) People make fun of bloggers who are writing "only for their mother". But what about the legion of programmers writing code paths that will never be traversed. Wasted effort!

Some of that may be inevitable. You try experimental things. Sometimes they don't work. Everyone can't be maximally productive 100% of the time, so there may be lesser-value tasks that still keep the engine warm and have some marginal utility. But still. Evolutionarily, frustration is useful. It kicks us out of non-productive ruts. People should get frustrated more easily. Frustration should be driven by an awareness of futile effort.

* * *

Without business models, bizdev deals for distribution, and market economics that afford a place for your product, it doesn't matter how pretty the code is. Ugly code and awful products win all the time.

From an engineering perspective, it's simply zooming out the field-of-view to include the entire market, including the users, competitors, and so on. They're part of the total engineering solution. If you've written an app with some web forms and a database, but you haven't solved the problem of how to get users to come to the web form, then you've left part of the problem unsolved.

Greg Linden details some of the tricks that have been used by startups to get a leg up in a crowded world. He wonders if you have to be, perhaps, a little bit evil to have a hope. I'm not sure, but you should have some idea of how you're going to launch the bird, and the market and distribution economics that let it stay aloft.

All of this is a long-winded way of explaining why I include all this gunk about network effects and switching costs and distribution and brand perception on my blog. Because the world, full of competitors and networked humans with their set of behavior patterns, is part of the spec. If you're designing a product, but don't understand how the system of networked humans will work around it, you really can't understand how your product will work either.

No minute lost comes ever back again

Take heed and see ye nothing do in vain

