A year ago I posted on open source business practices and conversion rate myths. A colleague asked yesterday whether we shouldn't still use it as a measure until a better one is found, since it's the best we have.

The problem isn't that it's a poor measure, but I believe it's a wrong and misleading measure. It is NOT similar to lines-of-code in software engineering (a poor measure). Variation in LoC is still within a few percentage points when you resolve for the project differences. Conversion rate is orders of magnitude different between companies. Business measures are typically within percentage points. Think of ratios like gross margin. They vary between types of businesses by as much as 30% but within an industry they're pretty steady and act as a benchmark.

Using a wrong or misleading measure will put you in jeopardy if you're the marketing or community manager. (You're being judged against the metric.) If you distribute for "free" as open source software, you capture your user base depending upon how good you are at getting the word passed onward. If you're good, you build community around your software. Remember software only forms part of the value network that is your solution — and customers buy solutions to problems. Mårten Mickos well observed that your early community is willing to trade time to save money and your later community is willing to trade money to save time — and the later community contains his customers. The observation implies a timeline for community growth depending upon the software project. Companies starting a business around a new project need to be sensitive to this. Customers (people that are going to buy something) will use free access to test that your software might solve their problem before they pay for your solution.

Many users in community do not have money to spend today. They can't be converted. As a "lead" they will never become a "qualified" lead, and getting in their way trying to sell to them will only be a source of frustration for everyone involved. BUT while in a normal proprietary software company this means you've disqualified someone that isn't a customer in the pipeline, in the open source corporate world, I believe they sit in a different process. Instead of a manufacturing metaphor (pipelines) we need a more organic social metaphor to track success and growth of the community. Lead generation is a measure of community engagement. You want to grow your community. Customers will self-qualify.

This WILL be reflected in a number of downloads, but this is correlative not causal, and it's an ambiguous metric. It's very software function dependent. If a user downloads one copy and installs it on a hundred machines, your knowledge is already off by two orders of magnitude — counting downloads is useless.

The greater user community still needs care and feeding because they contribute code, bugs, time to answer one another's questions, beta bodies, and interesting applications of your software you may not have thought about or had time to develop yet. They are a source of innovation, not simply within the software (small contributions) but at the solution level. They will tell your BUSINESS new places to go to capture money in the network around your software. These are very valuable people. Growing your community is important for this reason. Don't simply measure downloads and compare to a number of customers. Measure everything you can think of about your community:

Unique members on the forums.

Time to answer a question (internally versus community). [And don't reward your employees for answering quickly or your community will never get a chance to answer.]

Number of code contributions.

Number of bug reports.

Number of documentation contributions (howto, tutorials, etc.)

Feature requests.

etc.

For numbers that tie back to specific releases of software, use them to tune your engineering process, i.e. involve your users in your engineering process. Consider letting USERS vote on importance of bugs so they feel a part of the process. (Don't draw the line between customers and users. It will be tempting, but customers probably buy something else from you in their solution. Disenfranchising users is disenfranchising the larger community of interest and probably a false feature.) Engage their time (appropriately) and you will win community mind share. Mind share is everything for profitability. It's the stickiness. It will create customer advocates.

There will come a time when the users move to other organizations bringing their knowledge with them and the new organization will self-qualify to become a customer. Or the user's own company grows to a point that they become customers. But they don't "convert" in the traditional sense. The lead did not become a qualified lead and you don't need to "sell" to them to convince them to buy your solution. They have internal needs that makes buying your solution more important than using your software. I'm betting those internal needs are very function-centric (e.g. one company will buy JBoss support while using RHAS for free). Determining those functional decisions in relation to your solutions will give you better insight into how to grow your customer base, i.e. revenues, without sacrificing your community in "conversion" games.



This comic is licensed under a Creative Commons Attribution-NonCommercial 2.5 License from xkcd.org.