Talk about starting a business based on open source software and the conversation will inevitably shift to Red Hat. That's because the Linux vendor is a shining example of a company that's making money from an open source product. But how easy is it really to establish an open source startup that makes money? For every success story like Red Hat there are companies like Cyanogen that fail to thrive and projects that are abandoned.

It's tempting to believe that the Red Hat business model, which is based around selling subscriptions for support to a maintained and tested version of Linux (or a closely related model that offers consultancy and customization to an open source software solution as well support and maintenance), is the most viable way to make money from open source software. But Sam Myers, a principal at Balderton Capital, a technology venture capital company, says that most open source startups are unlikely to succeed using these business models.

"Despite Red Hat, it is actually quite challenging to make money selling customization, support and consultancy," Myers says. "Why? Because it is head-count driven, the model doesn't scale, and you get low renewals. And you have competition from other consultancies."

Myers admits that the subscription model can occasionally be successful, but asserts that a more promising business model is to build a product line around an open source core. This can involve developing premium software modules that add features to the core open source software or, alternatively, building supporting applications that complement the core.

SuiteCRM, for example, offers its open source CRM software for free but charges for modules like an Outlook plugin. "What can upset people is when you develop new code that makes the core better but keep it proprietary, but if you build apps that work on top of it then there is no issue there," says Myers.

Adding an Asterisk to open source

Another open source startup business model involves offering hardware that is suited to the software (in the way that Digium sells telephony hardware to run the open source Asterisk telephony software.) But Myers warns that this model can be difficult to sustain because customers typically only buy the hardware occasionally. Instead, he recommends looking for a business model that produces a recurring revenue stream. These can include offering open source software as a service or charging for API use in addition to selling premium modules or supporting applications as mentioned above.

What quickly became apparent from speaking with Myers is that there is no "best" open source business model, and Allison Randal, president of the Open Source Initiative, says that open source startups should avoid searching for one. "The mistake people make is thinking about an open source business model. They should be thinking about a business model and how open source software fits into that," she says. "VCs are only beginning to understand open source and how to make money, but the way is the same as for any other business: by offering better value and making customers happy. "

A defining feature of many open source projects is the community that surrounds them, and there is always a danger that a company seeking to make money from open source software may alienate that community. That's because members of the community may feel that their volunteer efforts are being exploited for someone else's financial gain. So how much consideration should open source startups give to the project's community?

In some instances, like when a company provides almost all of the code commits to a project, Myers says that not much consideration needs to be given to community. "The main benefit of open source software isn't necessarily that development is crowdsourced. In some cases, something needs to be open source so that companies that use it don't have to be worried about vendor lock in," he says.

The business benefits of community

But in many cases there are great benefits to be had from adopting a business model that involves fostering an active community. "If you are looking for commits from outside then it is important to focus on the community, both to get developers working on code and also to see who your users are, because these are your leads for upselling."

Myers warns that communities don't just spring up and thrive by themselves though, so it's vital for an open source startup to spark interest in the community through marketing and communication. Avoiding alienating the community in the way that Cyanogen Inc. did when it decided to monetize the community's work communication is especially important, he says.

What kind of marketing and communications are necessary? "You need to make sure that you say that X percent of your resources will be devoted to developing the open source project and Y percent will go to developing the proprietary modules or other applications that you plan to sell," says Myers. "If you can manage to do that successfully then you can manage any perceived conflict of interest."

Alex Freedland, CEO of Mirantis, a company that has built a business around the open source OpenStack cloud operating system, says that to foster widespread adoption of an open source product you need an ecosystem around it, and to get that you also need a strong community. For that reason, he says, Mirantis makes a point of ensuring its contributions to a project never exceed 25 percent of the total in a given time frame.

Freedland also says that the community should trump the business when it comes to choosing the direction software development should take. "You need to decide what is of benefit to the community and do it, even if it goes against the short-term interests of the company," says Freedland. "It is also important that community members feel they won't be punished for their actions — you need to foster a culture where community members can do whatever they feel will benefit the community."

He also advocated that open source companies devote a proportion of their resources to seeding other groups to expand the ecosystem.

Avoiding the pitfalls

Myers says there are two mistakes that open source startups that want to make money should make a point of avoiding. "If you just take the community's code (and build proprietary modules around it) then you are bound to alienate the community, and I can't see that ending well," he says. "And another mistake is trying to charge too soon for premium versions before a project has a strong community around it. You need to build a large audience for an open source software project before you can start to monetize it."

The Open Source Initiative’s Randal says that while most communities don't mind a company trying to monetize a project, it is key that the community still has a life of its own — in the way that Red Hat has fostered the Fedora community. "What drives a community away is when you take the wind out of its sails and it feels taken over," she says. Randal adds that little things can make a big difference: if Cyanogen Inc. had chosen a different name (in place of Cyanogen OS) for its commercial product, which was based on the Cyanogen Mod project, then the community may not have felt so offended by it, she says.

Mirantis’ Freedland adds that open source projects should be run as meritocracies and remain open to new ideas because startups that try to micromanage the direction of a project are inevitably on the path to failure. "There are always religious zealots, but as the leader of a startup you need to limit your own influence. If you do that you won't alienate the community, but if you don't then it will come back to bite you in the end."

Related video: