IBM is threatening to pursue legal action against TurboHercules, a company that sells services relating to the open source Hercules project, an emulator that allows conventional computers with mainstream operating systems to run software that is designed for IBM System Z mainframe hardware.

In a letter that IBM mainframe CTO Mark Anzani recently sent to TurboHercules, Big Blue says that it has "substantial concerns" that the Hercules project infringes on its patents. The letter is a brusque half-page, but was sent with nine additional pages that list a "non-exhaustive" selection of patents that IBM believes are infringed by the open source emulator.

This move earned the scorn of well-known free software advocate and patent reform activist, Florian Mueller. In a blog entry that was posted Tuesday, Mueller fiercely criticized IBM, accusing the company of abusing its patent portfolio and harming open source software in order to retain monopolistic control over its expensive mainframe offerings.

"After years of pretending to be a friend of Free and Open Source Software (FOSS), IBM now shows its true colors. IBM breaks the number one taboo of the FOSS community and shamelessly uses its patents against a well-respected FOSS project," wrote Mueller. "This proves that IBM's love for free and open source software ends where its business interests begin."

He contends that IBM's support for open source software is insincere. As evidence of the company's hypocrisy, Mueller points out that two of the patents that IBM listed in its letter to Hercules are included in the list of 500 patents that IBM promised to not assert against open source software in 2005. Mueller is convinced that the patent promise was a manipulative attempt to placate government regulators.

How emulation intersects with IBM's mainframe business

IBM's position in the mainframe market has posed contentious antitrust issues for years. The company's software licensing model ties its mainframe operating system to its underlying System Z hardware, guaranteeing that companies who have built their own applications for the platform can't easily migrate to other hardware options. This lock-in strategy has proved lucrative for IBM, generating billions of dollars in revenue.

Despite the extremely high cost and the fact that some companies don't necessarily derive value from the hardware's unique characteristics, they continue buying IBM's mainframe solutions because doing so remains cheaper than rewriting all of their legacy applications. We explained this phenomenon several years ago when we looked at the reasons why IBM's mainframe business is still profitable despite the declining relevance of the technology.

A well-designed System Z emulator that allows users to migrate their own mainframe applications to commodity hardware would obviously pose a serious threat to IBM's mainframe business, but IBM's software licensing terms have historically prevented such a threat from materializing. Users would have to run IBM's mainframe operating system inside of the Hercules emulator in order to run the applications—but they aren't allowed to do that.

It's certainly possible to run modern versions of IBM's mainframe operating system with Hercules, but it can't really be officially supported or publicly condoned by the project's developers due to the licensing issues. Much like Hackintoshing, it is fairly trivial on a technical level but constitutes an unambiguous violation of the end-user license agreement. As such, Hercules never really posed a threat to IBM in the past. The legal issues simply preclude commercial adoption and deployment in production environments. Hercules is principally used by enthusiasts to run z/Architecture Linux variants, an activity that doesn't erode IBM's lock-in strategy.

In many ways, the project arguably benefits IBM by encouraging interest in the mainframe platform. That is largely why IBM has shown no hostility towards Hercules in the past. In fact, IBM's own researchers and System Z specialists have lavished Hercules with praise over the years after using it themselves in various contexts. The project was even featured at one time in an IBM Redbook. What brought about IBM's change in perspective was an unexpected effort by the TurboHercules company to commercialize the project in some unusual ways.

TurboHercules came up with a bizarre method to circumvent the licensing restrictions and monetize the emulator. IBM allows customers to transfer the operating system license to another machine in the event that their mainframe suffers an outage. Depending on how you choose to interpret that part of the license, it could make it legally permissible to use IBM's mainframe operating system with Hercules in some cases.

Exploiting that loophole in the license, TurboHercules promotes the Hercules emulator as a "disaster recovery" solution that allows mainframe users to continue running their mainframe software on regular PC hardware when their mainframe is inoperable or experiencing technical problems. This has apparently opened up a market for commercial Hercules support with a modest number of potential customers, such as government entities that are required to have redundant failover systems for emergencies, but can't afford to buy a whole additional mainframe.

IBM's response

As you can probably imagine, IBM was not at all happy with that development. Following IBM's initial threats of legal action, Hercules retaliated by filing an antitrust motion in the European Union, calling for regulators to unbundle IBM's mainframe operating system from its mainframe hardware. IBM responded harshly last month, claiming that the antitrust motion is unfounded and that a software emulation business is just like selling cheap knock-offs of brand-name clothing. The conflict escalated, leading to the patent letter that was published on Tuesday.

When faced with patent litigation, companies often try to keep the conflict quiet and hope for an out-of-court settlement because they don't want the threat of a lawsuit to scare away potential customers. That's certainly a factor here, because IBM's threats raise serious questions about whether it is truly permissible to use the Hercules emulator in a commercial setting. TurboHercules is taking a bit of a risk by disclosing the letter to Mueller for broad publication. The startup likely chose to publicize their predicament with the hope that the open source software community will notice and respond by shaming IBM into backing down.

As Mueller points out in his blog entry, the broader open source software community has some reasons to side with TurboHercules in this dispute: some of the patents cited by IBM cover fundamental functionality of virtualization and emulation. Those patents reach far beyond the scope of Hercules and could pose a threat to other open source software projects.