Patent litigation has become a huge problem for the software industry. And on Thursday, the Supreme Court could have solved that problem with the stroke of a pen. Precedents dating back to the 1970s place strict limits on software patents. The court could have clearly reiterated that those old precedents still apply, and that they rule out most patents on software.

Instead, perhaps fearing the backlash from invalidating billions of dollars worth of patents, the court took an incremental approach. It ruled that the specific patent at issue in the case was invalid. But it didn't articulate any clear rules for software patents more generally. In effect, the court kicked the can down the road, leaving a huge question mark floating over most software patents.

The problem, at root, is that the courts are confused about the nature of software. The courts have repeatedly said that mathematical algorithms can't be patented. But many judges also seem to believe that some software is worthy of patent protection. The problem is that "software" and "mathematical algorithm" are two terms for the same thing. Until the courts understand that, the laws regarding software patents are going to be incoherent.

The rise of software patents

Since the 19th Century, the courts have held that abstract ideas couldn't be patented. In a 1972 ruling, the court applied that rule to an AT&T patent application. AT&T wanted a patent on a method for converting between two binary number formats. But the Supreme Court ruled that this kind of mathematical algorithm was exactly the kind of abstract idea that was excluded from patent protection. The courts extended this reasoning in 1978, invalidating a proposed patent on a method to use software to trigger an alarm when an industrial variable such as temperature or pressure reaches an unsafe level.

the Supreme Court could eliminate software patents with the stroke of a pen

In 1981, the court allowed a patent on a software-controlled rubber-curing process. Some have interpreted this decision as legalizing software patents, but a careful read of the opinion leads to a different conclusion. In its opinion, the court reiterated that mathematical algorithms — which is what software is — were not eligible for patent protection standing alone. The rubber-curing process was patentable, the court held, because it was focused on a physical manufacturing process that happened to have a software element, rather than trying to claim software standing alone.

In 1982, Congress created a new appeals court called the Federal Circuit Appeals Court and made it responsible for all appeals in patent cases. The court has proven to be the most patent-friendly court in the nation. And in the 1990s, it set about dismantling the limits on software patents that the Supreme Court had established in previous decades.

A turning point came in 1998, when the Federal Circuit approved a patent that claimed the concept of using software to manage mutual funds. When I spoke to the legal scholar Pam Samuelson earlier this year, she told me that it was "not possible" to square this ruling with the Supreme Court's own precedents. Rather, Samuelson said, the Federal Circuit didn't like the Supreme Court's position on software patents, so "in effect, they overruled it. They thought the Supreme Court was going to continue to pay no attention, and so they had a free hand."

The Patent Office interpreted this ruling as a green light for software patents, resulting in a bonanza for patent lawyers but a disaster for the American economy. Since 1998, the number of software patents has soared, and so has the costs of patent litigation.

Data from James Bessen, Peter Neuhäusler, John Turner, and Jonathan Williams

A new type of firm, dubbed "patent trolls" by their detractors, began accumulating armloads of broad software patents and suing people who infringed them by accident. Big technology firms began hiring armies of patent lawyers to beef up their own patent portfolios. Today, trolls and large technology companies have so many software patents that they form an impenetrable patent thicket. It's almost impossible to write useful software without accidentally infringing a bunch of them.

The evidence suggests that in the software industry, the patent system does more to hinder innovation than to reward it. Inventors spend more money defending themselves against patent lawsuits than they earn from patent royalties. More and more entrepreneurs are losing sleep about the risk that patent litigation will drive them into bankruptcies. And companies that traditionally haven't had to worry about the patent system, like restaurants, grocery stores, and casinos, are facing demands from trolls over dubious patents, most of which involve software.

The nuclear option

If it had wanted to, the Supreme Court could eliminate software patents with the stroke of a pen. The Federal Circuit can ignore the Supreme Court's precedents, but it can't overrule them. The Supreme Court has already ruled that mathematical algorithms can't be patented. If the court made clear that software is nothing more than a collection of mathematical algorithms — which it is — that could have been the end of software patents.

One reason the courts might hesitate to do this is that it would be a big blow to the bottom lines of some of the biggest companies in America. Such a ruling would have invalidated thousands of dubious software patents held by trolls, but it also would have invalidated Amazon.com's infamous 1-click patent, the "data detectors" patent Apple used to sue Samsung, and Google's patent on its search ranking algorithm. Invalidating software patents would have wiped billions of dollars off the balance sheets of some of America's largest technology companies. The Supreme Court generally tries to avoid making waves, and those would have been some very big waves.

justices hope to salvage the good software patents while getting rid of the bad ones

But some members of the Supreme Court also seem to genuinely believe that some software patents should be allowed. In a 2010 opinion that was signed by four justices, Justice Anthony Kennedy worried about the risk that he might "create uncertainty as to the patentability of software, advanced diagnostic medicine techniques, and inventions based on linear programming, data compression, and the manipulation of digital signals."

This passage isn't legally binding because it was only signed by just four of the court's nine justices. But it suggests that some justices hope to salvage the good software patents while getting rid of the bad ones.

The problem is that the courts haven't been able to figure out how to tell the difference between good software patents and bad ones.

Software and algorithms are the same thing

In Thursday's ruling, the court rejected a patent that claimed the concept of using a computer to hedge against "counterparty risk" — the risk of making a bargain and then having the other guy not pay up. The Supreme Court complained that in the process described in the patent, "each step does no more than require a generic computer to perform generic computer functions." Such a generic patent, the court said, isn't eligible for patent protection.

The problem is that this criticism can be leveled at literally any software patent. At root, software is nothing more than a sequence of mathematical operations. "Algorithms are exactly as basic to software as words are to writers," the computer scientist Donald Knuth once wrote.

If a sequence of conventional mathematical operations isn't patentable, then no software should enjoy patent protection. For example, the "data compression" patents that Justice Kennedy wants to preserve simply claim formulas for converting information from one digital format to another. If that's not a mathematical algorithm, nothing is.

This is the fundamental confusion at the heart of America's software patent jurisprudence: many judges seem to believe that mathematical algorithms shouldn't be patented but that certain kinds of software should be patentable.

If a sequence of mathematical operations isn't patentable, no software should enjoy protection

The court has suffered from this confusion since that very first software patent case in 1972. In that case, the court wrote. "It is said that the decision precludes a patent for any program servicing a computer. We do not so hold." But then the court invalidated the patent because "the mathematical formula involved here has no substantial practical application except in connection with a digital computer" and would "wholly pre-empt the mathematical formula and in practical effect would be a patent on the algorithm itself."

It's hard to see how these two statements can be reconciled. A "program servicing a computer" and a "mathematical formula" used "in connection with a digital computer" are two ways of describing the same thing. If one isn't patentable, then neither is the other.

You can only patent hard math?

In practice, the courts have only allowed patents that claim complicated mathematical algorithms. For example, in a 2011 decision the Federal Circuit approved a patent because the mathematical algorithm it claimed "required the manipulation of computer data structures." Of course, a "computer data structure" is just the way a computer organizes numbers and symbols.

When I taught computer science in graduate school, I used to draw data structures on the blackboard and then walk students through the steps a computer would take to manipulate them. In principle, you can do this with any computer program, though of course there's no blackboard large enough to draw the data structures in Microsoft Word. Programs that use data structures are no less mathematical, and no less algorithms, than a program for converting between binary number formats.

If a patent claims a mathematical formula simple enough for a judge to understand how it works, she is likely to recognize that the patent claims a mathematical formula and invalidate it. But if the formula is too complex for her to understand, then she concludes that it's something more than a mathematical algorithm and uphold it.

But this makes the law highly unpredictable, since it effectively depends on the mathematical sophistication of the judge who happens to take the case. And it's also logically incoherent. The courts originally excluded algorithms from patent protection because they are basic building blocks for innovation — that's as true of complex algorithms like data compression as of simple ones.

The patent system works best for capital-intensive industries like pharmaceuticals or automobiles. In these industries, innovations are produced by large teams of engineers and scientists. Consulting a patent lawyer to see if a new drug or engine design is patented is relatively easy.

In contrast, software development is an individual, creative process more akin to writing a novel than designing a new car. A single programmer working over a weekend can write software that employs numerous sophisticated mathematical algorithms. It's unreasonable to expect that programmer to consult a patent lawyer before releasing his software to the public, just as it would be crazy to ask a novelist to consult a patent lawyer to make sure none of the ideas in his novel had been patented.

The mathematical ideas in software, like the literary ideas in novels, are part of society's common intellectual heritage. Neither should be eligible for patent protection.