SAN FRANCISCO — A federal jury Monday delivered a messy split verdict in the trial over Google’s (GOOG) popular Android mobile software, by finding that Google infringed copyrights held by tech rival Oracle (ORCL) but failing to answer a key question that Oracle needed to press its case for nearly $1 billion in damages.

The result was a blow to Oracle’s quest for a share of profits in the world’s leading smartphone operating system. And it put some of the most important issues of the case back in the hands of a judge to decide.

“You could say it’s a setback for Oracle,” said Edward Naughton, a Boston attorney who specializes in technology cases and has been following the trial closely. But he added, “The situation is still quite up in the air.”

The closely watched case has pitted two of Silicon Valley’s most powerful companies against each other in a rare courtroom showdown, in which the stakes are high for each side: Google, the Internet search giant, has based much of its plans for future growth on a strategy that relies on people using Android to access Google services on their smartphones and other mobile devices. Oracle, which is known for commercial software, wants to assert its rights and increase its revenue from Java, the widely used programming system that Oracle acquired when it bought Sun Microsystems two years ago.

After nearly a week of deliberation, jurors agreed that Google infringed Oracle’s copyrights by copying a small amount of Java code in Android and by essentially mimicking the structure and organization of certain elements known as Application Programming Interfaces, or APIs.

Jurors also rejected some other infringement claims that were considered less important to the case. But on a key question, the jury was unable to reach a unanimous decision on Google’s argument that using the APIs was permitted under the legal concept of “fair use,” which allows using excerpts from a copyrighted work under limited circumstances that are beneficial to the public interest.

The API issue represents the bulk of damages that Oracle sought in the case. But since the “fair use” question is unresolved, U.S. District Judge William Alsup indicated in court that Oracle at this point may seek only a limited amount of damages — not a share of Google’s Android profits — for nine lines of code that the jury agreed were copied from Java into two Android files.

Court order sought

Under the law, experts said, it appears Oracle could seek no more than $150,000. Oracle has also said it would ask for a court order that could force Google to alter Android or pay a licensing fee; it was unclear how that would be affected by Monday’s verdict.

Alsup, however, indicated Monday that he will consider motions from both sides that could drastically alter the case’s outcome. While a Google attorney immediately moved for a mistrial on the API question — arguing there can be no infringement if the jury did not resolve the “fair use” question — Oracle attorneys have asked the judge to make his own determination that the “fair use” defense does not apply in the case.

And on another key point, Alsup has indicated he will make the final decision on whether APIs are protected by copyright, although he instructed the jury to assume they were, for their deliberations. That issue could have broad effect in the software industry, where APIs are widely used in a host of programs.

Second phase

The judge has not indicated when he will rule on those points. He ordered both sides to proceed Monday with the second phase of the case, which is focused on Oracle’s allegations that Android also violated Java patents. The issue of damages won’t be considered until a third phase of the trial.

If Alsup rules the APIs are not covered by copyright, that would hand a clear victory to Google, said Santa Clara University law professor Tyler Ochoa. But if Alsup decides that copyright applies, Ochoa said, Google may face an uphill battle in proving its “fair use” defense.

A typical example of “fair use” is quoting short excerpts in a review or scholarly article, Ochoa said, adding that the law discourages uses that undercut the value of the copyrighted work. Oracle contends Google’s development of Android has essentially created a competitive alternative to Java and potentially cost Oracle some licensing fees.

Stanford law professor Mark Lemley, however, said it seems unlikely that the judge would rule against the “fair use” defense, because he would have to find that no reasonable jury would accept fair use. And in this case, the jury’s impasse means at least some of the jury apparently did accept that defense.

In a statement after the verdict, a Google spokesman said, “Fair use and infringement are two sides of the same coin. The core issue is whether the APIs here are copyrightable, and that’s for the court to decide. We expect to prevail on this issue and Oracle’s other claims.”

An Oracle spokeswoman, meanwhile, issued a statement that said: “The overwhelming evidence demonstrated that Google knew it needed a license” for using Java and that “every major commercial enterprise — except Google — has a license for Java.”

Contact Brandon Bailey at 408-920-5022. Follow him at Twitter.com/brandonbailey.