Scholars have long dreamed of a universal library containing everything that has ever been written. Then, in 2004, Google announced that it would begin digitally scanning all the books held by five major research libraries. Suddenly, the library of utopia seemed within reach.

Indeed, a digital universal library would be even better than any earlier thinker could have imagined, because every work would be available to everyone, everywhere, at all times. And the library could include not only books and articles, but also paintings, music, films, and every other form of creative expression that can be captured in digital form.

But Google's plan had a catch. Most of the works held by those research libraries are still in copyright. Google said that it would scan the entire book, irrespective of its copyright status, but that users searching for something in copyrighted books would be shown only a snippet. This, it argued, was "fair use" – and thus permitted under copyright laws in the same way that one may quote a sentence or two from a book for the purpose of a review or discussion.

Publishers and authors disagreed, and some sued Google for breach of copyright, eventually agreeing to settle their claim in exchange for a share of Google's revenue. Last month, in a Manhattan court, Judge Denny Chin rejected that proposed settlement, in part because it would have given Google a de facto monopoly over the digital versions of so-called "orphan" books – books that are still in copyright but no longer in print, and whose copyright ownership is difficult to determine.

Chin held that the United States Congress, not a court, was the appropriate body to decide who should be entrusted with guardianship over orphan books, and on what terms. He was surely right, at least in so far as we are considering matters within US jurisdiction. These are large and important issues that affect not only authors, publishers and Google, but anyone with an interest in the diffusion and availability of knowledge and culture. So, while Chin's decision is a temporary setback on the way to a universal library, it provides an opportunity to reconsider how the dream can best be realised.

The central issue is this: how can we make books and articles – not just snippets, but entire works – available to everyone, while preserving the rights of the works' creators? To answer that, of course, we need to decide what those rights are. Just as inventors are given patents so that they can profit from their inventions for a limited time, authors were originally given copyright for a relatively short period – in the US, it was initially only 14 years from the first publication of the work.

For most authors, that would be enough time to earn the bulk of the income that they would ever receive from their writings; after that, the works would be in the public domain. But corporations build fortunes on copyright, and repeatedly pushed Congress to extend it, to the point that in the US it now lasts for 70 years after the creator's death. (The 1998 legislation responsible for the last extension was nicknamed the "Mickey Mouse Protection Act" because it allowed the Walt Disney Company to retain copyright of its famous cartoon character.)

It is because copyright lasts so long that as many as three-quarters of all library books are "orphaned". This vast collection of knowledge, culture, and literary achievement is inaccessible to most people. Digitising it would make it available to anyone with internet access. As Peter Brantley, director of technology for the California Digital Library, has put it: "We have a moral imperative to reach out to our library shelves, grab the material that is orphaned, and set it on top of scanners."

Robert Darnton, director of the Harvard University Library, has proposed an alternative to Google's plans: a digital public library, funded by a coalition of foundations, working in tandem with a coalition of research libraries. Darnton's plan falls short of a universal library, because works in print and in copyright would be excluded; but he believes that Congress might grant a non-commercial public library the right to digitise orphan books.

That would be a huge step in the right direction, but we should not give up the dream of a universal digital public library. After all, books still in print are likely to be the ones that contain the most up-to-date information, and the ones that people most want to read.

Many European countries, as well as Australia, Canada, Israel, and New Zealand, have adopted legislation that creates a "public lending right", where the government recognises that enabling hundreds of people to read a single copy of a book provides a public good, but that doing so is likely to reduce sales of the book. The universal public library could be allowed to digitise even works that are in print and in copyright in exchange for fees paid to the publisher and author based on the number of times the digital version is read.

If we can put a man on the moon and sequence the human genome, we should be able to devise something close to a universal digital public library. At that point, we will face another moral imperative, one that will be even more difficult to fulfil: expanding internet access beyond the less than 30% of the world's population that currently has it.

Copyright: Project Syndicate 1995–2011