If the theme of the decades after the invention of the Gutenberg printing press was “publish”, the theme of today might well be “search”.

Last week I suggested the idea of standard, commoditized news reporting as a utility, rather like electricity or water. It is essential to modern civilized life, yet rarely appreciated in itself. Standard straight up and down news reporting of parliament, the courts, media conferences and other public events might not be all that glamorous, but we sure as hell would notice the difference if it wasn’t there.

But the utility metaphor only gets us so far. Another way of thinking about what media organizations might be good in a world where anyone can publish is to think about other kinds of public assets. Such as roads, maps and the internet itself. These things enable us to do stuff that would otherwise be difficult or impossible.

If the theme of the decades after the invention of the Gutenberg printing press was “publish”, the theme of today might well be “search”.

Once the role of reporters might have been to find things out and write. That is still important. But we might also reconceive our role as being to make sure that when citizens search, that process is as easy as possible, and there is something worth finding.

If I were founding a media organization today, I might well adopt the motto “Seek, and you shall find.”

Many years ago when I was young in journalism, I learned to do company searches and land title searches. Back then, this involved a lengthy trip to the relevant office, and a tedious hunt through either paper based records or microfiche. Doing even a single search could take most of a morning or afternoon. Often you could not be sure that you had the complete picture. The upside was that searching was very cheap and in some cases free.

I kept doing this kind of work as the computer made searching easier, and aspects of the privacy act closed down some useful records – such as births, deaths and marriages. The costs went up. But one thing remained constant. Whenever I visited public records offices I would find members of the public haplessly attempting to do their own investigations. I would end up helping them to find their way around the many different forms, and the impenetrable way that information, when they managed to find it, was presented.

Thanks for signing up We look forward to seeing you bright and early with your need-to-know talking points and tidbits for the day ahead. Get Crikey FREE to your inbox every weekday morning with the Crikey Worm. Please enter your email address Sign up

It was clear that there were many citizens attempting to do this kind of thing. Their motives were various. Some were nutters bent on proving conspiracy theories. Some were ordinary citizens who had been ripped off, and were trying to find out how and why. Some were job seekers trying to find out about an employer before an interview. There were also residents opposing property developments. All sorts.

Some, whether they knew it or not, were attempting to do what these days we would call citizen journalism. They knew or suspected something. They were trying to check it out. Then they would look for a media organization interested in the story.

These days, of course, they would be just as likely to publish the material themselves, on line.

Today I do this kind of public record searching through an information broking company via the internet, without leaving my desk. So the citizens bumbling around public records are invisible to me. But I am sure they are still there.

The way in which the records were presented – and this hasn’t changed — made searching by the uninitiated very difficult. It was a skill that reporters used to learn by osmosis — part of the job.

Today it is different. One of the many powerful pieces of evidence that Australian journalists have lost the plot is that in the average newsroom these days, very few will actually know how to do a company search or a land title search for themselves.

Instead, they hire the lawyers or a search company to do the work for them, adding to the expense and making it less likely that serendipity will reveal more than they knew to look for.

The irony is that this is happening at the very moment when we are right bang slap in the middle of a revolution in the way that governments in the western world hold and make available what, in the buzz word of the times, is known as public sector information.

In Australia all the important moves in this revolution are being made, not by journalists, but by governments and by engaged and web savvy citizens, often on a volunteer and not for profit basis.

Take a look at Open Australia — a resource put together by volunteers.

It allows you to search what your parliamentarian has been doing, and sign up for an email alert when they speak in parliament.

You can also, for the first time, access federal parliamentarians’ registers of pecuniary interest online. Now these have been public documents since Methuselah was a child, but before the heroes at Open Australia got hold of them, most citizens wouldn’t have been able to access them unless they knew enough to go along to parliament and ask. Now, anyone can see that Tony Abbot has declared income as a freelance journalist, for example.

Now why on earth wasn’t a media organization the first to digitize registers of pecuniary interest? Could it be that they were instinctively preserving their privileged access to information?

Don’t we care about the free flow of information? Isn’t that meant to be our core business?

Why doesn’t The Age or the Herald Sun do the same thing for the Victorian State Parliament? Or The Advertiser for South Australia? Why are we leaving this kind of thing to others, thereby all but insuring that, over time, we will become increasingly irrelevant to the act of searching for information?

A natural role for media organizations would be to build a database that brings together public sector information in a form that is easy to navigate and which allows researchers, including journalists and members of the public, to detect and interrogate relationships between data sets, including relationships between, say, property developments approved and political donations. Or marginality of electorates and sports grants awarded.

Here I must make a declaration. This is exactly what the Foundation for Public Interest Journalism, which I chair, is hoping to do with its Public Interest Journalism Resource Centre, which seeks to develop an existing prototype, the LobbyLens tool, into an engine to power investigations. We are calling it the Public Interest Lens, or PIL. But at the moment we have no funding for this, although we are on the way to doing something about that. It is a vision and an aspiration.

In an ideal world, it shouldn’t be left to us. A forward thinking media organization would already have done it.

How many journalists do you know who can tell you about what is happening under the Government 2.0 agenda? My bet is, none or not many. Yet this reform movement sweeping governments in the western world holds immense implications for journalists and their audiences.

The core idea, part of the Obama administration in the USA but also seizing Europe, is that social media and Web 2.0 tools offer hold potential for a more open and engaged relationship between governments and the governed.

The Federal and state governments have commissioned a number of inquiries and reviews examining aspects of information policy. A common theme has been “public sector information” as a public resource, and the need for proactive disclosure on open licensing terms. The public has already paid for the information to be acquired, the thinking goes. Therefore the public should be able to access and use it.

In July 2010 the then Minister for Finance and Deregulation, Lindsay Tanner, released a Declaration of Open Government. This had been the central recommendation of a key federal government inquiry into public sector information, the Government 2.0 Taskforce.

The declaration asserted that government in the future would embrace a culture of openness and transparency. It set out a clear objective of promoting public participation in democracy and recognising the benefit of citizen collaboration to ‘enhance government processes and outcomes’.

Tanner has gone, of course, and is nothing but bitter about mainstream media. Yet the open government agenda is now being carried forward by the newly established Office of the Australian Information Commissioner, which recently issued a paper outlining a program of releasing government information in a form that was easy for citizens to use and reuse.

Public sector information, the Commissioner said, could when made readily available, stimulate innovation and economic prosperity, be used in business and lifestyle planning, and be used to evaluate the performance of government and hold it democratically accountable.

Once that would have been pretty much a job description for media outlets and for journalists.

Yet the mainstream media has scarcely reported on these developments, let alone got on board.

Journalists and citizens are entitled to be skeptical or even cynical about governments who claim they want to release information. Witness the history of the Freedom of Information Act, and the ways in which it has been frustrated.

Nevertheless, there are signs of significant progress – not the least of which is FoI reform.

And while government makes key moves, journalists, (to my immense distress) have ACTUALLY OPPOSED increased openness. They have become the new conservatives.

One of the aspects of FoI reform, for example, is that once information is released to citizens, government agencies have to make it generally available on their websites as part of a disclosure log.

The response of the mainstream media? Instead of celebrating the openness, journalists have whinged about how this means everyone will get their scoop.

For heaven’s sake, guys, get with it.

The Government 2.0 Taskforce report anticipated that once public sector information is liberated as a key national asset, possibilities — foreseeable and otherwise — will be unlocked through the invention, creativity and hard work of citizens, business and community organisations. Open public sector information, the report said, is thus an invitation to the public to engage, innovate and create new public value.

Traditionally, journalists have been information brokers. I would suggest that part of our new role should be to facilitate exactly the kind of creativity and hard work that the Taskforce was talking about.

There have been experiments in the kind of thing I am talking about in Australia – sadly none of them by journalists.

In 2009, as part of the Federal Government’s Government 2.0 Taskforce exercise, a “mash-up” contest was held as a practical demonstration of the potential benefits of open access to Australian public sector information.

The resulting “mash ups” resulted in potentially useful data and tools, now sadly quickly going out of date. The full range can be seen at Mash Up Australia. It includes tools that give the geographic reach and impact of arts funding, NSW crime data displayed by local government area, and maps allowing citizens to inform local government of problems in their area.

One of the leading entries in the “mash-up” – judged Best in the Show at the Gov 2.0 “hackday” — was LobbyLens. The LobbyLens developers correlated data about Federal Government business over the previous 18 months, and showed the connections between government contracts, politicians’ responsibilities, lobbyists, and clients of lobbyists.

Datasets used by LobbyLens included Contract Notices from AusTender , Census Geographic Areas Digital Boundaries (Postcodes) from the Australian Bureau of Statistics, Australian Federal Electoral Boundaries from Australian Electoral Commission, and lobbyists registers from federal and state governments, as well as information on political donations from the Australian Electoral Commission.

You would think that LobbyLens would have been picked up and developed by a mainstream media organisation by now.

Nope. They are too intent on keeping on doing what they have always done, while waiting for the cool new toy or magic potion that will mend the broken business model.

Yet overseas, some media organizations are experimenting with exactly the kind of new role that I am suggesting here.

Take a look at DataStore a program of the Guardian newspaper in the United Kingdom. A website is used to provide access to government data “vaults” worldwide, combined with visualisations and applications to make the data understandable and usable. The project seeks to be “the ultimate gateway” to international government data, including that in Australia. Users can search government data sites in one place and download the data.

Users are encouraged to rank datasets to assist the developers to find the best ones, and to participate in collecting data. Top visualisations and apps developed on the site include statistics from the UK Department of Transport, used to show how trucks use motorways. There is a wide range of government economic data presented in easy to understand form.

There is also a tool that allows users to be alerted when planning applications are lodged with local authorities. This latter application, by the way, is something that the citizens behind Open Australia seek to add to their data base. It is a natural project for a community newspaper organization to pick up. But where are News Limited community newspapers, or Fairfax Community Newspapers? Nowhere to be seen.

The Office of the Information Commissioner has identified a number of government information publishing projects already underway. They are listed on the data.australia.gov.au website.

Here you can see that the MySchool website (one of the few releases of public sector information that has made the headlines) is part of a much broader trend.

You will find Australian Bureau of Statistics information, which only a few years ago you had to pay for, made freely available. There is the Australian Early Development Index that provides a national snapshot of childhood development; the Australian Spatial Data Directory that provides access to spatial data used by industry, government and the community; the Environmental Resources and Information Network that publishes datasets relating to the Australian environment; the Australian Social Science Data Archive, which is a university based service that collects and preserves data relating to social, political and economic affairs.

Then there is Mapping our Anzacs, which is collection of service records published by the National Archives; the Australian Newspapers Digitisation Program that has digitised out-of-copyright Australian newspapers from the 1800s to the mid-1950s; the National Public Toilet Map that shows the location of more than 14,000 toilet facilities around Australia; and more.

Moves are also underway in the states. The recent Operation Halifax inquiry by the NSW Independent Commission Against Corruption (Declaration: I gave evidence at this inquiry, alongside one of the developers of LobbyLens, Doris Spielthenner) found that a lack of transparency in the lobbying industry was a major corruption risk, and recommended a new regulatory regime that would see details of lobbying contacts with ministers and others made publically available, including the time and nature of the contact, and the clients of the lobbyist.

There is a new set of government data released almost every day. The challenge is to make it mashable – that is, uniform in type so that it can be combined with other datasets, so that one can search it and the patterns and relationships made immediately apparent.

The new releases of data add to the existing public records that I have been using throughout my career – company searches from the Australian Securities and Investment Commission, land titles searches from state land titles offices, the electoral roll, information about electoral donations from federal and state electoral commissions, court records, and that wonderful but journalistically under utilized resource, the Australian Legal Information Institute.

All that information is available, and has always been available, to those who know how to look.

Imagine if a media organization not only harvested and made usable all this, and added intelligence from its archives. The archives, after all, are surely one of the truly unique, and truly valuable assets of legacy media. They are the residue of the operation of journalism over decades and centuries Time to exploit them, I would have thought.

The result would be a close to indispensable utility for any citizen who wanted to find out anything.

And could you charge for it? I would be willing to bet that you could. The chances that people will pay for commoditized news content available anywhere are thin indeed.

But pay to use an engine that powers their own investigations and inquiries? That empowers them? That harvests and makes usable public sector information and adds the weight of research encapsulated in media archives? That is brokered and curated by journalists?

That is a different proposition entirely.

This is episode three of my series on innovation in journalism, which is running each week here on a Monday. You can read the first two episodes, which include the rationale, here and last week’s here.



Next week: Political Reporting. What’s Broken, What Might Work?