Much of that data belongs to companies, but some of it belongs to governments. And some of those governments are ours. Most people have probably never heard of open data even though it has for more than two years been federal government policy to encourage its dissemination in the public sector. Officially, since 2009, information gathered and held by federal government agencies should, by default, be freely available to everyone and should be distributed through electronic media. Further, that information should be published under a copyright that allows people to play with the information without restraint. These were the core recommendations of the Government 2.0 Taskforce, initiated by the Rudd government with a mandate to consult widely about the best way to engage with modern web practices, including social media. The taskforce, led by economist Nicholas Gruen, hosted Hack Days in which previously secret data was analysed, reimagined, warped and enlivened by teams of computer specialists - people who might also be called ''hackers''.

The result was a series of web applications. More than one team focused on building a better public toilet map. The winning team created an animated, interactive, searchable directory of government lobbyists, called LobbyLens. No wars were fought over this unprecedented release of government data. There were no late-night raids or computer seizures. By all accounts, the geekiest kind of fun was had by all concerned. The headline on the Government 2.0 Taskforce website read: ''GovHack: govt data + hackers + caffeine == good times.'' It was hardly The Girl with the Dragon Tattoo. In that spirit, the taskforce recommended a revolutionary approach to the way the Australian government manages its data - throw open the floodgates. Why should the government provide such access? Why would any of this matter? The first answer is transparency: better access to government data means everyone with the skills can assess the area of government activity that interests them. But some of the best arguments for open data invoke the unknowable benefits that might flow from innovation that access to data can stoke. Data, as boring as it might sound, is what drives some of the coolest things in our lives. Facebook and Twitter (to name just two) are nothing more than tricked-up databases.

In the hands of the right developer, even apparently mundane information can sing. An interactive, smart-phone-ready public toilet map might sound flippant, but what if you are disabled? How useful is it to know that the nearest was built to be accessible? Now imagine the kind of applications that might be built from full access to databases holding particular kinds of health information, real-time crime data or data about schools; the possibilities are almost endless. The provision of data is acknowledged around the world as a new and developing way to encourage innovation and create new businesses. For governments, it can mean allowing businesses to build the apps that provide train timetables, public toilet maps and other such services. This can save government money and encourage the development of software industries. Countries are responding by opening their data stores to the public. Data.gov in the US and Data.gov.uk in Britain are only two examples. But to be able to deliver such services, developers need to be able to play with the data they collect. That means data needs to be protected only lightly by copyright and it needs to be delivered in formats that are friendly to manipulation.

The federal Treasury is regarded as a pioneer in this area because it publishes the budget under copyright that allows copying and manipulation. Last year, however, The Age sought access to the spreadsheets that comprise the federal budget. The government responded that the tables were provided on the web in html, the language that defines the content of web pages. But the tables are not provided in a way that makes extraction easy. Images of the tables are provided. To extract the numbers would be like trying to extract just the haircuts from the pictures of every parliamentarian on the Parliament House website: possible, but extraordinarily time consuming. ''It's not enough to just open up the licensing,'' said Sebastian Chan, former manager of web services and digital at NSW's Powerhouse Museum and one of the members of the taskforce. Rather, Chan says, it is vital to make data available ''in a format that is useful to people''. The taskforce recommended government agencies abandon Crown copyright, which restricts the public's access and use of data published by the government. Instead, the government should adopt a creative commons scheme that would allow open access and use. Furthermore, data should be available free of charge: ''free as in libre and free as in beer'', the report reads.

Government data should be distributed in a manner that is ''easily discoverable'', ''understandable'', ''machine-readable'' (that is, available in a format accessible to computers), and ''freely reusable and transformable'' (that is, available in a format hackers can hack). ''Information collected by or for the public sector is a national resource, which should be managed for public purposes,'' the report reads. ''That means that we should reverse the current presumption that it is secret unless there are good reasons for release, and presume instead that it should be freely available for anyone to use and transform unless there are compelling privacy, confidentiality or security considerations.'' The Rudd government accepted 12 of the taskforce's 13 recommendations with very few reservations (the 13th recommendation, which suggested ''information philanthropy'' should be tax deductible, was deferred). By October 2010, agencies were expected to have established creative commons as the default copyright licence for public sector information. The website data.gov.au was established to be a clearing house of government data. In the most tangible victory for the taskforce, the 2010 budget was published under creative commons copyright. And yet, change feels slow to many. As the two-year anniversary of the taskforce approached, Alexander Sadleir, a member of the GovHack team that created LobbyLens, surveyed 187 federal government websites and found that only 20 had adopted the recommended creative commons copyright.

Just 46 agencies were engaged with Twitter. Many more - 117 - had dedicated email accounts for requests under freedom-of-information laws. About the same number hosted links to that agency's publication scheme. Former public servant Craig Thomler posted his assessment on his blog, eGov AU. ''The Australian government has implemented and completed the vast majority of the commitments they agreed to,'' he wrote. ''There are a few areas where commitments were not actually made (though some might have liked them to be), a few where meeting the agreed commitment might have been done in practice but not in spirit and a few where changing circumstances have changed how commitments were implemented.'' Today, data.gov.au, which is administered by the Australian Government Information Management Office (AGIMO), contains some gems. For example, there is a full copy of the federal government's staff directory, with phone numbers and email addresses for government employees. However, much of the data is historical. Little distinction is made between data from, for example, 2007, and data from last month. There are ways to make it easier to collect new updates, but these are not necessarily plain to a visitor to the site. While 114 agencies are represented, the steady contributions tend to come from a small set of agencies that appear to be devotees. There are at present 19 apps based on the data available. Three of them are public toilet maps. The current sum total of the collection is 1120 data sets.

John Sheridan, AGIMO's assistant secretary, concedes that the total ''doesn't sound like a lot when you compare it to the 40,000 [data sets] that the UK is talking about''. However, he says the wealth of data published by the Australian Bureau of Statistics and the Bureau of Meteorology lifts Australia's standing in the stakes substantially, though both of these institutions existed, and were publishing prolifically, before the taskforce report. Perhaps most importantly, he says it is not AGIMO's responsibility to push disclosure on agencies. Rather, it provides one portal through which disclosures can be made. Agencies might also choose to publish data on their own website. A survey released this month shows that nearly 90 per cent of government agencies published information on their websites, as required. Only about half, however, said that all or most of the information they published was available in recommended formats. Asked if he thinks enough is being done to encourage open data, Sheridan says the government is ''doing what it said it would do''.

Nicholas Gruen tells The Sunday Age that progress was always going to be slow, but he's encouraged by changes. Being involved in the taskforce was, he says, ''like going back to the 1960s'', because of the disjuncture between the old guard of public servants, trained in a culture of tight control, and the younger crop of public servants, who have grown up in the internet age, in which data is expected to be open and free. ''There is a cadre of people who use expressions like 'does this person get it?','' Gruen says. As people become increasingly engaged with data applications (such as Facebook or Twitter), they will demand more and more data be available by default - not just to encourage transparent government, but to create business opportunities. ''I'm someone who has extensive ideas about what might be possible, but I'm also a realist,'' Gruen says. ''I'm also quite aware that, once things have started to change, they can change quite quickly.''