We live digital lives now, flitting from Facebook to YouTube, checking our iPhones and BlackBerries, and chatting with our loved ones on Skype. Very few of us worry too much about tweeting our personal opinions on politics or chatting with a new social network "friend" on the other side of the world, whom we barely know and often forget in a matter of a few hours or days.

Yet all these interactions have become fodder for a new industry that secretly vacuums up the data and preserves it forever on high-end servers that hold many petabytes (a million gigabytes) of information. This industry offers new tools to search that data and reconstruct our past, and even our real-time movements via our mobile phones, in a way that could well come back to haunt us.

WikiLeaks has just released the Spy Files – a trove of almost 300 documents from these companies that shine a light into this industry. At the Bureau of Investigative Journalism, where I work, we trawled through these documents, and tracked down yet more material which our research team – Matthew Wrigley, David Pegg, Christian Jensen and Jamie Thunder – used to create an online database that will soon cover over 160 companies in some 25 countries.

It's worth spending some time browsing through this material because what this new industry offers to do is nothing short of Orwellian.

"We all aware of traditional spy stories of intelligence agencies like MI5 bugging the phones of one or two people," Julian Assange, the founder of WikiLeaks, told the Bureau. He continued:

"In the last ten years, something else has happened. We now see mass surveillance, where computer systems of an entire country are infected by surveillance programs, where the entire phone calls of a nation can be and are recorded by a company. "Previously, we had all thought, why would the government be interested in me, my brother? My business is not interesting; I am not a criminal. Now these companies sell to state intelligence agencies the ability to spy on the entire population at once and keep that information permanently. In five or six years' time, if your brother or someone becomes of interest to that company or the government, they can go back in time and look to see what you said or what you emailed."

For example Glimmerglass, a company in Hayward, California, offers "non-intrusive intercepts of any fiber asset" such as "submarine landing stations", which this slide presentation demonstrates. Eric King, a human rights adviser to Privacy International, told me that means that Glimmerglass equipment can help government agencies secretly tap into the undersea cables that convey all the data and phone traffic between continents.

WikiLeaks also released a brochure from SS8 of Milpitas, California, touting its Intellego product that allows security forces to "see what they see, in real time" including a "target's draft-only emails, attached files, pictures and videos".

Yet others, like Blue Coat of Sunnyvale, California, sell web filtering tools that allow countries to block websites "to meet cultural and regulatory requirements" – a tool much sought after by authoritarian governments to make sure that citizens do not access dissident sites. Not too long ago, Syria bought some Bluecoat machines via a reseller in Dubai. (The company is not allowed to sell directly to Syria because of US sanctions.)

Government authorities and the makers of these products argue that there is an urgent need for these tools – to track down criminals and terrorists, to block child pornography and computer viruses – a practice known as "lawful interception". This is big business for i2, a company based in McLean, Virginia, which sells two major products – CopLink and Analyst's Notebook – software that allows law enforcement to make sense of reams of data. Government Computer News ran a story earlier this week about the "Digital Dragnet" – extolling the benefits of data analysis.

The magazine quoted Bob Griffin, the CEO of i2: "When we started this more than ten years ago, we talked about things like information-sharing and gathering as much data as you could," Griffin said. "In those days, people would look at you like you're a green banana. Why would I want to share information? Why would I want to bring information from business licenses or hunting and fishing licenses into the policing environment?"

Police authorities are excited about the potential: Jason Scheiss, analytical services division manager at the Durham police department in North Carolina, told Government Computer News that they were hoping to expand the data-collecting to include data on water and sewage billing, visitor logs from parks and recreation facilities and correlate it with the daily jail list. "So we could say, 'Hey, look here. All of these crimes only occur when this one guy's not in jail,'" he told the magazine.

Therein lies the rub: apart from the massive violation of individual privacy, or the risk of abuse by corrupt officials, these tools could easily allow security agencies to jump to the wrong conclusion. Indeed, these tools have the potential to make computer cables as dangerous as police batons.

"What we are seeing is the militarisation of cyberspace. It's like having a tank in your front garden," says Assange.

You have been warned and you have a choice: you can avoid the wonderful world of the internet (unlikely, since you are reading this online) and digital data (virtually impossible if you pay for electricity or go camping) – or you can join the movement to say there need to be limits to how government authorities use our information against us. And if you choose the latter, check out WikiLeaks and Privacy International.