Stretch your mind from here to the end of the universe. Now imagine that space filled with bits and bytes, facts and factoids — a mass of information that never ends, never dies and, like the universe, is always expanding.

You can’t?

That’s because Big Data — the unimaginably vast trove of material logged here on Earth about everything on Earth and beyond — is just too big to fit between our merely human ears.

It began with the ability to compile and calculate large amounts of information on computers, got a rocket boost from digitalization, and has now reached “datafication,” which transfers every bit of information to a “data format” to make it quantified — and used in new ways we haven’t even begun to envisage.

“As a result,” say authors Viktor Mayer-Schonberger and Kenneth Cukier in their book Big Data, “we can unlock the implicit, latent value of the information.”

The implications are staggering, for better and worse. Some of them have left critics gasping at the reach of spy services into our lives, creating a subterranean 21st-century Stasi state.

Others inspire scientists with a wealth of sped-up discoveries that eliminate decades of earlier research, plot the dangers of epidemics, organized crime and climate change, and save the lives of patients whose treatment is directed by doctors, or even robots, halfway around the globe.

Still others determine how much we’ll pay for our groceries, airline flights or babies’ diapers, calculating our lives by their commercial worth or worthlessness. And at the same time threaten our dangerously interdependent global economic system with cascading market meltdowns.

Welcome to Big Data: the good, the bad and the ugly.

THE GOOD

Can all this accumulated knowledge make our world a better place?

Some scientists, medics and humanitarian aid workers believe so.

They have found that access to Big Data can be an engine for problem-solving, producing patterns that would never be detected or confirmed in other ways.

Not only do the results produce massive data sets, but they “better serve the billions of people who generate the data, and ultimately the societies in which they live,” says a Harvard University project on Big D.

That includes creating models to show when, where and why people may go hungry in Africa; how best to spend scarce resources on education in developing countries; how cross-referencing cellphone data can help upgrade services for the 1 billion people who live in slums. Early detection of climate change can protect populations in vulnerable areas who would otherwise be caught in crisis.

The public health implications are huge, from data mapping of diseases and treatments to better target results, to developing new gene therapy, to locating organ donors and tracking epidemics that can alert health authorities to the ways in which the diseases spread.

People surviving natural disasters can be located and helped more quickly and accurately with bigger data sets. Those too poor or illiterate to get banking services may be linked with willing financial companies. And the sheer abundance of data sheds light on a government’s transparency or corruption.

The possibilities for improving the world through Big D are only beginning to be understood.

“For those of us involved in causes it’s a new frontier,” says Canadian Rob Muggah, research director of the Brazil-based Igarape Institute.

Muggah, a leading researcher on the deadly global arms trade, was startled to get a call from Google Ideas — a think-tank offshoot of the mammoth company — asking him to help locate some arms dealers.

“It seemed that Google had put serious resources into disrupting illicit networks like human trafficking and guns,” he said. And it was seeking the widest possible input for the next step.

For Muggah, it was a dizzying introduction to the world of Big Data, which led to the creation of a new tool to monitor all UN-logged imports and exports of arms from dozens of sources over two decades.

Developed in partnership with Google and the Peace Research Institute Oslo, it revealed patterns and trends in the global arms and ammunition trade, including “hot spots” that could be predictors of new conflict zones.

There was also a note of caution: some of the raw arms data arrived incorrectly entered. “Big data is only as good as the smaller data you get,” says Muggah.

An argument not for less, but for better data collection.

THE BAD

A furious man walked into a Target store in Minneapolis and demanded to know why his teenage daughter was sent some inappropriate coupons in the mail.

“She’s still in high school and you’re sending her coupons for baby clothes and cribs?” he fumed. “Are you trying to encourage her to get pregnant?”

But when the manager called the distraught dad a couple of days later to apologize, he was told it wasn’t necessary. After a startling talk with his daughter, the man had learned she was, in fact, pregnant.

This, says New York Times writer Charles Duhigg, is the tip of the iceberg for contemporary consumers, in an age when Big Data can tell a marketer a customer’s baby is due before her family knows.

And it’s a cautionary tale for everyone who makes a commercial transaction, from buying a can of soup to mortgaging a home. Through “the exhaustive rendering of our conscious and unconscious patterns into data sets and algorithms,” marketers have radically transformed what they know about us, and how to aim their products at our sweetest shopping spots.

How do they siphon off enough data to predict with often-startling accuracy, our personal consumer IDs?

“Once upon a time the battle for creative people was to bring their selling ideas to market,” says Terry O’Reilly, host of the CBC’s Under the Influence. “They were trying to throw a dart at the big advertising dart board. Now, with data mining, that’s all changing. All the data about customers is right there to be used.”

Communications expert Joseph Turow, author of The Daily You, says that data mining is so sophisticated that thousands of connections can be made among the tracked data to produce profiles for advertisers and marketers.

And he writes, “the centrality of corporate power is a direct reality at the very heart of the digital age.”

Plan a trip to Togo, and before you know it, your screen is offering you Togo nightlife and beautiful Togolese women.

Buying a special on Aspirin, you think, may not give too much away. But couple it with your Google health searches, non-prescription drug purchases, exercise equipment shopping and numerous other data-mined info-nuggets and a startlingly accurate portrait may emerge.

Data warehouses have sprung up, stockpiling information as diverse or intimate as airline flights, phone calls, warranty cards, job applications, school and university records, loan applications, bank locations, insurance plans, credit card usage, demographic details, GPS tracks, entertainment buys. A trove mined from every form of electronic device you own.

“Those pieces may be pretty innocuous by themselves,” says Turow. “But companies are collecting and selling data to marketers in fascinating ways.” Data brokers are the new power brokers of advertising, peddling ready-made customers to sales-hungry corporations for ever-mounting profits.

A recent multi-region study of company behaviour by Tata Consultancy Services found that more than half of the 1,200 companies they surveyed had conducted Big Data activities within the last year, spending an average of $10 million. A quarter sold their data, with an average of $22 million in profits.

Many of those on the receiving end don’t realize it, Turow says. When deliberately dull, complicated “privacy policies” pop up on screens, they are accepted by people too busy or impatient to read them — which is most of us. “If you think they insure that the site won’t share your information with other websites or companies, think again. It means ‘we share with anybody we want to.’”

Cookies, the cute-sounding storage files that can be planted by companies, have also acquired new guile. “They find reams of data and connect them with other reams. They’re added to other databases and cookies. The data follows you around.”

That may just lead to aggressive — and sometimes useful — targeted marketing. But, Turow says, it can also identify you as a worthwhile or worthless customer, as he found out recently in Chicago’s O’Hare airport, when his connecting flight was abruptly cancelled. “There was a big sign on the monitor that said, ‘The amount of time you’re in line is determined by your status with the airline.’”

“It’s not just rich versus poor,” he adds. “Age, lifestyle and loyalty are also issues.” So a large grocery chain has stopped displaying discount prices on the shelves, and instead gives price breaks to customers who demonstrate their loyalty by buying their new app.

Age is a factor, because someone buying an expensive car at 30 is more valuable to the auto company than someone who is 75, whose shopping span is much shorter: the 30-year-old will get the best deal, while neither is any the wiser.

To target even more narrowly, companies are experimenting with body sensing through closed-circuit cameras and the cameras and microphones built into the devices people use every day. A flickering eye, a quickened heartbeat become valuable data.

But as Big Data in the service of marketing grows, it also has unintended consequences.

While consumers may shrug off the impact of data tracking, it becomes more serious if the information is passed on to government security services.

And, says Turow, “the elephant in the room is what data mining is doing to the media. The history of the 20th century was that the media is supported by ads, so you don’t really have to pay. Now that’s come back to haunt them. Advertisers are spending more, but not on the ‘legacy’ media.”

By turning to social media instead of buying air time or space, advertisers can now buy customers, direct and content-free.

THE UGLY

From spycams to webcams, biometrics to GPS, mobile “metadata” to Internet data drilling: are we living in an era of total surveillance?

If not here already, it’s coming closer, says Viktor Mayer-Schonberger, co-author of Big Data. “We’re collecting more information than ever before and most of it is in a digital format. That creates a much more comprehensive capability for surveillance.”

And, he adds, Big Data never dies. “The ephemeral quality has been taken out of our communications. We used to depend on things we said or did being plausibly deniable. They would just go away. Now what I say will stay with me and may be used against me in the future.”

Surprisingly, many North Americans say “meh” to these concerns, even in the swirl of growing scandal around U.S. Secret Service data mining, and reports that Canada has gone down the same electronic rabbit hole.

The U.S. scandal began last month with reports of the secret seizure of phone records from The Associated Press to track a leak of material the government said endangered national security.

It escalated with revelations that the super-secret National Security Agency’s data-mining activities extended from scooping large-scale “metadata” from customer phone records to the PRISM Internet surveillance program, which allowed for digging through emails, videos, photos and other material of foreign users from nine leading technology giants — including Google, Microsoft and Facebook.

Debate also raged north of the border, where the covert Communications Security Establishment Canada — the NSA’s Little Brother — was in the spotlight with accusations that it too could be targeting Canadian email and phone logs.

But one person’s surveillance is another’s security.

Recent U.S. polls by the Pew Research Center show that many citizens, if not a majority, believe that governments have the risk versus reward equation just right.

In an age when fear of terrorism has far exceeded fear of more common domestic-based violence, the public is willing to tolerate higher levels of surveillance in return for what it believes is protection against terrorist plots.

In the U.S. this week, intelligence officials said that NSA surveillance had helped to thwart more than 50 such plots, including one to blow up the New York Stock Exchange, and another by an Afghan American planning suicide attacks in New York.

Officials maintain these are not rogue spy missions, but surveillance programs overseen by layers of agencies, the Justice Department and judges on a special Foreign Intelligence Surveillance Act court — as well as House and Senate committees.

But when so much data is mined by so relatively few in such pervasive secrecy, critics are not reassured. While the lives of those targeted by surveillance are increasingly open to scrutiny, those doing the targeting remain opaque.

“It’s not that bad people are abusing power,” says Robert McChesney, author of Digital Disconnect. “It’s that when anyone in government has such broad powers you should expect them to be abused.”

Ron Deibert, who heads University of Toronto’s Citizen Lab, is worried by the complacency of those who trust that data collection is benign. His book Black Code: Inside the Battle for Cyberspace, is a “wake-up call” to millions of electronically connected people.

Loading... Loading... Loading... Loading... Loading... Loading...

“Terrorism is a serious concern,” he says. “But that obscures a bigger trend.

“There has been a sweeping rollback of safeguards around national security relating to terrorism. At the same time, there’s an enormous explosion of Big Data: its volume is extraordinary and it tracks our every movement. It’s opened business opportunities of mining and analyzing data partly for commercial purposes but also for national security.”

Because we don’t know how our data is being mined, or by whom, or how it’s being used as it’s relayed and routed across the world — and because it’s all but impossible to unplug from the interconnected system — it’s time to step back and take a harder look, Deibert says.

Making cyberspace secure as a medium for the sharing of ideas and democratic debate means openness, but also restraint, he argues. “There should be checks and balances on government, law enforcement, intelligence agencies and on the private sector.”

Many who hear of sweeping surveillance powers shrug because they don’t believe that law-abiding citizens will be targeted.

But a new report by Beau Hodai for DBA Press and the Wisconsin-based Center for Media and Democracy shows how those with dissenting views within the law can be singled out for surveillance and placed on wide-ranging watch lists relating to terrorism.

The report, “Dissent or Terror,” analyzes thousands of pages of freedom-of-information records obtained from counterterrorism and law-enforcement agencies who targeted members of Occupy protests, many in Arizona.

It shows how data, including from facial recognition technology, Facebook, driver’s licenses, and infiltration of protest groups, was pipelined from local police — sometimes hired by corporate interests the groups’ members were protesting — to state-based “fusion centres” with links to federal security services and funding from the U.S. Department of Homeland Security.

“The term ‘fusion centre’ was coined by the Department of Defense,” says Hodai. “It’s a fusion of municipal, county and state counterterrorism personnel and agencies.”

It’s also an “information food chain,” he adds. “If you’re standing in the park with Occupy and a street policeman gives you a trespass warning, it gets passed up the chain.”

A terrorism analyst would begin monitoring, and a range of data gathered about you would be passed on to intelligence agencies to create a file that could expand throughout your life, regardless of whether your behaviour is within the law. A chilling thought for those who dissent.

“The information never goes away,” said Hodai. “It’s not just about gathering, it’s about storing indefinitely. And it’s also used not just by federal authorities, but the guys at the low end of the totem pole.”

DATA ON DATA

1 gigabyte

Size of one full-length feature film

-

10 million

Photos uploaded on Facebook every hour

-

400 million

Tweets on Twitter each day

-

500 million

Internet-connected devices in 2003

-

12.5 billion

Internet-connected devices in 2010

-

6.86 billion

World population in 2010

-

$50 billion

Google’s revenue for 2012

-

$1.2 billion

Cost of new NSA spy centre in Utah

-

51

Percentage of U.S. 18- to 29-year-olds who prefer anti-terrorism surveillance to privacy

-

66

Percentage of U.S. adults over 30 who prefer anti-terrorism surveillance to privacy

-

Sources: Big Data, Pew Research Center for the People and the Press, Washington Post, Star news services, Cisco Internet Business Solutions Group, World Bank

Read more about: