We’re finally on the brink of the cashless society that futurists and other have been forecasting for years. The average consumer owns at least two credit cards and early adopters have begun ditching plastic for virtual wallets. Even businesses that used to rely heavily on cash — think taxis, food trucks or even craft fairs — can now go cashless, thanks to new technology like Square.

Yet, the more we abandon paper bills for plastic, smartphone payments and even cryptocurrencies like Bitcoin, the perils of the new, cashless economy are becoming more apparent. Recent security breaches at Target, Neiman Marcus and other retailers illustrate the vulnerability of electronic payments to hacking attacks.

Related: Why Cash May Not Be King Anymore

There were 2,164 reported security incidents exposing 822 million records last year, nearly doubling the previous highest year, 2011, according to Risk Based Security, a data security firm. The pace seems to be continuing this year.

Chip-and-pin cards, in use in Europe for years, would remove some of the threat. Computer chips embedded on the cards verify unique personal identification numbers consumers punch into terminals. The cards would help reduce fraud, but it's not clear how much. For instance, they don't help prevent online fraud.

The problem is cost. Making the cards and installing the new terminals needed to read them is expensive. Consumer advocates want Congress to press retailers and banks for fast adoption of chip-and-pin technology.

"Some institutions in the U.S. say they will switch to this technology in the next few years, but we need a stronger commitment from all stakeholders," stated Delara Derakhshani, policy counsel for Consumers Union, in testimony to Congress. "Policymakers must also take action to encourage investments in technology to tighten up the security of our financial institutions."

Data breaches aren't the only downside. Going cashless in a world of Big Data means all your purchases — and therefore your privacy — is up for grabs. Did you hear the one about the woman who wanted to surprise her husband and tell him that after three years of trying, she was finally pregnant? Too late. While on line one night on their home computer, he noticed all these products geared to pregnancy and childbirth.

Of course, it’s not just companies that want to make a buck. A recent episode of “The Good Wife” showed how a fictional (we hope) NSA might operate. The agency had been listening to the “good wife’s” phone conversations at the office as well as her estranged husband, the governor. Therefore, the governor had an aide post a “for sale” notice at a local mosque for an incredible deal on a pre-owned car — the number to call was that of the local head of the NSA. Dozens of men with Arabic names called the man at his office, and he was fired.

Privacy experts suspect that if the NSA is capable of recording information about people’s phone calls, it is certainly monitoring electronic purchases. "That is a civil liberties concern," says Scott Shay, chairman of Signature Bank. "When governments have information they are sometimes tempted to use it, and sometimes they attempt to use it in ways that are not fully vetted by due process."

Since electronic payments can be traced, authorities may be able to curtail or even eliminate crime, including everything from prostitution to terrorism. But constant government monitoring could turn everyone into a potential suspect, creating distrust between government and citizens that could lead to "vast consequences," economist Domagoj Sajter of the University of Osijek in Croatia wrote in a research paper published in March 2013 titled "Privacy, Identity, and the Perils of the Cashless Society."

Individuals and organizations may be unwilling to promote unpopular or unconventional ideas if they know their transactions are being monitored. "Ultimately, this would lead to a weaker form of democracy, in which certain voices could not be heard and lobbied for."

Related: 6 Items That Could Replace Paper Money

It is possible that the government could monitor electronic purchases in an attempt to police crime, says David Stearns, a historian and sociologist specializing in electronic payments at the University of Washington. From a technological standpoint, the feat would be relatively easy. But whether it will happen remains uncertain, he says.

Even it does happen, criminals would simply use a different currency, be it digital or physical, he added. For instance, in times of hyperinflation in the past, people bartered with cigarettes. "The idea that it will eliminate all fraud and crime is a little optimistic."

Stearns sees a cash-limited society as far more likely than a cashless society.

"There are a number of cultural rites that we use cash for today," he says, citing small donations, tips, religious contributions, and gifts. "It's a lot more meaningful for grandma to send a crisp $20 bill than a piece of plastic. If we went to a cashless society, that's what we'd have to figure out."

Top Reads from The Fiscal Times: