Being Big Brother isn't quite as glamorous as you might think. Disheveled desks, piles of used coffee cups and the quiet hum of professional voyeurism: That's how one email surveillance industry expert described the offices at a major financial institution where low-level employees sift through a seemingly endless stream of emails automatically flagged for review. "You didn't want to walk in there and make a noise, because it looked like anyone could crack at any second," the expert said. "The people who stare at that stuff start to get paranoid … they start getting conspiracy theories in their heads. I don't know anybody who's done it for more than two years." That description offers a rare peek behind the scenes into the employee surveillance systems used by financial companies to ferret out wrongdoing, fraud and other undesirable behavior. SEC and FINRA rules require companies to supervise employee communications, but there is little government guidance about what exactly companies should monitor those communications for.

Zachary Scott | Getty Images

A CNBC review of email search software found programs that monitor for conduct that could be simply embarrassing or costly for a corporation. For example, one piece of software attempts to ferret out employees who might be about to become whistleblowers informing authorities of wrongdoing inside a company. Another program scans outbound emails for the known email addresses of major media organizations in an effort to spot corporate leakers. CNBC obtained a list of the media organizations targeted by the program, which includes all of the major television networks and newspapers including The Washington Post and The New York Times. "This is the kind of electronic surveillance that only the East Germans and Bernie Madoff could love," said Patrick Burns, acting executive director of the Taxpayers Against Fraud Education Fund, a nonprofit that supports whistleblowers. Janus Capital and at least one other large financial firm have used the software program that identifies potential whistleblowers, according to the industry expert, who asked not to be named to prevent damage to his client relationships. Janus declined to comment on the firm's policy.

"Keep in mind that the compliance industry is based on 'complying' with the law, not doing what is right," the expert said. "Some firms take that to the extreme when it comes to things they need to watch out for." Insiders say the cottage industry of email surveillance exploded after the Enron scandal and passage of Sarbanes Oxley financial industry regulations in 2002. For most companies, what constitutes a potential risk is laid out in sprawling "policy" documents that seek out signs of broker error, coercion, threats, deception, bid-rigging and other inappropriate or illegal actions. Other policy options include lists of racial slurs, terms linked to fantasy sports leagues and indications that an employee might be getting ready to resign. CNBC reported last week that Goldman Sachs monitored its employee email for a list of more than 180 phrases, including expressions as seemingly mundane as "I am not a happy camper." Despite the sophisticated targeting, the digital dragnet still brings in a huge haul of emails that must be reviewed by company employees. "When there's 90 or 99 percent false positives, it's a brutal, tortuous exercise to keep focused and look for the 1 percent among all the crap," said Tim Estes, CEO of Digital Reasoning, which has worked for Goldman Sachs and Credit Suisse.

The scope of communications monitoring has widened dramatically over the years. It's not just emails any more — some companies also track instant messaging, Twitter, LinkedIn, Facebook, text messaging on company devices, web activity and trading systems like the Bloomberg terminal. Companies can track when an employee entered or left the building, when they logged on to their computer and when they made phone calls or trades, said Christopher Amatulli, director of services and architecture at Technically Creative, which offers supervisory services. "The industry is getting more and more advanced at identifying who you are and what you're doing," said Amatulli. "It's getting to the point where they can almost predict when somebody is going to do something." Employees' personal lives can be dredged up in that expansive dragnet. According to the industry expert, the company's email filter revealed an affair between a department head and a person in the office. The relationship was going poorly and the employee was threatening to tell the department head's wife, using language that jumped the message up to a reviewer. "I believe that companies need to manage their risk," said the expert. "But when it crosses the line to looking at the guy's personal LinkedIn or Facebook page, that's when I'm starting to think this is getting a little ridiculous." Industry experts said companies have expanded their monitoring programs into other communication channels at the guidance of regulators, which made it clear that business-related messages are not limited to email. Neither FINRA nor the SEC would comment on whether tracking journalists and whistleblowers are a reasonable interpretation of the regulators' guidelines.