On the morning of March 29, 1933, dozens of reporters filed into the Oval Office for a press conference with the new president. Franklin Roosevelt had taken office earlier that month amid the greatest economic crisis the US had seen: 5,700 banks had failed, 25 percent of the country was unemployed, and more than half of all mortgages were in default. This article has been reproduced in a new format and may be missing content or contain faulty links. Contact wiredlabs@wired.com to report an issue. Hope for a recovery was dim; the public had lost faith in the entire financial system. The number of American investors had exploded, from a few hundred thousand before 1916 to more than 16 million. Yet few of them understood the investments they held, many of which had proven to be junk. Supposedly sound companies were exposed as pyramid schemes. Of the $50 billion in securities sold in the previous decade, half had become worthless. And yet, as reporters huddled around his desk, Roosevelt sounded confident. "I have something on the Securities Bill today," he announced. That day, members of his brain trust were on Capitol Hill, submitting a plan that would spark the creation of the Securities and Exchange Commission. One overriding concept lay at the center of the legislation: transparency. Louis Brandeis, before becoming a Supreme Court justice, had written an exposé of the financial system for Harper's Weekly, and one passage in particular had lodged in Roosevelt's brain: "Sunlight is said to be the best of disinfectants. Electric lights the most efficient policeman." The proposed bill would require, for the first time, companies to file detailed accounts of their financial health and activity, and bankers would have to report their fees and commissions. As Roosevelt explained it to the reporters around him, the bill "applies the new doctrine of caveat vendor in place of the old doctrine of caveat emptor. In other words, 'Let the seller beware as well as the buyer.' In other words, there is a definite, positive burden on the seller for the first time to tell the truth." Now, here we are again, 76 years later, facing another crisis of trust that threatens the entire financial system. This time, the issue is no longer a lack of transparency. Since the 1933 Securities Bill, corporate America has been required to disclose a deluge of information in a multitude of ways—10-Ks and 10-Qs, earnings calls and Sarbanes-Oxley-mandated 404s. Between 1996 and 2005 alone, the federal government issued more than 30 major rules requiring new financial disclosure protocols, and the data has piled up. The SEC's public document database, Edgar, now catalogs 200 gigabytes of filings each year—roughly 15 million pages of text—up from 35 gigabytes a decade ago. But the volume of data obscures more than it reveals; financial reporting has become so transparent as to be invisible. Answering what should be simple questions—how secure is my cash account? How much of my bank's capital is tied up in risky debt obligations?—often seems to require a legal degree, as well as countless hours to dig through thousands of pages of documents. Undoubtedly, the warning signs of our current crisis—and the next one!—lie somewhere in all those filings, but good luck finding them. Even the regulators can't keep up. A Senate study in 2002 found that the SEC had managed to fully review just 16 percent of the nearly 15,000 annual reports that companies submitted in the previous fiscal year; the recently disgraced Enron hadn't been reviewed in a decade. We shouldn't be surprised. While the SEC is staffed by a relatively small group of poorly compensated financial cops, Wall Street bankers get paid millions to create new and ever more complicated investment products. By the time regulators get a handle on one investment class, a slew of new ones have been created. "This is a cycle that goes on and on—and will continue to get repeated," says Peter Wysocki, a professor at the MIT Sloan School of Management. "You can't just make new regulations about the next innovation in financial misreporting." That's why it's not enough to simply give the SEC—or any of its sister regulators—more authority; we need to rethink our entire philosophy of regulation. Instead of assigning oversight responsibility to a finite group of bureaucrats, we should enable every investor to act as a citizen-regulator. We should tap into the massive parallel processing power of people around the world by giving everyone the tools to track, analyze, and publicize financial machinations. The result would be a wave of decentralized innovation that can keep pace with Wall Street and allow the market to regulate itself—naturally punishing companies and investments that don't measure up—more efficiently than the regulators ever could. The revolution will be powered by data, which should be unshackled from the pages of regulatory filings and made more flexible and useful. We must require public companies and all financial firms to report more granular data online—and in real time, not just quarterly—uniformly tagged and exportable into any spreadsheet, database, widget, or Web page. The era of sunlight has to give way to the era of pixelization; only when we give everyone the tools to see each point of data will the picture become clear. Just as epidemiologists crunch massive data sets to predict disease outbreaks, so will investors parse the trove of publicly available financial information to foresee the next economic disasters and opportunities. The time to act is now. An exhaustive study by the Transparency Policy Project at Harvard University's John F. Kennedy School of Government—analyzing disclosure rules for everything from restaurant cleanliness to SUV rollover risk—found that there's a very brief window after any calamity for government to institute changes. (Wait too long and the special interests start regaining their confidence and pushing back.) In the financial world, the old order is still trying to find its new shape. So the window is, briefly, cracked. Caveat vendor.

Philip Moyer, CEO of Edgar Online, says data is the key to spotting crises before they start. Photo: Angela Cappetta Philip Moyer, CEO of Edgar Online, walks into his conference room in midtown Manhattan a half hour late, clutching an inch-thick stack of copy paper. He's a broad-shouldered guy with dark brown hair pushed back from his forehead, as if a fan is constantly blowing directly onto his face. He slams the paper down theatrically: "One reason I'm a little bit delayed is that I started printing out a Bear Stearns free writing prospectus," he says. "The assets cover 462 pages. I got about 70 pages through." Every bank that issues mortgage-backed securities—pools of home loans packaged together and sold as a single entity—is required to file a free writing prospectus, which lists every individual mortgage in each pool. An FWP contains endless columns of pure data, most of which don't even track from page to page. And each FWP is different: The banks have no uniform information that they're required to present in their filing. Even when they do report the same data, they do so using entirely different language. And yet somewhere among all this impenetrable code lie the bugs that destroyed the American economy. Numbers Don't Lie Dan diBartolomeo head of a Boston financial analysis firm, spotted Bernard Madoff's $50 billion scam. Here's what he sees coming next. —Daniel Roth Dan diBartolomeo head of a Boston financial analysis firm, spotted Bernard Madoff's $50 billion scam. Here's what he sees coming next. —Daniel Roth Wired: In 1999, you were hired by a money manager to reverse-engineer Madoff's investment strategy. When did you realize something was amiss? Dan diBartolomeo: All we had were the monthly returns that Madoff reported to investors. We spent a couple of hours on mathematical analysis, playing around with regressions and spreadsheets, and concluded that the results couldn't have come from the strategy he described. Wired: Did you immediately think fraud? diBartolomeo: It was possible that he was using some other strategy he wasn't disclosing. But to get returns like that, he would have needed to be three or four times more skillful than the next-best manager. He also could have been using a strategy that gave him an illegal edge. That would have accounted for the returns being high, but not steady. The third possibility was that the numbers were just made up. And that's what I reported. Wired: Do you think your degree in applied physics means you look at the market differently? diBartolomeo: One of the things you learn in engineering is to be rigorous. If you build a bridge that falls down on a windy day, there's going to be hell to pay. Financial markets are not like that; they are very noisy. It's hard to tell who's skillful and who's just lucky. And a lot of analyses are done in extremely haphazard, primitive ways, but the investing public doesn't know any better. Wired: Did your formulas predict last year's market collapse? diBartolomeo: We weren't surprised. Back in 1998, we looked at how ratings agencies were handling collateralized loan securities. They did a crap job. The math of this stuff is complex, and they took a lot of shortcuts in an effort to make it more understandable. Wired: Have you spotted any problems elsewhere? diBartolomeo: Today, a lot of pension funds have lost a lot of money. Actuaries evaluate them by taking future payouts—the money that will actually go to retirees—and discounting them by a single interest rate. It doesn't matter if they have to pay the money out in three weeks or 30 years. But if you look at financial markets, the interest rate you get on a three-month CD is different from what you get on a 30-year bond. It leads pension funds to take on more risk than they can afford. Wired: So could better math have prevented the market crisis? diBartolomeo: People are investing in complex securities they don't understand. The big failures aren't data failures; they aren't issues of "We don't know." They're issues of "We don't want to make the effort to be rigorous." Moyer discovered this in the spring of 2007, when two hedge fund managers independently asked for his help in making sense of some major banks' FWPs. Poring through all that paperwork by hand would take countless hours, and they wanted Moyer to extract and package the data in a way they could easily understand. Moyer, a former Microsoft executive, assigned four engineers to categorize and standardize the FWPs' contents—creating a Rosetta stone that could translate the 600 unique, inconsistent fields into 100 uniform categories. Three months later, he started delivering spreadsheets that clearly spelled out the risks in each of the pools, giving the financiers the ability to evaluate every aspect of the loans: location, proof of income, interest rate, appraisal value, and so on. They could drill down and compare the FWPs in a way that would have been nearly impossible before. And what they saw was a nationwide crisis in the making—as adjustable-rate mortgage rates ballooned, countless home-owners would default on their loans, rending the securities built on them worthless. Of course, the hedge-funders didn't publicize their findings; they were seeking an informational edge. But imagine if everyone had access to the same data-crunching tools: Risky mortgage-backed securities would have been exposed, and banks, anxious to protect their reputations, would have stopped offering them. With complete information—including much more frequent posting of loan status—the market would likely have self-regulated as risk-fearing investors fled from companies holding or issuing the risky securities. That's the kind of scenario that has kept Charlie Hoffman motivated for the past decade. A 50-year-old accountant from Tacoma, Washington, Hoffman is the originator of XBRL, a set of tags that standardizes financial information. Hoffman stumbled on the idea while trying to figure out a way to automate the tedious auditing process. ("Basically, I'm lazy," he says.) But while Moyer's team was forced to create complicated algorithms to codify kludgy financial documents after they were filed, Hoffman is agitating for companies to file their data in a standardized format from the very start. Today, nearly 50 companies report their information in XBRL to the SEC, but Hoffman says the protocol's real power will be realized only when every company starts using it—to keep track of their own operations as well as to report their numbers to investors and regulators. If all businesses are required to tag their every move, from each iPhone sold by Apple to every interest payment made by Exxon, they won't be able to engage in the kind of balance-sheet chicanery that kept Enron's investors in the dark. "Financial reporting should work the way that an iPod works," Hoffman says. "It should just be elegant and simple." A few years ago, when banking regulators started requiring filings in XBRL from its member banks, it found that the time it took auditors to review a bank's quarterly financial information dropped from about 70 days to two. More regulators are catching on: Last December, the SEC announced that by June, every company with a market capitalization over $5 billion will be required to submit all filings using the format. And all publicly traded companies and mutual funds must follow suit by 2011. The result, Hoffman says, is that every investor will soon have the same ability as Moyer's hedge fund managers to export, manipulate, and mash up financial data. "Look how blogs changed news reporting," he says. "Anybody is a reporter. With XBRL, anyone can be an analyst."

Transparency Now! A Wired Manifesto Set the data free Today, public companies and financial institutions disclose their activities in endless documents stuffed with figures and stats. Instead, they should be forced to file using universal tags that make the data easy to explore. Empower all investors Once every company's data carries identical tags, anyone can manipulate the numbers to compare performance. And they can see details of every financial instrument—not just balance sheets and income statements. Create an army of citizen-regulators By giving everyone access to every piece of data—and making it easy to crunch—we can crowdsource regulation, creating a self-correcting financial system and unlocking new ways of measuring the market's health. But the government is just playing footsie with the kind of reform that's needed. If future financial crises are to be avoided, XBRL shouldn't be limited to public companies. It should become the lingua franca of every investment bank, hedge fund, pension fund, insurance company, and endowment fund. Today these groups contribute to a multitrillion-dollar shadow banking system of lightly (or not-at-all) regulated financial instruments that move markets and tend to bring outsize riches—until they blow up. Take collateralized debt obligations. These are mortgage-backed securities blended with other assets—say, auto loans or credit card debt—into one asset-backed pie, sliced up according to risk and sold as an investment. It is impossible to track any one loan in a CDO; when it is combined and divided with other loans, it loses its independent identity. When the ratings agencies tried to determine default risks for CDOs, all they saw were vaguely defined pools of assets. They had little idea what was in them, and their models—like David X. Li's ubiquitous copula function (see Recipe for Disaster: The Formula That Killed Wall Street)—would prove inadequate at evaluating them. But if those mortgages and loans carried XBRL tags, and everybody who touched them along the way was required to use those tags as well, anyone would have been able to track their circuitous route through the financial industry and judge each CDO based on its actual content. They could have seen which loans were in default and which weren't, which CDO was overweight on Las Vegas real estate and which was in the relatively safe Louisville market. An amateur risk assessor could have separated the junk assets from those worth keeping and either bet against the companies holding the garbage, blogged about it, alerted the Feds—or all of the above. (The very act of disclosure may compel companies to behave better in the first place: When Los Angeles started requiring restaurants to post their hygiene grades in their windows, average cleanliness increased by 5 percent and revenues by 3 percent.) Tracking Wall Street's complex inventions may be difficult for regulators, but it's a snap given the right software. "I did a lot of work in clinical trials information when I was at Microsoft," says Moyer, who is a big believer in XBRL. "And if you look at the numbers that are involved in genomics, proteomics, and cell-level sequencing, those problems dwarf what we're dealing with here. It's a simple computer problem." When data is kept under lock and key, as mysterious as a temple secret, only the priests can read and interpret it. But place it in the public domain and suddenly it takes on new life. People start playing with the information, reaching strange new conclusions or raising questions that no one else would think to ask. It is impossible to predict who will become obsessed with the data or why—but someone will. Last fall, Kevin Bartz was seeking information about the mortgage business. Bartz, a PhD student in statistics at Harvard who had worked for Google, Microsoft, and Yahoo, was earning extra money doing consulting work for a mortgage broker in Pasadena, California. The company wanted to pool some of its mortgages and find buyers for the debt. But selling the securities required being able to explain how these assets had performed in the past. Bartz found that most of the information he needed was locked up in proprietary databases. There was no way to know basic information about the loans his employer wanted to hawk—where they had originated, whether they had been paid on time, whether they had defaulted. He was struck by the lack of transparency and broadened his project: Discover a way to assess credit risk and beat the banks at their own game. His research led him to LendingClub, a Web site that matches individual lenders with borrowers who need loans. Like other peer-to-peer lending companies, LendingClub asks borrowers to provide personal details—education, employment history, salary—and to write essays explaining why they want a loan and how they plan to pay it back. LendingClub runs its own credit checks, sorts borrowers by default risk, and comes up with interest rates. But LendingClub is unique in that it makes nearly all that information public (aside from data that could lead to privacy concerns), giving lenders the ability to sort through its database. It also tracks and publishes the history of every loan it helps broker. Bartz downloaded the database of 4,600 loans—every essay, every neighborhood, every late payment—and started searching for patterns. He identified the 300 most common words in borrowers' essays and correlated them with payment histories. Sure enough, certain words seemed linked to late payments. Among the red flags: need, bills, and business. "Those were all words that reflected that the borrower might be in financial difficulty at the moment," Bartz says. Another one was also, which Bartz theorizes meant that the loan was being used for more than one purpose. Bartz wasn't the only one poking around in the pixels. Besides providing the data on its customers, LendingClub posted to its Web site the formula it uses to measure default risk and determine the interest rates its borrowers had to pay. Most banks keep this information secret—a perfectly honed algorithm can give them a competitive advantage—but LendingClub open-sourced it and asked readers to submit their own tweaks and improvements. After receiving a slew of suggestions, the site's engineers decided to modify the equation, assigning less weight to debt-to-income ratio, for instance. Other LendingClub lenders downloaded the equation and came up with their own proprietary improvements, devising a better formula so they could cherry-pick borrowers who were wrongly categorized as risky and charge them higher interest rates without worrying about defaults. All this innovation benefited not just individual lenders but the entire ecosystem. LendingClub's default rate is a staggeringly low 2.7 percent (versus nearly 5.5 percent for prime credit cards).