U.S. stocks have more than tripled since bottoming out in March 2009 during the Great Recession, rising in value by a staggering $12.8 trillion.

But the average American household has been left behind. Most of the gains went to the wealthy and institutional investors including investment banks and hedge funds. Fewer than half of U.S. households own stocks either directly or indirectly, down from a peak of more than 53% in 2007.

American families that did have money were throttled by the 17-month bear market in stocks that pushed down the benchmark S&P 500 index SPX, -2.03% by 56% from a peak in October 2007. In the aftermath of the wealth destruction, caused by the deepest economic contraction since the 1930s, 8.7 million people lost their jobs and 15.5 million homes went into foreclosure.

Even as the economy has slowly recovered, many Americans remain deeply skeptical about Wall Street and its network of brokers, fund managers and retirement advisers, and even the fairness of the securities markets themselves.

The latest convulsion may have lasting effects, as a so-called new normal of slow growth in the economy has suppressed wages and slashed interest paid on fixed-income instruments. Still, loose regulations and innovative technology have enabled those with the means to embrace high-frequency trading and other controversial strategies to gain advantages unavailable to the average investor.

“There is a perception that the markets are rigged, and to some extent there is no question that they are,” says Mercer Bullard, a law professor at the University of Mississippi and founder and president of Fund Democracy, an advocacy group for mutual fund shareholders.

The bear market of 2007 to 2009 came only five years after the bursting of the Internet bubble, a period marked by conflicts of interest at Wall Street investment banks.

But once again, it was professional investors who struck it rich when the current bull market began a little over six years ago. Retail investors have pulled $661 billion from U.S. equity mutual funds and exchange traded funds since the end of 2007, while institutional investors poured in $665 billion, according to J.P. Morgan Asset Management.

With Americans ever more reliant on self-directed retirement plans such as 401(k)s, missing the post-crisis rally will have long-term implications on retirees’ standard of living for decades to come. About 46% of Americans had less than $10,000 in financial assets in the last year of their life, according to a 2012 study by MIT economist James Poterba.

Still, retail investors haven’t necessarily been irrational in their reluctance to return to the market. After the financial meltdown in late 2008, Wall Street has been scrutinized as never before, even as regulatory changes are criticized by investor advocates as insufficient. Persistent problems include financial advisers who peddle investments that are unsafe for the average person; corporations that enrich their CEOs more than they do shareholders; and a willingness by securities exchanges and regulators to let well-heeled investors buy access to data and information ahead of the general public.

Emergence of the retail investor

How did we get here? While U.S. speculators and wealthy investors had the opportunity to invest in equities since the early 1800s, either through so-called bucket shops or brokerages in big cities, the notion of a mass, retail market aimed at middle-class Americans didn’t emerge until after World War I.

Stock trading took off in the Roaring 1920s, says Charles Geisst, a finance professor at Manhattan College and author of “Wall Street: A History.” That was a particularly fruitful period for the U.S., which made the transition to a peacetime economy where industry flexed its muscles, producing cars, household appliances and other consumer goods. Installment buying, or purchasing on credit, took off. Fiscally conservative President Calvin Coolidge slashed taxes and supported laws that supported businesses.

Brokerage firm Dean Witter was the first to sell stocks to the masses, with Merrill Lynch soon joining the fray, Geisst says. The building of branch offices in cities across the country helped stoke business as investor mania took hold, a party stopped short by the Wall Street Crash of 1929 and the Great Depression of the 1930s.

At the time, the average investor regarded the stock market as a place to make a fortune overnight, not long-term gains. The elite fed dreams of riches to the masses, engaging in insider trading, front-running, pyramid and Ponzi schemes, and other manipulations that altered the playing field. Men such as J.P. Morgan Jr. and Joseph P. Kennedy Sr. amassed wealth that created or extended dynasties. (Ironically, Kennedy would become the first chairman of the Securities and Exchange Commission, where he outlawed insider trading.)

Mom-and-pop investors steered clear of stocks during the lost decade of the 1930s. In 1932, the height of the Great Depression, the unemployment rate soared to 34%, with 12 million people out of work.

The post-World War II boom rekindled Americans’ love affair with stocks. In the aftermath of the war, retail investors directly held more than 90% of U.S. equities.

And the percentage is even lower when excluding nonprofits and holdings of closely held companies.

Since 1980, the mutual fund industry has grown by leaps and bounds, aided in part by employers that scrapped defined-benefit pension plans in favor of defined-contribution plans, which put the onus for saving enough for retirement on workers. Policies that defer taxes on those retirement plans have also contributed to the shift, says Kristian Rydqvist, a finance professor at Binghamton University in New York state.

Last year, 36% of U.S. equities were held directly by households, down from more than 93% in 1945, according to the Federal Reserve. And that figure is inflated, since the Fed data include closely held companies and public-share holdings by nonprofits. Adjusting for those factors, it’s likely that less than 30% of publicly traded stock is directly owned by individuals, Rydqvist says.

These days, a retail investor is more likely to hold equities indirectly through a mutual fund or other pooled investment, and there is a good chance they hold it through a retirement plan, including a 401(k) or an individual retirement account (IRA). It is such vehicles that can fall prey to wealth-destroying fees.

In 2013, 48.8% of U.S. households held stock directly or indirectly, according to the latest Federal Reserve triennial Survey of Consumer Finances. The ownership rate had risen steadily since 1980, when it stood at 31.8%, rising to 53% by 2001. After dipping in the wake of the tech bubble in 2002 and 2003, the rate peaked at 53.2% in 2007 but has drifted lower following the Great Recession.

The definition of a retail investor doesn’t include participants in a defined-benefit plan, otherwise known as a traditional pension. But Americans with pensions are rare, and most private-sector employees must invest if they want to have a decent standard of living in retirement.

Whether trading for their own accounts or using mutual funds to save for retirement, individual investors face plenty of perils. Here’s a look at some of the biggest challenges:

(See how some brokers and advisers sell wealth-destroying products.)

1. Wealth-destroying products and brokers

Everywhere you look are dicey investments that could potentially cut into an investor’s nest egg, but high fees and conflicted advice are often hidden dangers.

A reason the securities market seems rigged is because “we have a system that suggests that investors, with enough research and analysis, can on their own do better in the stock market than simply owning an index fund,” the University of Mississippi’s Bullard says.

That perception is fed by the disclosure system, which assumes the average investor can stay up to speed simply by reading the myriad regulatory filings and announcements that publicly traded companies are required to produce, he says.

An example of massive company filings includes Chinese Internet retailer Alibaba Group Holding Ltd. BABA, -0.93% which gave about 40 pages of risk factors before its IPO in New York. With mutual funds, investors can face a confusing array of share classes, each with a different fee structure. Or you simply might not realize that you’re in a pricey fund.

Martha Keeley, 57, says she’s angry about how much she paid in fees for years because of a relatively expensive fund that she held in a retirement account. She recently followed the advice of a free online tool, FeeX.com, that analyzed her account and suggested she switch from the pricier fund to a Vanguard 2030 target-date fund VTHRX, -0.55% , estimating the move could save her about $68,000 over 30 years.

Keeley, a marketing and public-relations consultant in the Boston area, said she has been spreading the word about her experience in finding funds with far lower fees and better performance.

“I’ve been telling everybody I know, because I’m so pissed off that I’d been spending so much money on fees all these years,” she said in an interview.

BrightScope is another online service that tells investors whether their retirement account fees are high. Keeley’s Vanguard fund and others that track indexes charge less than conventional funds that are actively run by a manager. And beyond being cheaper, the index-tracking funds can provide better returns, given that most actively managed funds fail to outperform their benchmarks over time.

Read more: Why Vanguard’s target-date funds can be better than Fidelity’s

Pricey, underperforming funds are wealth destroyers, but so are some financial pros who sell such funds and even dodgier products. Brokers have been accused of pushing high-risk promissory notes to not-so-sophisticated investors with limited incomes, or just flat out stealing $15,250 from elderly clients. No wonder being a stockbroker has been described as a once-prestigious job that has lost its luster.

Even when financial pros sell seemingly harmless products, investors should be aware that some may be interested in sales commissions and are required only to meet a “suitability” standard — which means they sell products that are suitable for a client — rather than a higher “fiduciary” standard. The Obama administration is making its second attempt to subject more financial advisers to the fiduciary standard, estimating that current rules cost retirement savers $17 billion a year.

And there are dicey, albeit more niche, investing vehicles that can hurt investors. Leveraged exchange traded funds, meaning turbocharged funds that often have “3x” in their names, can end up delivering a very different return than you might expect. For example, a 3x S&P 500 fund returned a negative 14.9% in 2011, even as the S&P 500 Index returned a positive 2.1%. It got smashed by volatility, illustrating why average long-term investors should “avoid these things,” as Morningstar fund analyst Michael Rawson put it.

Non-traded REITs and non-traded business-development companies also have gotten blasted by the likes of MarketWatch’s Chuck Jaffe and Ritholtz Wealth Management CEO Josh Brown, known for his Reformed Broker blog. Non-traded REITs are insufficiently transparent and raise too many questions, according to Jaffe, while Brown has criticized non-traded BDCs for their high fees, opacity and illiquidity.

What about initial public offerings? Yes, they often deliver big gains, but MarketWatch’s Howard Gold argues that IPOs aren’t what they were a decade ago. A highly developed market for shares of private companies (SecondMarket or SharesPost) now enables insiders to cash out and institutional investors to buy in before a company goes public, and that can take the wind out of an IPO’s sails, Gold says.

(Read about how companies put their executives first, on the next page.)

2. Is the company in which you’re invested putting you first?

Investors plug their hard-earned money into companies’ shares, but all too often, corporate boards do their utmost for their executives, rather than for shareholders.

One recent study found corporate boards reward chief executive officers for making lots of deals. The problem is that the boards are just compensating for deal-making volume, rather than whether any buyout, alliance or spinoff actually proved valuable to the company.

The study — produced by researchers from Drexel University, Northern Illinois University and the University of Texas — builds on the corporate world’s dismal deal-making reputation of creating no wealth for acquiring firms’ shareholders.

The mergers and acquisitions wave of 1995 to 2000, for example, is estimated to have destroyed at least $1 trillion of shareholder wealth, according to the Harvard Business Review.

Another recent study detailed how CEOs can co-opt their boards over time, resulting in less turnover in the top job when a company struggles, as well as unwarranted pay increases. Those pay hikes aren’t linked to performance, says the 2013 study from researchers based at Drexel, Utah and other universities.

Other studies have suggested that when CEOs get paid more, shareholders end up earning less, as a Wall Street Journal column by Jason Zweig explained.

A paper by Harvard Law School professor Lucian Bebchuk found that CEOs who earn more than the average “pay slice” of 35% of a firm’s total compensation for its top five executives significantly underperform their peers. That is because such companies make poor acquisition decisions, reward their CEOs for “luck” when industry conditions improve, fail to hold CEOs accountable for poor performance, and grant options that are timed “opportunistically,” Bebchuk found.

On the other hand, Zweig has noted that shareholders have gotten a little more “say on pay” lately.

(On the following page, read about how investors are always a step behind.)

3. A step behind

If you never scored a big-time college athletic scholarship, but then played pickup games against a friend who had, you know what it’s like to feel more than a few steps behind, with only so much that you can do about it.

That is essentially the plight of retail investors without deep pockets. For example, the Securities and Exchange Commission’s own system for distributing company filings has allowed high-speed trading firms to get access to market-moving documents before other investors.

Three traders last year filed a class-action lawsuit against CME Group Inc. CME, -2.13% , the owner of the world’s largest futures market, accusing it of making secret deals to offer order data to high-frequency traders ahead of other investors.

“It is about a party that is supposed to be a referee taking sides,” says attorney Tamara de Silva, who represents the three traders.

The money lost might be “in the range of the billions,” with farmers, big companies and anyone who relies on the futures market potentially harmed, de Silva said. She said the case has been reassigned to a new judge. She declined to make the three traders available for comment.

However, a Reuters analysis has argued that unless the deals were truly clandestine or discriminatory, the suit is “dubious,” since exchanges are allowed to sell differentiated access to trading data, just as airlines sell first-class and coach tickets. CME Group has blasted the suit as “devoid of any facts supporting the allegations and, even worse, demonstrates a fundamental misunderstanding of how our markets operate.” Similar lawsuits against stock exchanges were recently dismissed.

With the SEC, there have been lags between when paying subscribers — meaning superfast traders, newswires and others — have received filings from an SEC contractor’s direct feed and when the documents became publicly viewable on the government agency’s website, as The Wall Street Journal and others reported in October, citing research from two groups of academics. The SEC said in December that it was starting a process to level the playing field.

Don’t be surprised if you soon hear of another instance of mom-and-pop investors being relegated to the slow lane. After all, nongovernmental sources often sell early peeks at market-moving information to trading firms, even if a few (the University of Michigan, Business Wire) have stopped doing that.

New York Attorney General Eric Schneiderman and federal authorities reportedly have been investigating whether speedsters have an unfair advantage, with Schneiderman describing early looks at market-moving data as “Insider Trading 2.0.”

The only strategy for the average investor is to stick to longer-term holding periods for investments. Randy Frederick, managing director of trading and derivatives at the Schwab Center for Financial Research, says individual investors shouldn’t be trying to compete with high-frequency traders, adding that he thinks the “jury is still out” on how harmful or helpful so-called HFTs are, given that they add liquidity to the market.

(On the next page, “market breakdowns”)

4. Market breakdowns

This era’s tech-heavy investing landscape can look scary, thanks to market breakdowns like the infamous Flash Crash on May 6, 2010, when the Dow industrials plunged nearly 1,000 points in a few minutes only to whipsaw back to previous levels. That nosedive left many individual investors frustrated and disoriented, and it put a spotlight on regulators’ inability to oversee the exploding trading world of superfast computers and hundreds of new ways to execute transactions.

Still, the stock market resumed its multiyear bull run after the Flash Crash, and Schwab’s Frederick notes that breakdowns like that are “very, very rare.”

In addition, it’s worth noting that regulators responded to the Flash Crash by implementing a so-called circuit-breaker program, in which trading is halted on particular stocks or ETFs amid sharp moves. More sophisticated “limit up/limit down” rules have since taken effect, requiring trades to be executed within recent price bands for a security.

Meanwhile, the distribution of stockholdings threatens to amplify concerns over growing inequality.

Stock ownership is highly skewed by wealth and income class, noted Edward Wolff, a finance professor at New York University, in a paper. The top 1% by wealth owned 35% of all stock held by households in 2010, while the top 20% held 91% of the total.

“The main conclusion is that the rise in the stock market certainly doesn’t benefit the average household,” Wolff writes. “The reason is that stock ownership, including 401(k) plans, is highly skewed,” with the average 401(k) amassing only $13,000 in equities in 2010.

Investment Company Institute

It isn’t all doom and gloom. Investors have grown more disillusioned with active managers, particularly after the financial crisis, leading them to put a growing share of money into passive investments.

And more advisers are switching to fee-based business models and putting their clients in lower-cost funds rather than relying on trading commissions. That means lower costs for 401(k)s and mutual funds.

In the end, that all adds up to a “a direct transfer of wealth from the financial-services industry to the pockets of working Americans,” Bullard says, “and that is a great story.”

With reporting by Phil van Doorn.