Since 1999, I've helped to oversee an investment club – with results, I'm happy to report, that make most mutual funds look inept by comparison. We have beaten most investment professionals, and left diversified portfolios of index funds in the dust.

From 1999 to early 2011, I bought individual stocks for my personal account as well, outperforming the returns of the investment club by a healthy margin.

Yet earlier this year, I sold $700,000 worth of individual stocks to ensure my personal portfolio was 100 per cent in low-cost index funds. Why? It's the realization that a big part of my success was luck. The more I learn about investing, the more I realize how difficult it is to gain a sustainable edge on the market.

Story continues below advertisement

As a stock picker, I used to feel smug. I researched companies more than anyone I knew, ordering at least five (usually 10) years worth of annual reports, and scrutinizing them from back to front (the juicy stuff is always in the back!).

Using Warren Buffett's tenets as a guide, I bought strong businesses with low debt levels, great managers, high returns on capital, predictable business outlooks – and made sure that I always paid a great price.

You might be thinking that I threw in the stock-picking towel after a few of my stocks went south. But that's not the reason.

During research for a book on personal finance, I was struck by how few investors beat the market over a lifetime. History is filled with dozens of "can't miss" fund managers who eventually get spanked by the stock market.

Take Bill Miller. His mutual fund, Legg Mason Value Trust, beat the S&P 500 for 15 straight years, prompting Fortune magazine's Andy Serwer to call Mr. Miller in 2006, "the greatest money manager of our time." But shortly after, Mr. Miller was force-fed large chunks of humble pie, as his coveted fund lost dramatically to the U.S. index.

Even during Mr. Miller's incredible stock market run, my personal returns were outperforming his from 1999 to 2006. And from 2006 to early 2011, my stock holdings continued to beat the market (unlike Mr. Miller's).

Am I that much better than Mr. Miller? I would love to believe it, but in reading interviews with the man, I've been struck by his intelligence. His IQ is off the charts. Plus he has access to more research than I do.

Story continues below advertisement

Before proclaiming myself a financial genius, I decided to look at a bigger picture. I concluded that good fortune had played a large role in my success.

In 2003, I wrote a magazine article, "How To Invest Like Warren Buffett." My picks in that article yielded stellar results – the best being McDonalds, then trading at $11 per share. It's now $85.

But my 2005 follow-up piece recommended stocks like Pier 1 Imports at $18 per share (now $12), The Gap at $21 per share (now $18) and Trex Co. at $35 (now $25). There were a few winners, but coupling the winners with the losers wouldn't have given the article's followers any advantage over the S&P 500 index .

I didn't buy the stinker stocks above, but I easily could have. I thought they were great stocks, at great prices. For whatever reason, I didn't invest in them. If I had, I couldn't brag about my stellar returns.

Then there's a detailed assessment that I did on General Electric in 2002. With a long history of paying dividends, it had tripled its dividend payout in the short period between 1991 and 2002. It was also viewed at the time as an impenetrable blue chip fortress.

At $38 per share it traded at 26 times earnings – too rich for my taste. But the notes that I scribbled in 2002 reveal that I considered it a steal at $22, a price it would reach the following year. If I had paid $22 for GE shares, I would have lost to the market. Today GE trades at $20 per share. What prevented me from buying GE in 2003 at $21 per share? Luck. That's it.

Story continues below advertisement

People who are lucky enough to beat the markets generally have concentrated portfolios. I had roughly a dozen stocks, so a single wrong move could have cost me dearly.

Yes, my investment results have handily beaten the market over the past dozen years. But I choose to look at those results with a broad lens, rather than concluding what I wanted to see with a pair of rose-coloured glasses. Twelve years is a stock market blip. Mountains of evidence describe the futility of trying to beat index funds over a lifetime with actively managed portfolios.

Only a handful of geniuses like Mr. Buffett have ever managed to pull off the feat. I'm definitely not a genius. But I am smart enough to recognize my limitations.

------------------------

Andrew Hallam is a Canadian who teaches English and personal finance at Singapore American School. He is the author of Millionaire Teacher: The Nine Rules of Wealth You Should Have Learned in School. This is the third in a series of columns based on his book.

Special to The Globe and Mail