Poorly constructed and strict immigration policy is one of the biggest threats to sharemarket returns, say some of the country's biggest money managers.

An expert panel at the Stockbrokers and Financial Advisers Conference in Sydney on Thursday agreed Australian investors could face deteriorating returns thanks to short-sighted policy that curbed population growth.

Since 2000, Australia's population has lifted by more than 19 per cent, or about 4.6 million people. This growth has translated into a demand for services, infrastructure, housing, education and healthcare. Credit:Philip Gostelow

"One of the key drivers of good returns in Australian stocks is, and has been, our population growth," said Paul Taylor, head of the Australian equities fund at Fidelity Investments.

"I understand there's a lot of tension around this in the community but, if we did cut back on immigration or stop immigration, that would be a very, very serious threat to stock market returns."