The departing Secretary of Defence Dennis Richardson has accused China of spying and exerting "unreasonable" influence over Chinese communities and media in Australia.

In a farewell speech to the National Press Club, the one-time ASIO boss has argued Australia should build relations with China, but cautioned there are complexities.

"It is no secret that China is very active in intelligence activities directed against us — and it is more than cyber," Mr Richardson said.

"That is no reason to engage in knee jerk anti-China decision-making, or to avoid seeking to build a stronger relationship with China. It is simply the world in which we live.

"The Chinese Government keeps a watchful eye inside Australian Chinese communities and effectively controls some Chinese language media in Australia."

His warnings follow similar concerns raised by former diplomat Chen Yonglin, who last year warned the number of Chinese spies and agents working in Australia was growing.

Sorry, this video has expired How many spies does China have in Australia?

Mr Richardson is considered one of Australia's most respected public servants and has served as director-general of ASIO, ambassador to Washington and secretary of the Departments of Foreign Affairs and Defence.

Known for his frank and fearless advice across 48 years, the public service veteran issued some blunt advice to modern bureaucrats who complain about being restricted by their political masters.

"I think public servants who say that are simply cowards," Mr Richardson told the National Press Club.

"I have never worked for a prime minister or a minister with whom I have not been able to talk frankly.

"The odd minister might not like that for 24 hours or 36 hours, but ministers — politicians generally — are very skilled at moving on."