Big internet firms know more about the lives of private individuals than any intelligence agency ever has and that is a dangerous threat to democracy, the former head of the spy agency GCHQ has said.

David Omand was at the centre of British intelligence for decades, beginning his career at GCHQ after he graduated from Cambridge in 1969 and becoming its director from 1996-97.

At the Cheltenham literature festival on Tuesday, where he was promoting his new book exploring the ethics of spying, he suggested that people should be more concerned about what companies such as Facebook and Google know about us than the state.

“The big revelation over the last couple of years has been not about government intelligence agencies, it’s been about the private sector. It is about the internet companies knowing more about me, you, everyone in the hall than any intelligence agency ever could or should know about us.

“The Cambridge Analytica scandal, the way in which our personal data which is freely given in return for having an internet free at the point of use so we can do our searches and so on … that information is monetised, it is sold.

“That is the feedstock now for political campaigning where a political party can tell different messages to different groups of people because they already know what an individual’s likely political preferences are.

“This is truly dangerous. I think it’s a major threat to democracy and it’s uncontrollable.”

After GCHQ Omand was permanent secretary at the Home Office and later security and intelligence coordinator in the Cabinet Office before retiring in 2005.

He said he was an outsider now but he saw the operations of GCHQ as being “extraordinarily strictly regulated” and that was as it should be in a democracy.

“The warrants have to be countersigned by a senior judge, you have this army of inspectors, all of that apparatus which is necessary and I approve of. It’s designed to make sure that under any potential future government we’re not going to lapse into authoritarianism.

“But nobody has worked out how to control the private use of our information.”

He said the pioneers of the internet never imagined it could become so dangerous, just as the inventors of the internal combustion engine never imagined it could be used in bank robberies.

“There is nobody in GCHQ monitoring the British population,” he said. “You need to get access to bulk data, a big haystack, in order to find particular things … patterns or IP addresses of terrorists or whatever it might be. But please preserve us from a situation in which we are under surveillance. Surveillance is persistent observation of the population, or a big chunk of the population. Thankfully that doesn’t happen here.”

He said there were rigorous checks on how intelligence agencies operated. His confidence in the independence of senior judges had been boosted, he said, by the recent unanimous supreme court decision against Boris Johnson and the proroguing of parliament.

Was there a file on Johnson, he was asked. “I can honestly say I haven’t seen it and I don’t even know if it exists.”

Cheltenham is the home of GCHQ and Omand was aware that current employees were probably in the audience. “Who works in intelligence?” asked the session’s chair, Oliver Balch, at the beginning. No hands went up. “Who works in intelligence and can’t tell us?” Reassuringly, still none.