MI5’s deputy head will take the top job at the spy agency next month promising a sharper focus on China and to work more closely with the private sector in harnessing artificial intelligence in tackling hostile state and terrorist activity.

Ken McCallum, a career MI5 officer, has been the agency’s deputy director general since April 2017 and was seen by insiders as the heir apparent at an organisation that prides itself on internal appointments to its leading position.

The Glaswegian is the youngest ever boss of MI5, although the organisation will only say he is “in his 40s” – and replaces Sir Andrew Parker, who had been due to step down in April after seven years as the director general in charge of the the UK’s domestic security service.

His appointment was announced by the home secretary, Priti Patel, on Monday.

“MI5’s purpose is hugely motivating,” McCallum said. “Our people – with our partners – strive to keep the country safe, and they always want to go the extra mile. Having devoted my working life to that team effort, it is a huge privilege now to be asked to lead it as director general.”

Insiders said that McCallum wanted to be clearer about the threat posed by China – particularly in terms of industrial espionage and cyberwarfare – in the belief that the level of spying by Bejiing in the UK was not appreciated more widely.

But the agency recognises that its concerns about China, which predate the coronavirus crisis by many months, also need to be set against the fact that the vast country also remains an important economic partner for the UK.

MI5 is expected to continue to support the decision to allow Huawei to supply 5G mobile phone equipment, even if highlighting other threats from China could provide further ammunition to Bejing’s critics on the Conservative backbenches, who are threatening to try to block the Chinese company’s involvement.

McCallum also hopes to work more closely with technology companies to try to better exploit advances in technology at a time when the agency also remains concerned about the rise of hard-to-crack end-to-end encryption.

The spy agency believes it no longer has the internal capability to develop what it needs in fields such as artificial intelligence and data analysis, while McCallum will continue to demand that spy agencies have lawful access to secure messaging services.

At one point, around a decade ago, McCallum was responsible for MI5’s cyber activities, when the subject was less fashionable in intelligence circles. It was a period that helped shape his interest in China and working with the technology industry.

As deputy director general, McCallum was responsible for the agency’s operational work during a period when there were terrorist attacks in Manchester and London, and for MI5’s response to the attempted assassination of Sergei Skripal by Russia.

McCallum has 25 years experience within MI5, spending the first 10 years of his career working on Northern Ireland, focusing on terrorism and the development of the peace process around the time of the 1998 Good Friday agreement.

He was asked to take charge of counter-terror investigations and risk management relating to the London 2012 Olympics, before being promoted to become director general, strategy in 2015, responsible for MI5’s relationships with its sister intelligence agencies GCHQ and MI6.

McCallum will become the 18th director general of MI5 since its foundation in 1909, although its leaders have only been publicly avowed since 1993. Modern agency chiefs serve fixed periods of office, preventing them from becoming entrenched at the top of the agencies they run.

Parker, 57, had his term extended despite a difficult period in 2017 when the UK was hit by a spate of terrorist attacks; after the attack at London Bridge, the agency admitted that the ringleader, Khuram Butt, had been on its radar but the signs that the Islamist terrorist was planning the attack that killed eight with two associates were missed.

Fresh concerns also emerged this winter when Usman Khan, a man convicted of terror offences and released on licence, killed two in an attack also near London Bridge, prompting an emergency tightening of terror sentencing amid concerns that people were continuing to be radicalised in prison.

But the military defeat of Islamic State in Syria and the death of its leader Abu-Bakr al-Baghdadi has given some cautious grounds for optimism. The terror threat was reduced from severe to substantial last November.