The digital clock in the foyer of Thames House, the headquarters of MI5, is a constant reminder to visitors and staff of the security service's main priority over the coming weeks.

At first it was set to count down to zero, to coincide with the arrival in the UK of the Olympic torch in May. It has now been reset to a date nearer the start of the Games in London next month.

Leave has been deferred so the agency has a full complement of staff. Some non-essential duties, such as training, are likely to be suspended so there are extra bodies on hand during the busiest periods of the summer.

The Olympics has already proved a huge test for MI5, just as it has for Scotland Yard's anti-terrorist branch.

The threat to the UK is more diffuse now than four or five years ago, which reflects the disintegration of al-Qaida in Pakistan and Afghanistan, and the scattering of extremists to places such as Yemen and, to a degree, Somalia.

With less infrastucture to rely upon, there has been a growth in the phenomenon of the "lone wolf" terrorist – homegrown self-starters, some of whom have researched how to become a terrorist by reading Inspire, the online magazine published by al-Qaida in the Arabian Peninsula (AQAP). Inspire has essentially told its followers to get on with it, rather than waiting to be asked or trained.

The loners are far harder to identify, but the police and MI5 are also aware of hundreds of networks in the UK and abroad, a small number of which pose a credible, if not immediate, terrorist threat. Some of them have been trying hard to launch an attack here over the last five years without success, and the question facing investigators is whether they can realistically try any harder for the Olympics, when security will be far tighter than usual.

Security, that is, around the Olympic village and at the venues. If there is a group out there plotting an attack, or a loner with a gun considering a rampage, they would not have to pick a sporting arena to disrupt the Games – an attack anywhere in the UK would have a huge impact.

Liaising with a myriad of other agencies across the world has been a big task: keeping them up to date with security plans in the UK, reassuring those with extra concerns – with the US and Israel thought to be chief among them – and cajoling others who should have them. Giving advice to the Home Office about who can be given security accreditation for London 2012 has involved sifting through 500,000 domestic and foreign applications – another monumental effort.

MI5 has been in a period of expansion and transition, which has coincided with, and to some extent been accelerated by, the need to protect the Games.

The security service has not recruited extra people specifically for the event because it was growing anyway after the 9/11 and 7 July attacks. Its staff of more than 3,500 means it has doubled in size in a decade, and numbers will be supplemented further with help from MI6, the secret intelligence service, and GCHQ.

The Guardian understands that MI5 has not set up a specific Olympics unit because it believes the systems it has in place are robust enough, and that it would be wrong to draw a distinction between terrorism and Olympic terrorism. Improvements in investigating methods and updates to IT systems are said to have made a real difference.

Scotland Yard and MI5 remain hopeful that the likelihood of an atrocity remains low, but as the weeks pass, they acknowledge that attitudes to risk will inevitably change. Managing these "collapsing timescales", as investigators call them, are likely to test officers to the limit, even if no specific plots are found.

In the days before the Olympics start, their teams are likely to be deluged with extra intelligence from foreign agencies, who will feel less inclined to vet potential information before it is passed on. The network of MI5 agents in the UK may also start to provide more intelligence too, for the same understandable reasons. The front end of this intelligence funnel has been expanded to cope with the extra material, with all the usual criteria applied; is the intelligence specific, is it credible, who is the source, and just as importantly – is there anything that can be done with it?

This "triage" process will help to distil the reports, and if there is any lingering concern, the police, MI5 – and potentially the Crown Prosecution Service – will have difficult judgments to make.Scotland Yard and the security agencies can convene executive liaison groups to discuss the most serious potential threats.

The questions they will have to ask include, can we let this situation run any longer? What risks are we taking if we do? Should we disrupt and do we have enough evidence to arrest and charge?

Nobody is talking openly about them, but it is believed the number of the most serious potential threats does not appear to have increased in the last few months. But some still exist.

Over the last 10 months, the police have made a series of counter-terrorism arrests in places such as Luton and Birmingham, and more are likely to follow. The threat from international terrorism is currently rated as substantial – the third highest of five levels. This is set by the Joint Terrorism Analysis Centre, based within Thames House, though it is independent from MI5. When planning for the Olympics, it was presumed the level would be at the second highest available – severe.

If the threat level rises in the coming weeks, it will not be to keep people on their toes, sources say. It will go up because credible intelligence has emerged that is causing investigators immediate concern. That does not mean there is a specific plot afoot; but it might mean, with time running out, and risks to be managed, the police will make an early move. The police expect bomb hoaxes and stunts this summer, which will cause irritation. But if that is the sum of the disruption to the Games, then no one at Thames House or at New Scotland Yard will be too discomfited.