Jonathan Millican is a first-year university student from Harrogate in North Yorkshire. He says he doesn't think of himself as a "stereotypical geek", but having been crowned champion in Britain's Cyber Security Challenge, the 19-year-old is bound to take some stick from his undergraduate friends at Cambridge.

The competition is not well known, but it is well contested. About 4,000 people applied to take part this year, hundreds were seen by judges, and 30 were selected for the final in Bristol on 10 March.

After a day of fighting off hackers and identifying viruses in a series of simulations, Millican triumphed, giving him legitimate claim to be the brightest young computer whiz in the UK.

And though he may not recognise it yet, Millican has become a small player in a global game. There is a dotted line that links him to an ideological battle over the future of the internet, and the ways states will use it to prosecute conflicts in the 21st century.

The remaining cold war superpower, the United States, is slowly squaring up to the emerging behemoth, China, in a sphere in which Beijing has a distinct advantage: cyberspace.

Experts estimate China has as many "cyber jedis" as the US has engineers, and some of them, with backing from the state, have been systematically hacking into and stealing from governments and companies in the west, taking defence secrets, compromising computer systems, and scanning energy and water plants for potential vulnerabilities.

The scale of what has been going on is only now being recognised, and with a discernible sense of panic, the US and the UK are trying to make up lost ground.

One important way of shoring up the west's defences involves recruiting a rival army of computer specialists to defend the systems being attacked.

This is why the UK began the Cyber Security Challenge in 2011, and why Millican and otherparticipants have been discreetly courted by GCHQ, the government's electronic eavesdropping centre, which is on the frontline of this new power struggle.

The explosion in internet use, and the almost complete reliance on computer systems to run and record our daily lives, has opened up endless opportunities for thieves, spies and vandals to exploit the platform.

Though it is still evolving, the push-back has started. The Guardian has spoken to senior officials in the US and UK government, as well as specialists and independent thinktanks in London, Washington and San Francisco, who agree that the west is galvanising itself to adopt a far more aggressive approach to a problem for which there is no precedent. The stakes have suddenly become very high.

Over the past 18 months, there has been a concerted effort to highlight the relentless nature of day-to-day attacks on businesses and government departments. The Obama administration estimates that 60% of small firms that are hacked go broke, and billions of dollars worth of intellectual property have been stolen from industry, including military blueprints from leading defence contractors.

And in the political shadows in Westminster and Washington, they have moved to put cyberspace more formally into the military sphere, so that those responsible for the attacks understand that retaliation is now part of the game.

New military battleground



Though much maligned, Britain's 2010 strategic defence and security review may prove to have been a historic punctuation mark in this process.

The review made the threats from cyberspace a "tier one" priority, because Downing Street considered them a genuine threat to national security.

The US is moving in this direction, too. On 17 January, the head of the US military, General Martin Dempsey, set out a significant change in position. In a 70-page document that was largely ignored and almost completely impenetrable, he said the US intended to treat cyberspace as a military battleground.

"Disrupting the enemy will require the full inclusion of space and cyberspace operations into the traditional air-land-sea battle space … [They have] critical importance for the projection of military force. Arguably, this emergence is the most important and fundamental change … over the past several decades."

The military has long had basic cyber capabilities, such as equipment for jamming signals, but the more sophisticated weapons are seldom spoken of, and rarely used, in part because there has been no formal code of conduct.

This has prevented the US from routinely deploying its most destructive cyberweapons, including during the Libya campaign last year, when the Pentagon gave President Obama the option of disabling Muammar Gaddafi's military computer network with a targeted cyber-attack. The White House decided against it, but the Dempsey doctrine will give the president, and General Keith Alexander, the head of US Cyber Command, more confidence next time.

Officials in the US and the UK privately concede they have been developing a range of new "offensive" cyberweapons – and a rulebook for their use.

"If we know that someone is about to launch a cyber-attack on us, then we will pre-empt it," said one Whitehall official. "We have that capability and we will use it, even if the bad actors are based abroad."

The state department now regards cybersecurity "as a foreign policy priority", and Obama administration officials insist "the laws of conflict apply to cyberspace".

"If there is significant information of a cyber-event, we reserve the right to use tools in our toolbox," said one. "When does a cyber-attack achieve critical level? When one can attribute an attack that deliberately causes loss of life."

Paul Rosenzweig, who spent four years as deputy assistant secretary in the department of homeland security until 2010, is sceptical that a cyber-only war will happen soon. But he added: "We may have cyberwar as part of another war. I would hope and pray and assume that they [China] are as worried about that as we are."

Frank Cilluffo, President George Bush's special assistant for homeland security at the time of the 9/11 attacks, said: "In cyber, we are where the counter-terrorist community was on September 12, 2001.

"I have come to the conclusion that we can no longer firewall our way out of the problem. We need to talk about offensive capabilities to deter bad actors. I don't think that you are going to see warfare without a cyber dimension in the future … that is a given. I think warfare as we think of it today will take on these dimensions."

With a buildup of cyberweaponry on both sides, Russia and China have called for negotiations to start on new treaties to govern what is permissible in the domain.

The Russians, in particular, have favoured arms control-style agreements, and last September Moscow and Beijing formally proposed to the UN a new international code that would standardise behaviour on the internet.

That has been flatly rejected by the UK and the US. They argue arms control treaties won't work because it will be almost impossible to verify the weapons each state has – computer viruses are more easily hidden than nuclear missiles.

And the new international code, the Foreign Office argues, is simply an attempt by Russia and China to stifle free speech on the internet in their own countries.

"It is too late for new formal treaties," said one senior source in the Ministry of Defence. "If we go down that road it will be years before anything emerges. This is China and Russia trying to kick the issue into the long grass."

But the alternative is almost as far-fetched, and perhaps more nebulous. The foreign secretary, William Hague, has been calling for countries to agree a "rules of the road" in cyberspace, with respect for international law, rights to privacy, and protection of intellectual property at their core.

This puts huge emphasis on goodwill between countries and the harmonisation of existing laws to make it easier for investigators to cross international boundaries. It is as unpalatable to China and Russia as their ideas are to the west.

"It's not at a point where I would call it cyberwar yet, but it's close," said Larry Clinton, president of the Internet Security Alliance, which represents a group of multinational companies, including many in the defence and aviation sectors.

"I think we are certainly seeing an arms race with respect to cyber. We did well to get through the nuclear age. We did well with chemical weapons. If we can do as well with cyber, that would be great, but we don't really have a theory; I am not sure what the theory is. We don't have a model set up for how we are going to deal with this."

Private fears



Developing cyberweapons, and a methodology for using them, is only one part of this complex new puzzle.

Though government departments are continually under attack, it is private industry that suffers most from hackers. The frightening scale of the theft of intellectual property, and the potential knock-on effect for fragile economies, underpinned the UK's decision to say it must now be regarded as a genuine threat to national security.

This, in turn, is forcing governments to expand the boundaries of what might trigger a military response to include theft, albeit on a massive scale.

Rosenzweig estimates that 85-90% of the US's digital infrastructure is in private hands. "I am pretty sure it's the same in Europe."

Though it is hard to make calculations, one survey last year commissioned by the Cabinet Office estimated the UK economy lost £27bn to cybertheft in 2010.

In America, they gave up trying to calculate precise values nine years ago, when the number of known "cyber-intrusions" reached 100,000 in a year; one respected Washington thinktank put the cost of cybertheft in the US last year at roughly $100bn (£63bn).

America's biggest companies have spent a similar amount beefing up their cybersecurity in the past five years, but analysts say this hasn't been enough to prevent "significant military losses" involving stealth, nuclear weapon and submarine technology, though none of the companies involved will admit it.

Without giving away details, Shawn Henry, executive assistant director at the FBI, confirmed that military networks and defence contractors had been hit hard by hackers. "A tremendous amount of information has been stolen from those networks by a variety of state actors."

But there is another dimension of cyber-espionage which is, in some ways, more disturbing.

"We know that Russia and China have done the reconnaissance necessary to plan to attack US critical infrastructure," said Jim Lewis, from the Centre for Strategic and International Studies, a Washington thinktank.

Lewis was commissioned by Bush in 2008 to write a cyber strategy for the government, which is still regarded as a benchmark.

"You might think we should put protection of critical infrastructure at a slightly higher level. It is completely vulnerable. It is totally unprotected.

"This isn't made up. I have been doing this for a long time. We know that people have done the reconnaissance, we know that control systems can issue commands to destroy critical infrastructure. We know all this and we have done nothing to defend ourselves … We have been trying for about seven years to deter people and it doesn't work."

Henry admitted his agency was now dealing with thousands of attacks every month. The agency has people in 63 countries specifically to deal with online threats. "We recognise that there are vulnerabilities in infrastructure," he said. "There are thousands of breaches every month across industry and retail infrastructure. We know that the capabilities of foreign states are substantial and we know the type of information that they are targeting."

He added: "We have seen adversaries that have been in networks for many months, or even years in some cases, undetected. They have essentially had free rein over those networks … looking at information that is transiting that network, with the ability not only to review that data, but potentially to change that data. They have complete ability to disrupt that network entirely."

Henry said attacks were becoming much more sophisticated. "Every step that the defence makes, the offence changes its tactics."

Rosenzweig believes this mapping of critical infrastructure – such as energy or water plants – is seen within government as "preparation of the battlefield". It is, he says, China's way of saying: "Don't send the 7th fleet to save Taiwan, or we will take out the electricity supply in Los Angeles".

The US is using the Idaho National Laboratory to run simulations testing the robustness of America's most important computer networks, but these take time.

With so much at stake, the Obama administration is pushing for proper domestic regulation and standards in cybersecurity, but that is being resisted by private companies, even though it may force them to close the gaps that are being exploited.

Three competing bills are currently vying for votes in Congress, including one from the former presidential candidate John McCain, who wants to fend off government oversight, and the prospect of companies being fined – or sued – if their cyber defences don't come up to scratch.

The role of China



Though the arguments are running along party lines, there is no argument about the fundamental problem, and where it is sourced from.

"Anyone who is significant on either side of the aisle is running around with their hair on fire," said Rosenzweig. "The influential voices on both sides are saying it's a problem. It's a real problem and it's a real problem right now. General Keith Alexander [head of US Cyber Command] says he is seeing it, and he's not the sort of guy to make things up."

There is no doubt about the main culprit, says Rosenzweig. "China denies it – but this is one of the bald-faced lies that people get away with because we don't want to face the consequences. China has more computer programmers than the west has engineers.

"Not everyone is a cyber jedi. But if you have 1 million computer programmers, you will find 1,000 jedis. We have a lot of IT professionals but they aren't the same thing; we don't understand the culture."

Dmitri Alperovitch, one of the world's foremost independent cybersecurity analysts, said: "The Chinese clearly have no restraints when it comes to espionage.

"In the US, economic espionage by either private sector or government is prohibited by policy and the Chinese are certainly not constrained by such measures. When it comes to volumes and sheer scale, no one even comes close to them."

The audaciousness of some of the attacks has been astounding. Earlier this month, Nasa's inspector general, Paul Martin, revealed the space agency's Jet Propulsion Laboratory headquarters in Pasadena, California, had been compromised by an attack that appeared to come from China.

The JPL manages 23 spacecraft, including missions to Jupiter, Saturn and Mars, and controls the International Space Station.

In remarkable testimony to Congress, Martin said hackers had "gained full system access" to JPL, allowing them to modify, copy, or delete sensitive files, create new ones, and upload hacking tools to compromise other Nasa systems. In short, they were running the network.

This was only one of 47 cyber-attacks on Nasa last year, 13 of which successfully compromised the agency's firewalls.

Martin said some of the intrusions "may have been sponsored by foreign intelligence services seeking to further their countries' objectives".

There is debate on how effective, and for how long, a cyber-attack from China could knock out an energy supply or communications hub. Larry Clinton said it would not be easy, but it would be foolish to think it was not possible.

"Older technologies tend to be safer than newer technologies. Copper wire is more secure than fibre. And the problem is the interconnections. We don't have nearly the degree of air-gapping that we once did.

"You can get into a weapons system and you won't even know that system is compromised until you set it off and then it comes back and hits you in the face … the sort of attacks that were considered sophisticated six years ago are considered elementary now."

If the threat is that great, and the belief that China is behind it so widely held, why hasn't the US been more robust in condemning Beijing? It's a question the state department refuses to answer. It will not even say if it has used normal diplomatic means – summoning an ambassador or expelling someone from the embassy.

Melissa Hathaway, who was director of the Joint Interagency Cyber Task Force under Bush and was on the National Security Council in the first year of the Obama administration, thinks the reticence is understandable.

"We need to think about our roles and the economic future of the world. What would you like the future of the economy to look like? Quite honestly, right now we are all dependent on China. All of us.

"They have bought a lot of European debt, they have bought a lot of US debt. They are helping to promote world stability right now."

The US has been pursuing another route to the Chinese, reaching out to Beijing using thinktanks as proxies, and engaging them in "cyberwar" games.

It is the only chance the Pentagon and the state department get to sit across the table from their Chinese counterparts, to express their own fears, and to hear those of China.

One hope is that the talks will lead to an equivalent of a "nuclear hotline" from Washington to Beijing, so leaders can talk before a situation gets out of control.

While the US may be pleased it is finally getting its message across, Lewis isn't convinced the Chinese are listening. And he doesn't think they will stop their activity in cyberspace either.

He has been dealing with the Chinese military for years, and says the People's Liberation Army is hostile.

"They see the US as a target. They feel they have justification for their actions. There is a sense that China has been treated unfairly, and so they have a right to catch up. Britain and France may have burned the summer palace, but the US has become the symbol of imperialism. And they think the US is in decline."