Europe, Canada, USA, Australia, and others are now running training exercises to prepare for the outbreak of cyberwar. Locked Shields is the largest simulation and we take you inside.

Berylia is under attack. Again.

The island nation, located somewhere in the cold waters of the Atlantic Ocean, relies on its state-of-the-art drone industry for a large part of its income. But recently its drone research labs have come under cyber attack from unknown assailants, forcing Berylia to deploy rapid-reaction teams of security experts to its labs, under orders to find out what's happening, and to stop the attacks as quickly as possible.

Over two hectic days, the teams will have to battle against mounting attacks on their systems, hijacking of their drones, and questions from a sometimes hostile press.

And it's not the first time Berylia has come under attack: strangely these cyber onslaughts happen every year at around the same time. And these incursions won't be the last time the country comes under attack either, because the fictional drone-building country is the setting for the NATO annual cyber defence wargame, Locked Shields.

The exercise is run from Estonia by NATO's cyberwarfare think tank, the Cooperative Cyber Defence Centre of Excellence (CCD COE). The annual event, which has been running since 2010, aims to train the security experts who protect national IT systems on a daily basis. While the exact scenario changes every year, the setting--the embattled Berylia--remains the same, and arch-rival Crimsonia often makes an appearance too.

Berylia might be a fictional state, but Estonia itself has first hand experience of these sort of digital attacks: back in 2007 its banks and government systems suffered weeks of disruption from hackers after Estonian authorities proposed moving a Soviet war memorial. Russia denied any involvement in the attacks, but the incident accelerated plans for the formation of the NATO's cyber think tank, located in the Estonian capital, Tallinn.

This year Locked Shields saw more than 1,700 attack carried out against 1,500 virtualised systems being protected by 20 teams, which separately had to defend online services and industrial control systems against real malware and digital attacks.

The wargame pits 20 'blue team' sets of defenders from NATO's member states, against a 'red team' of attackers which attempt to disrupt their networks. A separate 'white team' of experts runs the game systems. In total, the exercise involves around 550 people across 26 nationalities, 250 of which are the core planning team in Tallinn, where the main action takes place over a two-day period.

SEE: NATO: 'New realities' make internet a potential front line in conflict (ZDNet)

It's not the only big cyber wargame. The US runs its own 'Cyber Guard' event every year, which this year saw around 1,000 players from various government agencies. Those taking part included the UK, Canada, and Australia, all dealing with a fictional attack on an oil refinery, power grids, and ports, while the Bank of England has overseen 'Waking Shark' exercises across the banks in London. However, Locked Shields describes itself as the largest international technical cyber defence exercise.

All the Locked Shields teams get the same mission briefing, and the same set of virtual systems to defend. While the game is run from Estonia by NATO's Cooperative Cyber Defence Centre of Excellence (CCD COE), most teams log-in remotely from their own countries. The teams are playing simultaneously but separately, so it is in some respects 20 games at once, although the teams are allowed to share some information.

In the scenario, the teams are playing as a rapid reaction team that has just been dropped into a drone research lab. That means when the game starts, they don't even know precisely what systems they have to defend, and whether their adversary has already managed to breach any.

Even the technical information they are given about the systems they have been called in to protect is--as it would be in real life--shoddy and possibly incorrect, making it even harder for the teams to prepare their defences.

Image: Hans-Toomas Saarest

"We are trying to use hacking scenarios and attack scenarios that are taken from real life, so we are not playing on an abstract simulation, we are actually using the same operating systems that would be encountered in real life," Dr Rain Ottis, Locked Shields 2016 scenario master, said.

"We want to see how they handle themselves as a team in a situation where there's lots of fog of war, where you do not have full visibility of the scenario of the things that are happening to you," he said.

Over the course of the exercise things only get worse. Not only do the teams have to deal with incoming attacks, they also have to deal with getting blamed for attacks coming from their networks. "It is as realistic as we can make it," said Ottis.

The teams of defenders--each of around a dozen people--have to protect around 2,000 machines making up a realistic representation of what a business network would look like. The services the blue teams have to maintain range from websites, email, and online shopping services, to various kinds of industrial control systems.

The aim is to put constant pressure on the defending teams, to test them with the sort of full-scale cyber attack that hardened security professionals would hope to never experience in real life.

SEE: The new art of war: How trolls, hackers and spies are rewriting the rules of conflict

"We have absolutely everything in there, we have Windows 7, 8, 10, we have Apple OS X, we brought in most of the Linux versions, so what we want to do is have a wide spectrum of operating systems. Everything you can imagine in a regular office, all the software and hardware, we try to simulate that and show that in some way they can be vulnerable," said Aare Reintam, CCD COE's technical exercise director.

"We want to show them everything you have in the environment can be a target or a jumping point into your internal networks." Aare Reintam, CCD COE technical exercise director

"We want to show them everything you have in the environment can be a target or a jumping point into your internal networks," he said.

That means that everything from smartphones to humble printers could be a target. "We want to express that absolutely everything that you have in the network can be a target, that you have to defend everything. Attackers have to find only one thing to attack," he said.

As such, teams don't just have to protect standard PCs or servers, the Internet of Things is part of the security threat too. In the scenario, the teams are protecting a drone research lab, so one of the challenges they are faced with is keeping control of the command and control system for the drones--and regaining control of the drones if it's lost.

Image: Hans-Toomas Saarest

Perhaps one of the more unexpected systems they need to protect is an industrial command and control system. The one that runs the cooling in their own server room. If the teams lose control of that, then their mysterious enemies can turn up the heat, and shut their servers down (to add a little drama to the proceedings when this happens sparks shoot out of the server room simulation board).

The teams respond to the challenges differently, and one tempting option of course when faced with an overwhelming cyberattack is to pull the plug--to protect the systems by taking them offline. But that would be to miss the point: teams must be able to protect the systems while keeping them up and running, even if they have to prioritise.

For Reintam, this is one of the keys to the event: "We are teaching them how to protect our lifestyle. We have to make sure that the lifestyle that we are used to, that you wake up in the morning and you turn on your lights, that you turn on the water and can make yourself a coffee, that you can browse the news with your coffee... you have to pay attention to every aspect of the ecosystem and you have to protect it."

The game wouldn't get very far without the red team, which aims to create that fog of war that surrounds the defending teams. It has around 60 members to "entertain" the defending blue team, said Mehis Hakkaja, head of the red team and CEO of Clarified Security. The red team uses attack methods that are out in the wild to make attacks as realistic as possible, although still ones that can be defended against.

Download this article as a PDF in magazine format (free to ZDNet and TechRepublic members).

Even though the red team knows most blue team systems and vulnerabilities beforehand and even has pre-planted backdoors, the situation changes rapidly as soon as the exercise starts, he said: some of the attacks are based on cybersecurity basics like missing patches but can rapidly accelerate to attacks on complex industrial control systems.The red team can pretend to be various typical hacker groups--from stealthy 'advanced persistent threat' actors to noisier and apparently less skilled hacktivists--or perhaps both at the same time, depending on the scenario. The game plan changes depending on how well the teams respond. The attackers will attempt to do things like steal documents which are then leaked to the in-game media, but if the teams managed to thwart that heist then the game goes in another direction instead.

Playing through such a variety of attacks and threat actors from various angles allows the red team and organisers to evaluate the blue teams on their ability to notice and respond, whether their initial defensive plan worked, and whether they managed to retain control and a sufficient situational overview.

"Having a good initial defence strategy is good, but ability to adjust it on-the-fly is even more important." Mehis Hakkaja, CEO of Clarified Security and head of the red team

"Having a good initial defence strategy is good, but ability to adjust it on-the-fly is even more important," Hakkaja said, as it seeing the bigger picture, "because just blocking and blindly trying to apply defences, or only seeing some attack indications only gets you so far."

As well as the technical aspects of the game, the teams are also tested on their understanding of the legal issues involved with protecting against the attacks, how they deal with the press, and how well they report back to their fictional commanders or political leaders.

In the media element of the game, the teams for example have to be able to explain their actions and put across their point of view accurately, even when being questioned by hostile journalists who are trying to trick the teams into saying too much or saying the wrong thing, all of which plays out on the in-game news site.

Another element tested is around legal issues. The legal picture around hacking, and cyberwarfare in particular, is often unclear, so the teams have to do everything they can to ensure that they are behaving legally.

Image: Hans-Toomas Saarest

For example, the legal framework used during armed conflict is different to those used in standard policing, so working out whether a cyber incident has risen to the level of an armed conflict is a key factor, something that is hard for defenders to work out when many of these attacks are stealthy and anonymous. Malware doesn't wear a uniform or carry a flag.

During the exercise, the legal advisors on the team are tested, often in coordination with the other events in the game: for example, being asked to give military commanders advice on their options when dealing with hacked drones.

SEE: Cyberwar, out of the shadows (Q&A) (CNET)

"In every military operation the idea is to get the commander the options to chose from, and each of those option need to be assessed by a lawyer to say what legal issues do they raise, is it lawful in the first place, which is the best option from a legal perspective," explains Dr. Heather Harrison Dinniss, head of the Locked Shields legal team and senior lecturer in International Law at the Swedish Defence University.

It's only in the last few years--with the publication of documents like the Tallinn Manual which looks at how international law applies to cyberwarfare--has the legal framework around cyberwarfare has become clearer.

"The difficulty when you are dealing with cyber, of course, is you don't necessarily know who it is that is launching the attack," Harrison Dinniss said. "Cyber makes that assessment more difficult."

"There's a much greater acceptance now that the law applies," she added, although there are still things that are uncertain: for example, while it's generally agreed that a serious cyber attack could be considered the equivalent of an armed attack, there's less agreement about how to treat less physically destructive attacks.

"There are still interpretation issues, something that's still up in the air is what do we do about data-only attacks," she said. We're talking about ones that don't cause any physical damage but wipe computer systems, like the attack on Saudi Aramco in 2012 which wiped more than 30,000 devices.

"There is still a question of how do we treat that because there is no physical harm. What do you do when they wipe the computers and make them unusable. Is that enough? Is that a use of force? There's still significant disagreement on [that]," she said.

Teams also have to make sure they do the paperwork.

"We do want them to be able to write human-readable reports about what is going on, something they could send to a manager or a government minister--so condense what they know into something that a non-tech expert can understand, because we have seen time and again that this is a weak spot in the cybersecurity community. We like the lingo that we use and it's sometimes why the message gets lost, and we train for that," said scenario master Ottis.

"When under intense pressure, network security professionals have to monitor the environment, consider social, political, and legal consequences as well as keep ahead of the constant technical challenges." Thomas Svensson, inject master of Locked Shields 2016

The exercise puts a lot of emphasis on team communication, team leadership, and delegation. So what makes a good cyber defence team?

The best teams tend to have done some preparation by thinking through the skills and tools that they will need. Those teams typically figure out who is taking which role quickly, too, so they don't have to worry about who is looking after which systems when the action begins.

Winning teams try to understand the battlefield, predict what their attackers are going to do next, and try to be ready for it, said Ottis.

"We like to see where you are trying to figure out the battlefield, know yourself, know your adversary, and make your plan based on that," Ottis added. "Figure out where you need sensors, which service require more manual monitoring, and which ones you can leave on the back burner. We are talking about being proactive within the network that you have."

Head of the red team Hakkaja makes a similar point: "To see, understand, and communicate the big picture, not being lost in the small technical pieces, is probably the hardest for techies. Large scale cyber exercises like Locked Shields provide a unique opportunity for blue teams to be in such rapidly evolving situations where they rarely are in their daily job as a team."

However, there's one thing that teams can't do, and that is strike back against their adversaries. "This is a strictly defensive exercise so we want them to defend what they have, we want them we want them to cooperate if it makes sense, we want them to keep communications up with the rest of the world and with their higher command. But we do not want them to go on the offensive because that has very serious legal repercussions," said Ottis.

Image: Hans-Toomas Saarest

The team from Slovakia won this year's event at the end of April, closely followed by the NATO Computer Incident Response Capability (NCIRC) team from NATO and Finland, which won last year. The Slovakia team scored highest in the media challenges of the exercise and Germany came out on top of the forensic game, while NCIRC did the best in providing legal analysis, and the Czech Republic won scenario challenges.

"When under intense pressure, network security professionals have to monitor the environment, consider social, political, and legal consequences as well as keep ahead of the constant technical challenges," said Thomas Svensson, inject master of Locked Shields 2016.

Technical exercise director Reintam said there is huge demand for the exercise, reflecting how many countries in NATO are increasingly worried about cyber defence, especially the Baltic states. Worried about Russian cyber attacks, Estonia has even been discussing backing-up vast amounts of public data, from birth records to property deeds, in a secure location outside of the country.

As such, NATO has been taking cyberwarfare increasingly seriously in recent years, first making it clear that a serious cyber attack could trigger its collective defense clause and more recently defining cyberspace as a an operational domain--that is, a likely battlefield.

However, many members lack the trained staff to recognise or deal with a serious cyber attack on their critical national infrastructure. Events like Locked Shields are aimed at encouraging members to take their digital defences more seriously, and perhaps also to show potential aggressors that NATO takes the threat seriously, too.

Right now, all is quiet again in Berylia. But perhaps for not too much longer.

Download this article as a PDF in magazine format (free to ZDNet and TechRepublic members) and read our special report Cyberwar and the Future of Cybersecurity.

Subscribe to TechRepublic's Cover Stories newsletter and never miss one of our in-depth, up-close feature stories. Previous topics include NASA's VR training for astronauts, the remarkable odyssey of Apple's first employee, and the females who broke Hitler's codes in World War II. Subscribe See all TechRepublic Newsletters

The photo at the top of this article was taken by Hans-Toomas Saarest.