The active shooter drill is a legacy of Columbine. Prior to that tragedy, in which two classmates killed 13 students in suburban Colorado 19 years ago, kids hid under their desks during earthquake drills, crouched and covered their heads during tornado drills, and filed out to the football field during fire drills.

A decade after that horror, some schools had “turned themselves into near-fortresses,” as CNN put it, and lockdown drills had become routine in many districts. These procedures typically call for teachers to lock their doors, turn off the lights, and huddle students together in a corner, playing cards or reviewing multiplication tables until they’re given the all-clear.

Many schools still hold that kind of lockdown drill, but after the 2012 shooting at Sandy Hook Elementary School in Newtown, Connecticut, where a gunman killed 20 children, six staff members, and himself, this new type of proactive training emerged, along with the new recommendation from the DOE. In place of previous advice that teachers and students shelter in place when under threat, now they were told to employ the three step-method summed up as “run, hide, fight.”

“You can run away from the shooter, seek a secure place where you can hide and/or deny the shooter access, or incapacitate the shooter to survive and protect others from harm,” read a DOE guide released six months after Sandy Hook.

Greg Crane, a former police officer in Texas, has advocated for an alternative to the shelter-in-place approach since 2001, when he founded ALICE Training. The company advocates a five-step procedure — alert, lockdown, inform, counter, and evacuate — to handle violent intruders.

School administrators could follow the lead of a superintendent in Pennsylvania, who equipped every classroom in his district with a five-gallon bucket of rocks to throw at potential intruders.

The lockdown method is too restrictive, Crane says. “I wanted a more options-based program and a more proactive program. We don’t teach them to go sit in a corner and wait to be passive and static.”

Instead, teachers and students are now encouraged to get away from the bad guy if possible and fight back if necessary. ALICE Training teaches students to barricade their classroom doors with a tower of desks, chairs, and cabinets. In the event that they come face-to-face with a shooter, kids are taught to throw things at him. A children’s book published by the company includes a few recommendations: “A shoe from a cubby, a block of wood, or maybe a TV remote. A paperback book, a video game, or even a plastic goat.” Or school administrators could follow the lead of a superintendent in Pennsylvania, who equipped every classroom in his district with a five-gallon bucket of rocks to throw at potential intruders.

“It’s not easy to shoot somebody and hit them,” says Crane, adding that anything that can be done to distract a shooter could result in fewer casualties.

Crane believes reinforcing these strategies requires training with a hint of realism. He’s critical of the drills that look like 1980s-era action movies but says ALICE Training tries to “make it as real as possible without using fear as a motivator.”

“What some call fear, I’m calling maybe a little bit of an adrenaline rush,” Crane elaborates. “We do try to have people experience the adrenaline rush. That’s the whole point of reality training—to get people to understand what they might face.”

Katherine Cowan, communications director for the National Association of School Psychologists (NASP), says there’s no evidence that making people feel afraid “actually gets them to behave more successful in a frightening experience.” There are, however, concrete risks to such a strategy. “The challenge with trying to simulate a frightening experience is you don’t know what the risk factors are for the people involved,” Cowan points out. “If you have someone who has an anxiety around these things, to trigger them unnecessarily could cause real harm.”

Simulation-style drills were designed to train law enforcement officers, Cowan says, but “after Newtown, it got expanded to training the whole school community this way. That’s where it started to off the rails.” According to a report from the Giffords Law Center to Prevent Gun Violence, these drills “foster fear and anxiety” and “can intensify the fear of gun violence children already suffer and train them to persistently worry about their own safety.”

Rather than scaring kids by outfitting their chemistry teacher with a ski mask and setting him loose with an unloaded airsoft gun, the NASP says administrators should stick to lockdown drills, which Cowan calls the “gold standard” of securing schools. She acknowledges that lockdowns might not always be the best option for staying safe. Other approaches, such as fleeing and attacking, should be be discussed, she says, but not practiced. “They don’t fill the airplane with smoke and have it drop 10,00 feet in order to show people why it’s important to put on a mask,” Cowan says.

“Live training with fake weapons is only useful to tactical teams,” says Aric Mutchnik, CEO of global security firm Experior Group. He’s created a new type of active shooter drill that employs little red balls in the place of fake assault rifles. “You have a role-player who comes into the office and pulls the ball out of a bag,” Mutchnick explains. “The front-office secretary sees it and says, ‘Oh, no! The red ball. Am I dead?’”

From there, a conversation begins. If the secretary says she would push her panic button upon encountering an intruder, Mutchnick might have her push it right then. “A lot of times, they’re afraid to press the button. They think if they press it, it’s going to wake up the president. Getting them comfortable with that button is critical,” he says.

Mutchnick’s business in schools is modest at the moment, thanks to entrenched bureaucracy and a deference to law enforcement, he says. But a growing backlash against active shooter drills could change that. Parents, such as the one who launched a petition to stop a drill from including a noise “similar to that of gunfire, but created by clapping two pieces of wood together,” have complained about the hyperrealism. Teachers have sued after suffering hearing loss from blanks fired during drills. And administrators have been formally reprimanded for springing active shooter drills on teachers and students without warning.

“They thought someone was in their school attacking them,” April Sullivan, whose daughter’s middle school held a surprise active shooter drill in May, told the Richmond Times-Dispatch. “My daughter was traumatized. She literally thought she was going to die.”

Some administrators responsible for these drills go to such great lengths to replicate the horror of an actual shooting that it’s hard to imagine they’re not having a little fun with it.

Ronak Shah, a middle school teacher in Indianapolis, recalled participating in one staff-only training in which a man burst into the school wearing a bloody hockey mask and carrying a fake assault rifle. A first-year teacher, who was forced into the demonstration, struggled to respond and was berated by officers. “It was dumb and pointless,” Shah says.