An MRAP manufactured by Navistar International. Nearly 8,000 have been produced. (david_axe / flickr)

Last summer, Secretary of Defense Robert Gates said that the Pentagon's Joint Program Office for Mine-Resistant Ambush Protected Vehicles (JPO-MRAP), a $45 billion program to design, manufacture, and deploy 27,000 heavily protected vehicles into Iraq and Afghanistan, had saved "thousands and thousands of lives." The Joint Program Office drilled down a more specific figure: MRAPs, as they're known in military jargon, saved the lives of 40,000 troops in Iraq and Afghanistan. In other words, without MRAPs, the number of combat deaths from those wars would be comparable to the number killed in Korea or Vietnam.

The Joint Program Office put a specific number on what was becoming the conventional wisdom on the new military machinery. Over the past few years, media outlets, including the Boston Herald and USA Today, repeatedly reported on the life-saving power of MRAPs. The Pentagon underscored the news with its own official releases. Purchases of MRAPs have declined recently due to the withdrawal of forces from Iraq and Afghanistan, but the Defense Department continues to focus on the vehicle -- it recently awarded a contract for more MRAPs, with the authority to purchase another 5,244 of them in the future. Just last week, an opinion piece in The Wall Street Journal argued, "the success the Army and Marines have had with [the MRAP] shows what happens when the Pentagon throws out the bureaucratic rule book and takes on a more World War II-style business model."

But data from the battlefield does not support the claims that MRAPs are highly effective in decreasing the number of U.S. causalities. We recently conducted a study using For Official Use Only (FOUO) Pentagon data, which the Defense Department provided in response to a research request. We found that, relative to light and unprotected tactical wheeled vehicles, those with "medium" amounts of armor plating and mine protection were highly effective at reducing the fatalities in units exposed to heavy combat. For infantry units, one life was saved for every seven medium vehicles purchased, at a total cost of around $1 million to $2 million per life saved. However, tactical wheeled vehicles with "heavy" amounts of protection, such as the MRAP (which has higher quality armor and a V-shaped hull designed to improve resistance to IEDs), did not save more lives than medium armored vehicles did, despite their cost of $600,000 apiece -- roughly three times as much as the medium-protected vehicles.

According to the data supplied by the Pentagon, the MRAPs were first issued to units in areas that experienced especially heavy combat. Later, they were deployed to some units involved in lighter combat. Because those units who got MRAPs faced more violence to begin with than those that never did, a simple comparison of the two groups' fatality rates would understate the effectiveness of the vehicles. Yet before-and-after comparisons for specific units would have been problematic, too, because the combat environment was changing and IED attacks decreased substantially at the same time as the vehicle rollouts.

So we compared the change in fatality rates between units that received the new vehicles to the change in fatality rates in nearby units that faced similar baseline levels of violence but did not receive the vehicles. We found that the heavily protected vehicles were no more effective at reducing casualties than the medium armored vehicles. While the heavier vehicles are safer in principle, they are bulky and lack maneuverability, and they were introduced at a relatively calm time in the conflict, when there were few deaths for them to prevent.

In our study, we also examined two major policy shifts that occurred in a large number of Army units. In the first, the Army provided tactical wheeled vehicles with medium amounts of protection to all deployed units that were previously using unprotected vehicles. Those medium-protected vehicles reduced fatalities by about one fatality per seven vehicles. In the second, the Army provided vehicles with heavy amounts of protection to replace the medium-protected vehicles. But adding armor did not appreciably reduce fatalities. In part, the heavily armored vehicles were introduced at the wrong time (when IEDs were declining), and in part, they went to units that did not see heavy combat. Additionally, some units may have been reluctant to use bulky, heavily protected vehicles like MRAPs because they knock down power lines, have trouble on dirt and mud roads, and inhibit interaction with the locals.

There is no reason to suspect that Gates meant to mislead anyone by claiming that MRAPs had saved "thousands of lives" over the years. He was likely just basing his arguments on faulty information. In assessing the effectiveness of the MRAP program, the Joint Program Office made significant miscalculations.

Why did the Joint Program Office get the numbers wrong? Our analysis suggests that the office added up the number of enemy-initiated attacks in which MRAPs were involved, added up the number of troops who were in those MRAPs, and counted each one as a life saved. Doing so assumes that if, for instance, the Army used up-armored Humvees rather than MRAPs, every attack on a vehicle would have resulted in the death of everyone inside. This premise is not reasonable. For many months of the Iraq War, U.S. forces encountered 60 or more enemy-initiated attacks per day. The vast majority involved zero fatalities and zero injuries -- before and after the introduction of the MRAP. In addition, the Joint Program Office study apparently did not control for factors such as unit type or the intensity of the combat environment.

As Secretary of Defense Leon Panetta undertakes a round of budget cuts and restructuring, many in the defense industry are debating whether future tactical wheeled vehicles should look more like up-armored Humvees or MRAPs (see, for example, here, here, and here).

What does this mean? For most units, tactical wheeled vehicles with medium amounts of protection are just as effective as heavily protected vehicles at reducing casualties. And they are a fraction of the cost. It makes little sense to outfit administrative and support units with MRAPs that cost $600,000 each when they are mainly confined to bases and experience little combat, as is now the case. By contrast, relatively unprotected vehicles like softskin Humvees cost only $50,000 each, and up-armored Humvees with medium amounts of protection cost $170,000 each.

Accordingly, the purchase of MRAPs should be restricted to the relatively few units that are involved in intense combat, if at all. Only 2.5 percent of the 500-person units in our data experienced three or more deaths, on average, over the ten months that they appeared in the sample. Nearly 80 percent of these units suffered zero deaths. There may be a case for supplying MRAPs to some of these units in a more lethal situation, but it does not make sense for the Defense Department to purchase MRAPs in large numbers.

In the last few years, the government spent $45 billion on MRAPs. As a reference, this is relatively comparable to the entire $56 billion budget for the Department of Homeland Security. Now, with a new era of budget cutting dawning on the Pentagon, every dollar that the United States spends on MRAPs is one fewer dollar that it can spend on body armor, up-armored Humvees, aerial drones, or other technologies that more effectively help ensure the safety and security of U.S. military personnel and civilians. And buying a large amount of extremely expensive vehicles that the troops do not need would be a waste.