[siren wailing] <i>[indistinct radio chatter]</i> <i>[soft dramatic music]</i> TONY: Anybody see what happened? I got about—roughly about 2 to 3 feet away from him, and I could smell the strong odor of intoxicants coming from him. This person was clearly under the influence of alcohol... and clearly should not been driving a motor vehicle. <i>NATALIE: Every night, on roads across America,</i> there are thousands of officers like this. Eyes peeled for signs of drivers who might be drunk. TONY: How much you had to drink tonight? OFFICER: How much did you have to drink tonight? MAN: I stop roughly between four to ten cars a night. NATALIE: More than a million people a year are arrested for drunk driving. - Tell me what you’re looking for. TONY: I’m looking to see anything that’s abnormal. NATALIE: For months, <i>The New York Times</i> has been investing fundamental problems with the technology used to make those arrests. TONY: The first physical test that I do is like an eye exam. It’s very easy. The next test is the walk and turn test. - Do they take a breath test on the side of the road? How does that work? - People who are really close to the .08 limit, I’ll offer them the breath test... And blow into the straw. Just to see exactly where they’re at. Right now, you blew a .123. For me, in order to get a— to roughly about a .08, it’d probably take me about six beers, I believe—six, seven beers. It takes quite a bit in order to get somebody for a .08. - When do—are people taken and tested at the precinct? - Anytime I arrest ‘em for alcohol-related DUI. NATALIE: It’s on this larger Breathalyzer machine... - And blow. NATALIE: That the exact amount of alcohol in the driver’s system is supposed to be measured. These machines are among the most widely used forensic tools in law enforcement. The numbers they spit out influence a driver’s conviction and punishment. In most states, if you refuse the test, you lose your license. TONY: The legal limit is a .08, right? MAN: Yes, sir. TONY: All right. Well, you’re at a .107 and a .109. So you’re still over the legal limit, man. - Yep, yep, makes sense. TONY: Okay. You had too much to drink. - Understood. NATALIE: But our investigation found that tens of thousands of these breath tests have been thrown out by judges in more than a dozen states. What does it mean if this tool, hailed as accurate to the third decimal point, is unreliable? What if this cornerstone of the criminal justice system can’t be trusted? <i>[indistinct radio chatter]</i> <i>[dramatic music]</i> <i>♪ ♪</i> [wood clatters, fire cracking] [phone beeps] <i>WOMAN: Hello, you have reached</i> <i>Kittitas County Misdemeanor Probation.</i> <i>If you are simply calling for your regular check-in—</i> <i>ELECTRONIC VOICE: Wait while I transfer your call.</i> <i>[line trilling]</i> - Hi, this is Cindy Tyler calling in, with no changes. JESSICA: So tell me all the different kinds of driving you’ve done. - Well, I did school bus, dump truck. Oh, I did a party bus for a little bit, and I drove drunk people. I was getting a new job, driving for retirement home. I got the job. I just needed to get the background check, but when they pulled up my background check, it came to disqualify me. JESSICA: Just lead me through the night that this all went down. - Oh, the—the DUI? JESSICA: Mm-hmm. - Well, I went out to get something to eat... And I went to the saloon. I had a dinner and couple glasses of wine. And I pulled out, and I’m driving down the road, stopped at the stop sign, complete stop, and I didn’t know there was a police officer behind me. CINDY: And then I continued. There’s snow and ice on the right-hand side of the road, so I was hugging the center line. And about 100 feet down the road after the stop sign, I see blue lights. He had me, you know, walk the line. Then he gave me the Breathalyzer. OFFICER: And blow nice and easy. - He said it was a .09. - You’re a—you’re a .09— over an .08, let’s put it that way and roughly going up. - It’s like, that can’t be right. It can’t be right. JESSICA: There’s a problem here, though. These portable Breathalyzers are notoriously unreliable. Even mouthwash and gum can throw off the numbers. The results aren’t admissible in court, but they often give officers the grounds to make an arrest. - I was crying. I was crying. - Cindy says she has gastric reflux and a bridge in her mouth, both of which can alter results. - He goes, “No, we’re just gonna go down to the station.” And we get down there, gave me the Breathalyzer again. - Show me the— CINDY: So— - What are we looking at here? - So this is the— this is the breath analysis, .06, and then one is .59, and that’s under the legal limit. - But the officers decided to charge Cindy with a DUI that night anyway. - You know, there’s no reason why I should have gotten this DUI. And that’s been since November, and this is July. And so I’m waiting to go to trial. JESSICA: Across the country, this kind of confusion over DUI testing is growing. These are my colleagues, Stacy Cowley and Natalie Kitroeff. Together, we’re investigating the technology used to convict drunk drivers, who controls it, and how it’s being applied. JASON: DUI is unique as a crime because a person can be punished, can be convicted, can be sent to jail, and be guilty by machine. - Jason Lantz has a story to tell us. It’s one he’s told in court. It starts in 2009. Washington state had just spent nearly 1 million dollars on 60 new Breathalyzer machines, the Alcotest 9510 from the Drager corporation. JASON: Everyone knew the publicly available marketing literature that was available, but we really didn’t know, kind of, its inner workings. You know, how does it really work? And so I started trying to look into— is anybody doing something? - And the goal was to? JASON: It’s a sophisticated computer, and there’s no such thing as a flawless computer. - What you’re saying is people don’t question the machine. JASON: That’s exactly right. And where we decided to start was public records requests, and we were shocked by what we found. We had an email from Fiona Couper, the Washington state toxicologist, saying, we’re gonna throw caution to the wind and deploy the device. - The state toxicologist is saying that they’re just not gonna— - They never independently tested the software of the device. The other thing that was— really piqued our interest and gave us cause for suspicion was the Washington State Patrol was maintaining data, what they called the real-time breath curve data. So for every quarter of a second a person’s blowing into the machine, the machine’s recording their breath test result. NATALIE: Every quarter second— .073, .073, .074, .074, .075, .076, .077. There’s nothing at or above .08. - And then, what was printed on the ticket is a .080. They’re guilty of DUI. - At this point, you suspected that there were issues with this software... JASON: Right. - And with the algorithm... JASON: Right. - That had it rounding up. - That’s what made us want to— okay, now we think there’s smoke here. We wanna device. We want the software. We wanna hire experts. We wanna see, are there problems? Now we wanna actually dig into it. MALE ANNOUNCER: Department of inventions somebody ought to invent. Can mechanization keep intoxicated idiots from killing themselves and innocent people, to boot? JESSICA: The earliest convictions for drunk driving were based entirely on a police offer’s observations. It wasn’t until after Prohibition in the early 1930s that people started experimenting with other ways of keeping drunks off the road. - The Drunkometer a new machine which is a pretty good index of— or a perfect index, is it not, of how drunk you are. JESSICA Basically, vials of solution attached to a balloon. - [chuckles] JESSICA: Alcohol on the breath would trigger a chemical reaction... - How do you feel, Arthur? - Fine. JESSICA: Turning the solution a different color. - And I determined that Art is now at .22. He’s within the intoxicated zone. JESSICA: It was an indirect and rather crude way of measuring alcohol in the blood, but the technique became accepted, and in 1954, we got the first Breathalyzer. - You’re right at a .11 %. JESSICA: Law enforcement agencies quickly embraced the new invention. Smith and Wesson, the firearms company, bought the original patent and then sold it to a German company that was known for making gas masks and scuba gear— Drager. Today, the industry is dominated by Drager and two other companies— CMI and Intoximeters. All of them have faced litigation over the accuracy of their devices. Breathalyzers still rely on the same basic scientific principle, but the results are now determined with the help of a computer algorithm. And it was of these algorithms, the one used in the Drager device in Washington state, that had make Jason Lantz suspicious. So he did what lawyers do. In litigation, he demanded that Drager hand over its code. Then, to make sense of that code, he hired two computer scientists, Robert Walker and Falcon Momot— also known as Falcon Darkstar. That’s when things got weird. - This lockbox shows up, and a key showed up separately, and we had to agree to do the work in a specific place. We had a dedicated, specialized computer, and it wasn’t connected to the Internet. - But to be clear, you guys both deal with highly secure, confidential software, and this, even by those standards, is extreme. - I worked on the most confidential things that Microsoft does, and they don’t have any procedures like this. FALCON: It’s never come up. Nobody has even asked if they can have a camera on us while we worked, to have a security officer watch to see whether we’re going to leak information. It’s just odd. - So did you find issues? Did you find problems? - In the source code, we can’t tell you. FALCON: I can’t tell you. - Wait, why not? - Because Drager threatened to sue us. After we published a preliminary version of our report, they hit us with a cease and desist. - So I have a copy of your— this is the report, right? ROBERT: Legally, I have to ask you to destroy that right now. FALCON: Yeah, that shouldn’t exist. It’s supposed to be gone. [chuckles] NATALIE: I mean, now that it’s here— - I can’t discuss it at all, and I have to ask you to destroy it. - So we can’t talk about any of this, you’re telling me? ROBERT: No. FALCON: No. Unfortunately, no. - Okay. Well, I’m not gonna destroy it. NATALIE: Their report listed significant flaws in the code. Not only did they suspect that the machine sometimes rounded up, but they also determined that accuracy safeguards had been disabled. They even went so far as to say that the 9510 was not a sophisticated scientific instrument and doesn’t adhere to basic standards of measurement. <i>[soft tense music]</i> STACY: What’s the right way to balance this need that the company legitimately has to protect its trade secrets? You know, engineers are human. People make mistakes. The concern we hear from defense bar is saying these are devices on which people’s, you know, criminal convictions are hinging. Shouldn’t we be able to get a better view into how these things work? What’s the answer on that? - I think, ultimately, the answer is where we arrived, which is we really have nothing to hide, but we have something to protect. We’re aware of a situation that we did find. In one case, it was a problem in the source code. And another problem, it was a problem with a setting in the instruments. They were definitely mistakes and problems. And our posture is always, when we find out that something is not right, we aggressively take steps to address it and fix it. STACY: And the pushback we hear from the defense lawyers is them saying, well, we could find more of these things if we had access to the machines and the code. - Yeah, I mean, ultimately, we could extend that all the way to, you know, having them sit with us while we design the code. At some point, we have to draw the line before that analysis becomes paralyzing or disruptive for the business of the customer. STACY: We asked about the calculations Jason Lantz had questioned in Washington state. STACY: One of the questions that I was curious about was when you see the infrared sort of taking its measurements, you can see it kind of getting up towards .08, hitting .08, and then what gets printed on the ticket there is, they say, consistently higher. Like, you’ll see a .083, like in this case. - Right. Yeah, I’m glad you brought that up because that’s a pretty common misconception... STACY: Okay. - About the actual context of those numbers. These are preliminary measurements. They were never intended to tell the story of what the final value was. Those plot data points are pulled from the sensor very early in the measurement process and before any of the scientifically appropriate compensations are taken to account for things like pressure and temperature and the presence of carbon dioxide, and things like that. All those compensations happen later in the algorithm, if you will. STACY: But those compensations are at the heart of the machine’s accuracy, and it’s exactly why the lawyers want more visibility into the machine’s code. STACY: On top on the legal battles over the accuracy of the machines, there’s another problem. What happens when the humans using them are making mistakes? That question led us from Washington state to Washington, D.C. BRYAN: D.C. has multiple policing agencies. But the primary agency is Metropolitan Police Department or MPD. They were using the Intoxilyzer 5000. The problem was—is that the technician who was doing that work at the time, he didn’t under— he wasn’t a scientist. He was just an officer and didn’t really have anybody overseeing the work that he was doing. STACY: Oh, so what point did you start to figure out— like, what year roughly... - Well that... STACY: That the numbers are wrong? - That really was in 2010. STACY: That was the year that a veteran cop named Ilmar Paegle was assigned to the lab. - When Ilmar Paegle came in to take over the system, when he started testing all the machines, they were all wrong, and so he really felt like that information needed to get out. He really felt like that information should be shared with the public. STACY: Four days into his new job, Paegle blew the whistle. The testimony he gave would jeopardize his career. ILMAR: Good afternoon, Mr. Chairman. My name is Ilmar Paegle. I’m here today, testify in support of the bill being considered by the committee. - He knew what the problem was. - On my second day at work last year, I walked into the office and started poking around into the equipment, into the files, and discovered that the instruments had been misprogrammed to over-report breath alcohol tests by approximately 20 % and ranged as high as 40 %. STACY: This had nothing to do with source code. It was pure incompetence. - So a .08 would read as a .1? - Correct, just over that. - Now, the technician that did this, what he was doing was he was taking a solution— and I’m simplifying this a little bit— but he would tell the machine, when you see this amount of alcohol in the air, I want you to call that a .08. So he was mixing up his own solutions. STACY: Oh, he was literally home-brewing solutions? BRYAN: Literally, home— yeah, making his own solution... STACY: Wow. BRYAN: Which was just absolutely crazy. Initially, the government tried to blame the error on the makers of the 5000, but really the district’s problem was a systemic problem. - I discovered, as I looked into the files, that this accuracy testing had not been done for at least a decade and probably longer. The genesis goes back to this instrument technician’s decision that the accuracy testing protocol in the field didn’t really matter. And it mattered a great deal. PHIL: What was difficult was that, initially, the prosecution side of law enforcement was reluctant to be forthcoming with the problem. And even, there was some covering up of the fact that they weren’t being calibrated. I think it was several hundred cases were jeopardized and had to be dismissed. - I mean, do you feel like the full scope of the problem was identified? - I think it’s reasonable to assume that it— the problems predated what was admitted to. Things weren’t being done right, and people didn’t wanna admit to that. This is both protecting the innocent as well as protecting everyone from those who are guilty, and the police have got to get it right. JESSICA: But in more and more states, we were finding that the police were not getting it right. STACY: This is where we asked the Massachusetts Office of Alcohol and Testing for some various records and got a whole bunch of records related to machines that had failed calibrations. JESSICA: More than a dozen states have grappled with major breakdowns of their DUI programs. Colorado spent more than a million dollars on new Breathalyzers, then assigned an intern to calibrate the equipment. Florida drilled holes in their fleet of machines as a fix against low measurements. - We got photos of— this is the machine that we found a rat had made a nest in. JESSICA: Wow. Cover-ups, corporate secrecy, sloppy lab work— the evidence was piling up, of a system-wide failure. And the problems extended well beyond charges being brought against innocent people. - We have people who’ve been arrested up to seven times. JESSICA: Wow. This left us with a question. Were drivers who really were drunk now getting back on the road? LAWYER: There was an accident. The police were called as a result of this rear-end accident. They encountered Mr. Fernandez, the defendant, who was very unsteady on his feet. There was an odor of alcohol. STACY: Last year, in Massachusetts, more than 28,000 breath tests were thrown out. - There was a reading at that time. The reading was a .16. STACY: It was one of the biggest exclusions of forensic data in American history. All of those drivers could now get a second chance. JOE: He would like an opportunity to get his license back, because he doesn’t have it because of the plea in this particular case. STACY: What’s been happening with those cases, when people bring them back to the courts? - For the most part, the judges have been permitting the cases to be withdrawn— the pleas— or a new trial granted. - Are we now having situations where, you know, people that I think citizens would have real problems with having back on the road? People who have multiple drunk driving convictions, people who were involved in really bad situations, really bad accidents are now successfully getting some cases vacated or dismissed. - Let’s not fool each other. I’m not gonna sit here and tell you that situation and that dynamic isn’t gonna happen. Of course, it’s gonna happen. The question is, whose fault is it? Who did this? And the answer to that is the Office of Alcohol Testing. It really is 100 percent their responsibility. It’s too bad, but it’s happening. <i>[somber music]</i> STACY Bernard and a team of defense lawyers across Massachusetts collected thousands of pages of evidence. Evidence that exposed problems with the management of the state’s forensic lab and maintenance of the machines. Evidence that would lead to the dismissal of eight years of Breathalyzer results. <i>♪ ♪</i> We did Freedom of Information Act requests to a bunch of the district attorneys all throughout Massachusetts to get back records of which cases were affected by this. We found multiple cases of drivers who might have been convicted if their breath tests hadn’t been excluded. <i>♪ ♪</i> - Hi. - Hi. - I’m looking for Mark Delory. This was a case of someone who had a prior drunk driving charge. The officer saw a fair number of signs of inebriation in his written reports. And yet, after the breath test was excluded, this driver was acquitted. - It does say drunk driving, but the machine, unfortunately, was not calibrated. STACY: This seems like a case where losing the breath test really was critical to the prosecution’s case, and without it, they were unable to get a conviction. - A lotta people probably guilty... [laughs] But I was not guilty. - You don’t think you— had you been drinking that day or not? - Oh, I did. I had—I had probably three or four beers, and I supposedly backed into a little old lady, which I did not. - And were you able to get your license back after that? - I got—I got—I got my license back within 30 days. - Okay. It was interesting that he admitted he had three or four beers, at least. Usually, people only say they had two. So three or four was noteworthy. <i>[dramatic music]</i> - We’re heading to Sandusky, Ohio, to meet with John Fusco, who’s the retired former chief executive of one of the major breath test manufacturers called National Patent. He is kind of like a corporate whistle-blower. I’m hoping that he can help us make sense of what we’ve found so far. <i>♪ ♪</i> JOHN: The fact that some of these companies won’t even sell an instrument to a scientific community has got to tell you something about their level of confidence in their own product, and this is wrong. - What do you think is behind that? Why fight like crazy? JOHN: It’s a—I think it’s a— it basically is a paranoia that is brought on by pressure from their customers to not share this information, so that their customers don’t have to go to court and defend the product. If you don’t have enough confidence in your technology to show it and explain it, then maybe you really shouldn’t be using that technology. We never intended for these instruments to be evidence in their own right. All we even really intended for these things is an approximation. The legislatures themselves have made these instruments god. Anytime you put all your faith in technology, there’s a chance you’re gonna get burned. CINDY: Cedar, here. I haven’t been able to work since then. I couldn’t pay my rent. I put all my stuff in storage, gave a lot away. I had to move in my motor home. I had to file bankruptcy because of it. - How have you been dealing with other costs, like food and electricity? - Well, I’m on the state system now, which— - And what does that mean? Are you— - The—that means I’m on welfare, you know. And I’m able to work. I can work. I just can’t get a job because nobody will hire me. JESSICA: We’ve been told to trust the system, to believe in the machine. Breath tests save thousands of lives. But when that system... TONY: You having a hard time staying in your lane. JESSICA: Relies on a secret technology that few are allowed to examine, on labs that work with little or no oversight, perhaps it’s only by breaking it all open, that we can begin to fix it. [siren wailing]