Tesla CEO Elon Musk has worked hard to convince shareholders, media, and, most-importantly, car buyers that Tesla is the “safest vehicle” on the road. He said in May that the brand’s fatality rate is “approx 4X better than average,” expanding upon his claim by Tweeting, “According to NHTSA, there was an automotive fatality every 86M miles in 2017 (~40,000 deaths). Tesla was every 320M miles. It’s not possible to be zero, but probability of fatality is much lower in a Tesla.” What is the actual record of Tesla cars when compared apples-to-apples with their luxury peers? In this article, we answer that question with statistical rigor.

Musk is a phenomenal salesman. In 2016, he facilitated Tesla’s $7.4 billion acquisition of financially-troubled residential solar installer SolarCity — which he owned more than 20% of — by hosting a public event surrounded by an entire neighborhood of “solar houses,” richly illustrating the rationale for the business combination. Amidst much applause, Musk introduced “a solar roof that is better than a normal roof? That looks better, lasts longer, has better insulating effect, and where the cost of roof-plus-electricity is less than that of a normal roof… We’ve got an electric car, a Powerwall [battery storage], and a Solar Roof. The key is that it needs to be beautiful, affordable, and seamlessly integrated. If all of those things are true, why would you go any other direction?”

Musk holding a Solar Roof tile in the “solar neighborhood,” with the “solar house” with an installed Solar Roof and two Tesla vehicles in the driveway. Long after Tesla shareholders voted to approve the SolarCity acquisition, we learned that the solar panels were non-functional props.

A year later, Fast Company revealed that the Solar Roof product displayed that day was completely non-functional. The entire solar neighborhood was a Potemkin village. A source close to Musk told the author “It’s all about the narrative for Elon. Solar Roof was as ‘real’ as anything he’s ever shown [off to the public]. Was it a finished product? By no means.”

The reality of Tesla’s solar business is now bleak. Nearly two years after Musk’s “solar neighborhood” show, the Solar Roof is vanishingly rare and extremely expensive ($42/sq ft for solar tiles), Powerwall is a niche product with long wait times, and SolarCity installations have steadily fallen 70% below their peak.

Musk’s numerous ardent supporters might ask “What’s wrong with optimism when you are trying to save the world?” A skeptic might consider this “optimism” to be fraud, an attempt to mislead investors in order to avoid a SolarCity bankruptcy, massive personal financial losses, and a giant reputational hit.

Tesla fans and critics alike can agree that buyers of luxury cars do not wish to die behind the wheel of their automobile — a wish that is usually granted. Perhaps due to the superior construction and safety features of more expensive cars, or maybe because of the driving habits of their owners, luxury cars typically have a superior safety profile compared to the average car. Just how safe? Our firm analyzed the Insurance Institute for Highway Safety (IIHS) data to find out.

IIHS is a not-for-profit organization funded by auto insurers which uses accident data to calculate driver death rates for each common model of car in the United States. To do so, IIHS uses fatality data from the federal National Highway Traffic Safety Administration’s (NHTSA) Fatality Analysis Reporting System (FARS) and combines that fatality data with vehicle registration data from IHS Automotive (an unrelated, for-profit company). The most recent IIHS fatality data were published in May 2017 and include model years from 2011–2014 and accidents from 2012–2015.

IIHS reports the rate of driver deaths per million vehicle-years. The data are presented car model-by-model, which is great for car buyers who are considering a specific car. One limitation of this method is that, for models with a limited sample size, the 95% confidence interval is very large.

[A quick statistics review: fatal car crashes are rare, binary events. Either a driver had one or s/he didn’t. When the size of a sample is small, or the rate of events in question is low, that means there is wide uncertainty as to what the true population rate of events — the incidence of driver fatalities, in this case — actually is. The 95% confidence interval is a mathematical way of expressing the range of possible values for the true population average; in other words, given the sample size we have, there’s a 95% chance that the actual correct real-world value is within our confidence interval.]

For example, IIHS reports that the Audi A6 4WD had 0 driver fatalities over 101,164 vehicle-years of driving, for a rate of 0 driver deaths per million vehicle years. The 95% confidence interval is 0 to 36, meaning that we can be 95% certain, based upon that sample, that the true rate at which Audi A6 4WD cars of that vintage incur an accident fatal to the driver is between 0 and 36 per million vehicle years.

By looking at groups of models, for example “all large luxury cars” or “all cars made by Mercedes,” we can get a more precise estimate of the driver fatality rate for each group. We did exactly that, and here are some of the most interesting findings:

Large luxury cars are safe. The observed IIHS fatality rate for large luxury cars is 7 per one million vehicle years. Because we have a relatively large data set, we can say with 95% confidence that the fatality rate for large luxury cars is between 3 and 13 per one million vehicle years. Here are the data for the entire group:

Large luxury cars are nearly as safe as houses. Just 8 driver deaths in 1.2 million vehicle years.

In fact, all luxury cars rated by IIHS are generally safe. Across the entire universe of luxury cars, the driver death rate is only 13 per one million vehicle years (95% confidence interval 9–18):

All luxury cars are pretty safe. Just 13 driver deaths per million years (confidence interval 9–18).

German luxury cars — Mercedes, Audi, and BMW — may be modestly safer than the luxury car group average. All three of the German brands have a similar driver death rate, ranging from 10 deaths per million vehicle years for Audi and BMW to 12 deaths per million vehicle years for Mercedes.

German luxury cars appear slightly safer than the luxury car group, on the whole.

So what about Tesla? Is it “the safest car” on the road? Is it “4X better than average?” Is the “probability of fatality much lower in a Tesla?” I’m willing to accept a huge amount of puffery when it comes to Taco Bell menu items, considerably less when dealing with financial matters (apparently much less than most people, given that Tesla sports a $64 billion valuation after the SolarCity debacle), and very little when it comes down to things that may kill me. Tesla cars look cool and, like most high-performance electric vehicles, are extremely fun to drive. But I have a wife and kids to worry about. Like most potential luxury car buyers, I want a very safe car.

Neither IIHS nor NHTSA have yet published summary quantitative real-world fatality data on Tesla cars. As has been previously noted by astute observers, when Musk said, “According to NHTSA, there was an automotive fatality every 86M miles in 2017 (~40,000 deaths). Tesla was every 320M miles,” only the first part of that data was actually “according to NHTSA.”

Musk’s “every 320 million miles” wasn’t the correct fatality per mile figure, and Musk either knew or should have known that. Regardless, summary national data obtained dividing all deaths (including drivers, passengers, pedestrians, cyclists, etc.) by an estimate of total miles driven in the United States isn’t very useful for comparing one model of car against its peers. We don’t know how far most cars have actually gone. That’s why IIHS uses “vehicle years” instead of miles; we know if a car is registered or not. And IIHS counts only driver fatalities, since there is always one driver in a car, but some cars tend to have more passengers than others.

Apples-to-apples, how does Tesla’s real-world driver fatality experience compare with other cars in its class? IIHS’ latest published analysis didn’t have enough Tesla data to include the make in their numbers, however the NHTSA-FARS database is now searchable through 2016. We set out to use the NHTSA-FARS database to perform a comprehensive analysis of Tesla driver deaths through 2016. Along the way, we discovered several errors in the database, likely due to fact that Tesla is not assigned a unique model code but is rather part of an obscure basket of “Other domestic manufacturers” with Studebakers, Hudsons, Packards, and others. Worse yet, that isn’t even the only “other vehicle” basket. We found Tesla accidents coded correctly, Tesla accidents coded incorrectly, and Tesla accidents seemingly completely missing.

The NHTSA-FARS code for Tesla. Just choose (29) Other domestic manufacturers. It’s easy to imagine the local people entering this data at hundreds of thousands of disparate localities struggling here. Wrong ways to code a Tesla include leaving the field blank (-1), or choosing (69) other import, (97) not reported, (98) other make, or (99) unknown make.

A sample of make codes for other cars; very simple in comparison to Tesla.

California, Tesla’s largest market by far, accounting for approximately 45% of Model S sales during the period of our analysis, appears to be a massive source of error. From other public sources, we noted four Tesla driver fatalities in California between July 2014 and June 2015. In California’s SWITRS (Internet Statewide Integrated Traffic Records System) database, only one of the four Tesla driver deaths was coded properly as a Tesla. We found a second fatal accident coded with model year 2012 but no model information. We were unable to locate the other two fatal accidents in the California database at all. Those two were, unsurprisingly, also missing from the Federal NHTSA-FARS database.

California SWITRS data for Tesla is largely missing. Above: known Tesla death (from news) with a model year but no make. Only one death out of the four publicly known before June 2015 is correctly recorded; there are none recorded from July 2015 to December 2016.

Very concerningly from a data integrity standpoint, as the number of Tesla vehicles on the road increased, and ex-California fatalities increased from zero in the pre-December 2015 period to seven in the period from December 2015 to December 2016, reported Tesla driver fatalities from the entire state of California were zero. That’s exceedingly unlikely to be correct.

Given the data errors in California and NHTSA-FARS, which are likely specific to Tesla given its unusual make/model code, we decided to include all available public sources in our analysis, to get the most robust analysis possible. We include all accidents on or prior to December 31, 2016.

To have the largest sample size and best confidence intervals possible, we also included international accidents (three total, from Canada, Holland, and China). Given the higher incidence of fatal accidents per vehicle-year in China compared to the United States, and the lower incidence of fatal accidents in Holland and Canada, these three deaths should adjust to 3.63 deaths. However, for the sake of conservatism and to avoid a large debate about a rather small difference, we use 3.00 deaths in our analysis.

Because we included both U.S. and international accidents as the numerator, it is appropriate to use all (including U.S. and international) Tesla vehicle-years as our denominator. We included Model S and X, but not the very low volume Tesla Roadster. Model 3 was not available until 2017. Tesla quarters tend to be back-end loaded, with more vehicles delivered near the end than the beginning, so we made a very slight adjustment for this (see table below). We also assumed that every Tesla ever sold was still on the road at the end of the period, December 31, 2016, which has the effect of modestly underestimating the true fatality rate.

Total Tesla Model S and X vehicle years through December 31, 2016

To build our list of Tesla driver fatalities, we searched the NHTSA-FARS database using various methods, including for vehicles of make 29 & model 5 (Tesla), and separately for VINs beginning with 5YJ. This search found five Tesla driver fatalities. We conducted a Google search for Tesla driver fatalities reported by English-language sources, using a variety of search terms, and reviewed and fact-checked the crowdsourced Tesla mortality file maintained by @ElonBachman (EB) on Twitter, first published in May 2018. The EB list included three driver fatalities prior to December 31, 2016 not found in our NHTSA-FARS search. Our NHTSA-FARS search identified one fatality not found on the EB file (and incidentally also identified one additional Tesla-associated pedestrian fatality in a separate accident).

Documented United States Tesla driver fatalities through December 31, 2016

Documented international Tesla driver fatalities through December 31, 2016

Remember that BMW and Audi combined had 9 driver fatalities in nearly 900,000 vehicle years? Clearly, 11 Tesla driver fatalities in 265,000 vehicle years is not a good start. But even those numbers likely understate the danger of driving a Tesla.

We believe it is nearly certain there are other fatalities not discovered in our search. Many fatal accident reports in local media don’t bother to mention the make of the car(s) involved and NHTSA-FARS reporting of Tesla fatalities seems to have significant data entry errors due to the unusual make/model coding for Tesla. It’s nonsensical that Tesla fatalities dropped from four to zero in California for the last 13 months of the data, while the number of Tesla vehicles on the road increased dramatically and fatalities ex-California ramped dramatically as well — especially when we consider that only one of the four California fatalities in the early part of our data series was actually entered correctly in their state database. Internationally, many accidents are not reported by English-language media or do not contain the make of the car involved, so there’s likely some missing data there as well.

We interviewed @ElonBachman, who believes “[there is a] 0 percent chance I have all of the [fatalities]. There’s one on there that someone sent me because even though it doesn’t say Tesla in the article, they watched the video and saw it’s a Tesla; that sort of thing. Another was sent to me by a personal injury lawyer, etc. [I’m] probably missing a ton in China.”

The fatality we found that wasn’t previously reported, NHTSA case 17–964, occurred the day before Thanksgiving 2016 in Winnebago County, Illinois. A single-occupant crash which caused the death of Dr. Kevin Gander, it received little media coverage, and none mentioning the make of the car. The crash scene was dark, and the car was so badly damaged that the first policeman on scene told the dispatcher the car was “like a Chrysler 200.” It was a 2012 Model S, with special Illinois electric vehicle license plate 1375 EL.

Even without making any adjustment whatsoever for missing fatality data, Tesla drivers are much more likely to die than their peers driving other luxury cars. Eleven deaths in 265,290 vehicle-years is a stunningly high driver fatality rate of 41.46. That’s quadruple the rate of Audi and BMW, and more than triple the rate of all luxury cars combined.

Tesla’s driver mortality rate of 41 per million vehicle years compares extremely unfavorably with the luxury car average of 13 per million and the German luxury car rate of 11 per million. The extended bars on the chart represent 95% confidence intervals.

Tesla’s mortality rate (41 deaths per million vehicle years) is so much higher than the average luxury car (13 deaths per million vehicle years) that when comparing the two, the difference is hugely statistically significant. The difference is 28 additional deaths per million vehicle years, with a confidence interval of 11 to 63, and a p-value of 0.0001.

Remember that, unless California Tesla driver deaths magically went to zero in 2016 as Tesla increased production, and unless there were zero unreported deaths in the rest of the world, the true value for Tesla’s fatality rate is likely even higher. Regardless, Musk’s assertions that Tesla is the “safest vehicle” on the road or “four times better than average” are ridiculous untruths. Tesla’s driver fatality rate was massively higher than luxury peers, and at least 37% higher than the average 2011–2014 car. Musk’s “safest vehicle” on the road claim is as phony as his 2016 solar neighborhood.

Lest anyone believe that because we are short Tesla shares, our analysis somehow doesn’t matter, it’s important to note that we are short Tesla precisely because of our open-minded analysis of the stock’s prospects. We are a long-biased fund, meaning we own more shares of businesses than we are short. We could have have had a long position or no position in Tesla, but we are short because Elon Musk makes a lot of promises he doesn’t keep, because Tesla, though fun to drive, has more than its fair share of quality issues, because profitable manufacturing of a $35,000 Model 3 is almost certainly a fiction, and because demand at realistic price points is dramatically lower than the 400,000 cars or $29 billion of revenue that analysts project for next year.

Personally, I would have loved to see Tesla produce an extremely safe, high-quality, affordable Model 3. We’d have covered our small short position in a flash, instead of increasing it. I am a liberal and an environmentalist, one who grows native plants from seed as a hobby and who believes man-made climate change will cause one of earth’s largest-ever extinction events. I am glad for Tesla’s contributions to electric cars, and plan to buy an EV in the next year. I’m a Model 3 reservation holder, but after this fatality analysis, the chances of me buying a Tesla in the next couple of years have dropped to essentially zero, though I’ll probably keep my reservation as a first-hand way of gauging wait times, which seem much, much shorter than one would expect if the Model 3 truly had a massive backlog.

Thank you for reading our analysis, and whatever car you drive, please pay attention and drive carefully. Accidental death is common, often preventable, and always a tragedy.

Authors note: After completing our analysis, we reached out to IIHS and obtained their luxury car data set containing both the published numbers as well as the data unadjusted for age and gender, to see if IIHS adjustments were in any way responsible for our findings. IIHS adjustments in the luxury car category ranged between zero and trivial and did not change the outcome of any of our analyses.

About the author: A former military/academic physician turned long-short equity hedge fund manager in 2008, Chris’ strong background in science, evidence-based medicine, and data analysis includes the Society of Air Force Physicians Arthur Grollman Award for outstanding research by a medical resident, winning the National Beta Club Science Contest, and an event championship in data analysis at the National Science Olympiad.

Sources:

Links to the 11 documented Tesla driver fatalities through December 31, 2016:

Additional sources:

http://www.iihs.org/iihs/topics/driver-death-rates

https://forums.tesla.com/forum/forums/how-many-cars-sold-2014-and-2015

Binomial confidence intervals for individual cars except Tesla provided by IIHS

Binomial confidence intervals for Tesla and groups using Conflint.xls from John Pezzullo

Comparison of proportions performed using: https://www.medcalc.org/calc/comparison_of_proportions.php

https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/809952

Disclaimer: The author’s firm is an Exempt Reporting Adviser, and is not a Registered Investment Adviser in the United States or any other jurisdiction. Use of this material is at your own risk, and does not constitute personalized investment advice. In no event should our firm or any affiliated party be held liable for any direct or indirect trading losses caused by any information in this publication. Readers should perform your own research and due diligence, and consult your own financial, legal, and tax advisors before making any investment decision with respect to transacting in any securities covered herein. You should assume that as of the date of this publication, funds managed by our firm have a bearish position in the stock (and/or options of the stock) covered herein, and therefore stand to realize significant gains in the event that the price of the stock declines. Following this publication, we intend to continue transacting in the securities covered herein, and we may be long, short, or neutral at any time hereafter regardless of our initial recommendation, conclusions, or opinions.

To the best of our ability and belief, all information contained herein is accurate and reliable, and has been obtained from public sources we believe to be accurate and reliable, and who are not insiders or connected persons of the stock covered herein or who may otherwise owe any fiduciary duty or duty of confidentiality to the issuer. However, such information is presented “as is,” without warranty of any kind — whether express or implied. Our firm makes no representation, express or implied, as to the accuracy, timeliness, or completeness of any such information or with regard to the results to be obtained from its use. All expressions of opinion are subject to change without notice, and our firm does not undertake to update or supplement this report or any of the information contained herein.

This is not an offer to sell or a solicitation of an offer to buy any security, nor shall any security be offered or sold to any person, in any jurisdiction in which such offer would be unlawful under the securities laws of such jurisdiction. Such an offer or solicitation can only be made by the confidential private placement memoranda relating to the investment funds (“Funds”) managed by our firm, which we will provide only to qualified offerees. Past results of Funds are not necessarily indicative of future performance, and future performance may be volatile. Funds are not subject to the same regulatory requirements as a mutual fund, including the SEC’s registration and disclosure requirements. Funds are subject to various other risk factors and conflicts of interest that are disclosed in the Funds’ offering memoranda and subscription documents.