The TypeScript Tax

A Cost vs Benefit Analysis

Photo: LendingMemo (CC BY 2.0)

TypeScript grew a great deal between 2017 and 2019, and in many ways, for good reason. There’s a lot to love about TypeScript. In the 2018 State of JavaScript survey, almost half the respondents said they’d tried TypeScript and would use it again. But should you use it for your large scale app development project?

This article takes a more critical, data-driven approach to analyze the ROI of using TypeScript to build large scale applications.

TypeScript Growth

TypeScript is one of the fastest growing languages, and is currently the leading compile-to-JavaScript language.

Google Trends 2014–2019 TypeScript Topic Growth

GitHub Fastest Growing Languages by Contributor Numbers [Source]

This is very impressive traction that shouldn’t be discounted, but it is still far from dominating the over-all JavaScript ecosystem. You might say it’s a big wave in a much bigger ocean.

Google Search Trends 2014–2018 JavaScript (Red) vs TypeScript (blue) Topic Interest

GitHub Top Languages by Repositories Created: TypeScript is Not in the Top 5. [Source]

That said, TypeScript hit an inflection point in 2018, and in 2019, a large number of production projects will use it. As a JavaScript developer, you may not have a choice. The TypeScript decision will be made for you, and you shouldn’t be afraid of learning and using it.

But if you’re in the position of deciding whether or not to use it, you should have a realistic understanding of both the benefits and the costs. Will it have a positive or negative impact?

In my experience, it has both, but falls short of positive ROI. Many developers love using it, and there are many aspects of the TypeScript developer experience I genuinely love. But all of this comes with a cost.

Background

I come from a background using statically typed languages including C/C++ and Java. JavaScript’s dynamic types were hard to adjust to at first, but once I got used to them, it was like coming out of a long, dark tunnel and into the light. There’s a lot to love about static types, but there’s a lot to love about dynamic types, too.

On and off over the last few years, I’ve gone all-in on TypeScript full time and racked up more than a year of hands-on daily experience. I went on to lead multiple large-scale production teams using TypeScript as the primary language, and got to see the high-level multi-project impact of TypeScript and compare it to similar large-scale native JavaScript builds.

In 2018, decentralized applications took off, and most of them use smart contracts and open-source software. When you’re dealing with the internet of value, bugs can cost users money. It’s more important than ever to write reliable code, and because these projects are generally open-source, I figured it was nice that we developed the code in TypeScript so that it’s easier for other TypeScript teams to integrate, while maintaining compatibility with projects using JavaScript, as well.

My understanding of TypeScript, including its benefits, costs, and weaknesses have deepened considerably. I’m saddened to say that it wasn’t as successful as I’d hoped. Unless it improves considerably, I would not pick TypeScript for another large scale project.

What I Love About TypeScript

I’m still long-term optimistic about TypeScript. I want to love TypeScript, and there’s a lot I still do love about it. I hope that the TypeScript developers and proponents will read this as a constructive critique rather than a hostile take-down piece. TypeScript developers can fix some of the issues, and if they do, I may repeat the ROI analysis and come to different results.

Static types can be very useful to help document functions, clarify usage, and reduce cognitive overhead. For example, I usually find Haskell’s types to be helpful, low-cost, pain-free, and unobtrusive, but sometimes even Haskell’s flexible higher-kinded type system gets in the way. Try typing a transducer in Haskell (or TypeScript). It’s not easy, and probably a bit worse than the untyped equivalent.

I love that type annotations can be optional in TypeScript when they get in the way, and I love that TypeScript uses structural typing and has some support for type inference (though there’s a lot of room for improvement with inference).

TypeScript supports interfaces, which are reusable (as opposed to inline) typings that you can apply in various ways to annotate APIs and function signatures. A single interface can have many implementations. Interfaces are one of the best features of TypeScript, and I wish this feature was built into JavaScript.

The best news: If you use one of the well supported editors (such as Atom or Visual Studio Code), TypeScript’s editor plugins still provide the best IDE developer experience in the JavaScript ecosystem, in my opinion. Other plugin developers should try them out and take notes on how they can improve.

TypeScript ROI in Numbers

I’m going to rate TypeScript on several dimensions on a scale of -10–10 to give you a better sense of how well suited TypeScript may or may not be for large scale applications.

Greater than 0 represents a positive impact. Less than 0 represents a negative impact. 3–5 points represent relatively strong impact. 2 points represents a moderate impact. 1 point represents a relatively low impact.

These numbers are hard to measure precisely, and will be somewhat subjective, but I’ve estimated the best I can to reflect the actual costs and rewards we saw on real projects.

All projects for which impact was judged were >50k LOC with several collaborators working over several months. One project was Angular 2 + TypeScript, compared against a similar project written in Angular 1 with standard JavaScript. All other projects were built with React and Node, and compared against React/Node projects written in standard JavaScript. Subjective bug density, subjective relative velocity, and developer feedback were estimated, but not precisely measured. All teams contained a mix of experienced and new TypeScript developers. All members had access to more experienced mentors to assist with TypeScript onboarding.

Objective data was too noisy in the small sampling of projects to make any definitive objective judgements with a reliable error margin. On one project, native JavaScript showed a 41% lower public bug density over TypeScript. In another, the TypeScript project showed a 4% lower bug density over the comparable native JavaScript version. Obviously, the implementation (or lack) of other quality measures had a much stronger effect than TypeScript, which skewed the numbers beyond usability.

With margin-of-error so broad, I gave up on objective quantification, and instead focused on feature delivery pace and observations of where we spent our time. You’ll see more of those details in the ROI point-by-point breakdown.

Because there’s a lot of subjectivity involved, you should allow for a margin of error in interpretation (pictured in the chart), but the over-all ROI balance should give you a good idea of what to expect.

TypeScript Cost vs Benefit Analysis: Likely Negative ROI

I can already hear the peanut gallery objections to the small benefits scores, and I don’t entirely disagree with the arguments. TypeScript does provide some very useful, powerful capabilities. There’s no question about that.

In order to understand the relatively small benefit scores, you have to have a good understanding of what I’m comparing TypeScript to: Not just JavaScript, but JavaScript paired with tools built for native JavaScript.

Let’s look at each point in more detail.

Developer Tooling: My favorite feature of TypeScript, and arguably the most powerful practical benefit from using TypeScript is its ability to reduce the cognitive load of developers by providing interface type hints and catch potential errors in realtime as you’re programming. If none of that were possible in native JavaScript with some good plugins, I’d give TypeScript more points on the benefit side, but the 0 point is what’s already available using JavaScript, and the baseline is already pretty good.

Most TypeScript advocates don’t seem to have a good understanding of what TypeScript is competing against. The development tool choice isn’t TypeScript vs native JavaScript and no tooling. It’s between TypeScript and the entire rich ecosystem of JavaScript developer tools. Native JavaScript autocomplete and error detection gets you 80% — 90% of the benefits of TypeScript when you use autocomplete, type inference, and lint tooling. When you’re running type inference, and you use ES6 default parameters, you get type hints just like you would with type-annotated TypeScript code.

Example of Native JavaScript Autocomplete with Type Inference

In fairness, if you use default parameters to provide type hints, you don’t need to supply the annotations for TypeScript code, either, which is a great trick to reduce type syntax overhead — one of the overhead costs of using TypeScript.

TypeScript’s tooling for these things is arguably a little better, and more all-in-one — but it’s not enough of an improvement to justify the costs.

API Documentation: Another great benefit of TypeScript is better documentation for APIs which is always in sync with your source code. You can even generate API documentation from your TypeScript code. This would also get a higher score, except you can get the same benefit using JSDoc and Tern.js in JavaScript, and documentation generators are abundant. Personally, I’m not a big fan of JSDoc, so TypeScript does get some points, here.

Even with the best inline documentation in the world, you still need real documentation, so TypeScript enhances, rather than replaces existing documentation options.

Refactoring. In most cases, if you can gain a significant benefit from TypeScript in your refactoring, that’s often a code smell indicating that your code is too tightly coupled. I have written an entire book on how to write more composable, more loosely coupled code, called “Composing Software”. If TypeScript is saving you a lot of refactoring pain, there’s a good chance tight coupling is still causing you a lot of other avoidable problems. I strongly suggest reading the book, particularly the chapter “Mocking is a Code Smell”, which provides a lot of information on the causes of tight coupling and some best practices that can help you avoid them.

On the other hand, some companies run very large ecosystems of connected projects sharing the same code repository (e.g., Google’s famous monorepo). Using TypeScript enables them to upgrade API design choices to account for better designs and new use-cases. The developers responsible for those upgrades are also responsible for ensuring that their library changes don’t break any of the software in the monorepo that depends on those libraries. TypeScript may offer significant time savings for this very limited subset of TypeScript users.

I say very limited subset, because giant, closed monorepo ecosystems are the exception, rather than the rule. The process might scale across Google, but can’t scale to repositories that the library authors are not aware of. Making breaking changes to library APIs used by a broader ecosystem can break code you don’t even know exists.

In traditional, more decentralized library ecosystems, people avoid breaking changes to APIs, and instead create new features following the open/closed principle (APIs are open for extension, and closed to breaking changes). This is how the web platform itself has mostly evolved, with a few exceptions. This is why React still supports features that have been replaced by better options since React 0.14. React evolves and adds great new features, radically improving the developer experience without breaking old functionality. For instance, class components will still be supported by React, even after the much improved React Hooks API matures.

That makes changes across the whole ecosystem optional, rather than required. Teams can upgrade their software gradually, on an as-needed basis rather than heaping a whole-ecosystem code change project on the library team.

Even in cases where whole ecosystem code changes are required, type inference and automated codemods can help — no TypeScript required.

I initially mentally scored refactoring a zero and left it off the list because I strongly favor the open/closed approach, inference, and codemods. However, some teams are getting real benefits from it under limited circumstances.

There’s a very good chance that you’d be better served in other ways using native JavaScript.

Type safety doesn’t seem to make a big difference. TypeScript proponents frequently talk about the benefits of type safety, but there is little evidence that type safety makes much difference (really, static types seem to have very little impact) to production bug density. This is important because code review and TDD make a very big difference (40% — 80% for TDD alone). Pair TDD with design review, spec review, and code review, and you’re looking at 90%+ reductions in bug density. Many of those processes (particularly TDD) are capable of catching all of the same class of bugs that TypeScript catches, as well as many bugs that TypeScript will never be able to catch.

TypeScript is only capable of addressing a theoretical maximum of 20% of “public bugs”, where public means that the bugs survived past the implementation phase and got committed to the public repository, according to Zheng Gao and Earl T. Barr from University College London, and Christian Bird from Microsoft Research.

The authors of this study think they’ve underestimated the impact of TypeScript because they assume that all the other quality measures have already been applied, but they made no effort to judge the quality of the other bug prevention measures. They acknowledge the variable, but leave it entirely out of the calculations.

In my experience, the vast majority of teams have partially applied some measures, but rarely applied all important bug prevention measures well. On my teams, we use design review, spec review, TDD, code review, lint, schema validation, and company-sponsored mentorship, which all have dramatic impacts on bug density, reducing type errors to very near zero.

In my experience, all but linting have a larger impact on code quality than static types. In other words, I’m starting from a much stricter definition of zero than the authors of the paper.

If you have not properly implemented those other bug prevention measures, I have no doubt you’ll see 15% — 18% reduction in bug density using TypeScript alone, but you’ll also completely miss 80% of the bugs until they get to production and start causing real problems.

Some will argue that TypeScript provides realtime bug feedback, so you can catch the bugs earlier, but so do type inference, lint, and TDD (I set up a watch script to run my unit tests on file save, so I get very near immediate, rich feedback). You may argue that these other measures have a cost, but because TypeScript will always miss 80% of bugs, you can’t safely skip them either way, so their cost applies to both sides of the ROI math, and is already factored in.

The study looked at bugs that were known in advance, including the exact lines that were changed to fix the bugs in question, where the problem and potential solutions were known prior to introduction of typings. What this means is that even knowing that the bugs existed in advance, TypeScript was unable to detect 85% of public bugs — catching only 15%.

Update: We’re going to give TypeScript the absolute theoretical maximum benefit of the doubt and use 20% in our calculations, to drive home the point about exponentially diminishing returns.

Why are so many bugs undetectable by TypeScript, and why did I call that 20% reduction a “theoretical maximum” effect? For starters, specification errors caused about 78% of the publicly classified bugs studied on GitHub. The failure to correctly specify behaviors or correctly implement a specification is the most common type of bug by a huge margin, and that fact automatically renders an overwhelming majority of bugs impossible for TypeScript to detect or prevent. In “To Type or Not to Type”, the study authors identified and classified a range of “ts-undetectable” bugs.

“StringError” above are the classification of errors where the string was the right type, but contained the wrong value (like an incorrect URL). Branch errors and predicate errors are logic errors that led to the wrong code paths being used. As you can see there are a variety of other errors that TypeScript just can’t touch. There’s little potential that TypeScript will ever be capable of detecting more than 20% of bugs.

But a 20% sounds like a lot! Why doesn’t TypeScript get much higher bug prevention points?

Because there are so many bugs that are not detectable by static types, it would be irresponsible to skip other quality control measures like design review, spec review, code review, and TDD. So it’s not fair to assume that TypeScript will be the only thing you’re employing to prevent bugs. In order to really get a sense of ROI, we have to apply the bug reduction math after discounting the bugs caught by other measures which were not adequately factored in by the study authors.

It’s unsafe to skip other measures: Spec errors: 80% — Type errors: 20%

Imagine your project would have contained 1,000 bugs with no bug prevention measures. After applying other quality measures, the potential production bug count is reduced to 100. Now we can look at how many additional bugs TypeScript would have prevented to get a truer sense of the bug catching return on our TypeScript investment. Close to 80% of bugs are not detectable by TypeScript, and all TypeScript-detectable bugs can potentially be caught with other measures like TDD.

No measures: 1000 bugs

After other measures: 100 bugs remain — 900 bugs caught

After adding TypeScript to other measures: 80 bugs remain — 20 more bugs caught

Bar graph of bugs remaining after applying reduction measures: TypeScript provides little added benefit.

Some people argue that if you have static types, you don’t need to worry about writing so many tests. Those people are making a silly argument. There is really no contest. Even if you’re going to employ TypeScript, you still need the other measures.

Chart: Adding TypeScript after reviews, TDD catches a tiny fraction of the total bug count.

In this scenario, reviews and TDD catch 900/1,000 bugs without TypeScript. TypeScript catches 200/1,000 bugs if you skip reviews and TDD. You obviously don’t have to pick one or the other, but adding TypeScript after applying other measures leads to a very small improvement due to exponentially diminishing returns.