Congressional progressives’ push for a Green New Deal briefly put the question of a national high-speed passenger rail initiative back into the discourse. Then this week, we saw reality bite back sharply: Newly inaugurated California Gov. Gavin Newsom all but canceled the state’s ambitious plans for a statewide high-speed rail network, one that would link San Diego and Los Angeles to San Francisco, San Jose, and Sacramento via the major cities in the state’s Central Valley.

The dual tragedy is that, given the cost overruns and lack of federal support, canceling the project was likely the right call — and yet the basic idea of high-speed passenger rail to connect California’s major cities is a perfectly sound and reasonable one.

More broadly, high-speed rail (unlike hyperloops, maglevs, or hypothetical biofuel-powered airplanes) is a proven technology that has been deployed at scale in Japan, China, Korea, France, Spain, and several smaller European countries. It’s not viable as a substitute for all air travel, but given cities that are an appropriate distance apart, we have seen it can displace most air travel and some car traffic, giving people a superior transportation option that is also cleaner.

The United States is less densely populated than Europe or Japan, and our cities are less downtown-centric than European or Japanese cities, so it’s hard to imagine a scenario in which rail would achieve European or Japanese levels of popularity Still, the United States has plenty of city pairs that would benefit from high-speed rail connections.

But we don’t have any, and we aren’t making any progress toward building any, including in the regions of the country where political support for the idea is high, largely because the entire political model behind undertaking large transportation projects is completely broken.

A California program fatally compromised by politics

San Francisco and Los Angeles are the two largest cities in California, travel demand between them is massive, and they are an appropriate distance apart for a fast train to achieve a large share of the market. A reasonable concept would be to pick a train route between the two cities that’s the most cost-effective in terms of dollars spent per rider. Spend money, in other words, but only do so when extra money is likely to generate extra ridership — primarily by making the key connection as fast as possible.

But that’s not remotely what California did.

Instead, as Ethan Elkind, who directs the climate program at the Center for Law, Energy, and the Environment (CLEE) at UC Berkeley Law, wrote in 2014, a bunch of political considerations got in the way of that goal:

One of these compromises — taking a somewhat less efficient route through the Central Valley in order to hit more Central Valley population centers — was defensible on the merits, since hitting intermediate destinations increases ridership.

A second — taking a weird detour to Palmdale rather than going straight from LA to Bakersfield — was totally senseless, slowing down traffic at great expense purely to promote a single transit-oriented development scheme that happened to have sparked enthusiasm on the LA County Commission.

The third — which provoked endless fights among blog commenters years ago — was deciding to serve San Jose on the main line rather than with a spur, even though this cost more money while making LA-SF trips and trips from Sacramento to both Bay Area cities slower.

The key thing in all three cases was that the route adjustments increased the number of elected officials who could get “a win” from the project, at the expense of serving the project’s core function. As an economic development scheme for the Central Valley, you could make the case for this, but San Jose doesn’t need an economic development scheme, and the Palmdale concept is just a ridiculously petty thing to undermine a massive infrastructure project over. According to Clem Tiller, the Palmdale route made the north-south trip 12 minutes slower while costing $5 billion in extra spending.

Spending $5 billion on a transportation improvement is necessarily going to be a tough political lift. But spending an extra $5 billion to make the quality of the transportation worse is a disaster. The overall thinking was not that the core SF-LA project was so valuable that California should go do it. Instead, it was that the core SF-LA project was so valuable that it made the whole thing a “too big to fail” political juggernaut, which in turn led to some odd decisions about the order in which things would be done.

Perverse sequencing decisions

A giant project gets done piece by piece, and a natural way to approach that would have been to do a small, useful piece first.

One such useful piece would have been to upgrade the existing Amtrak route from Los Angeles to San Diego. It’s fairly popular already, and with investment in electrification, the trip could be made nearly an hour faster. California, however, decided to take the much more expensive option of planning to build a brand new rail line between the two cities on a different route — adding about $7 billion in extra costs to cut the trip time by 90 minutes rather than 60.

Except they didn’t actually plan to build that anytime soon. Nor were they planning to immediately build the LA-to-Bakersfield segment, which, while kind of small potatoes, would have been a useful transportation service. The fear was that the project would end up getting saddled with cost overruns and delays, and there would be political pressure to scale it back — pressure that might succeed if a smaller-scale project proved to be politically appealing.

So instead, they set about to construct the segment connecting Bakersfield and Merced, two smaller cities in the middle of the state, as the initial segment. The idea was basically that a Bakersfield-Merced high-speed rail was so obviously ridiculous that nobody would be content to build just that and end the project, so future governments would go find billions of extra dollars somehow.

But Newsom — seeing no path to obtaining more federal money for the project and not wanting to invest additional state funds in a bloated program that would count as fellow Democrat Jerry Brown’s legacy rather than his — just pulled the trigger on the unthinkable scale-back, which, if it actually happens, will leave California worse off than if it had never gone down this path before.

The absolute priority of roads

Rail fans sometimes get annoyed by discussion of planning snafus and cost overruns in the US train sector because, of course, at the end of the day, far more money is spent on highways.

There are, however, a number of significant differences. First and foremost, to the extent that US highway spending is wasteful (which it surely is), that’s overwhelmingly because the useful highways have already been built. Virtually every pair of American cities that could plausibly be connected on a direct route by a big highway is already connected — and when they aren’t (like Denver and Salt Lake City), it’s because there are enormous mountains in the way. Highway boondoggles involve either pointless roads or straightforward graft rather than inability to execute on perfectly reasonable projects.

Even in a very progressive jurisdiction, the Kings County (Washington) Council banned new fossil fuel infrastructure projects on January 31, then four days later celebrated the opening of a new $2 billion car tunnel through downtown Seattle.

But the true privileged status of automotive projects in the United States isn’t the willingness to spend money on them; it’s the willingness to actually make drivers’ interests the priority.

I-5 out of LA toward Bakersfield takes the direct route that was rejected for high-speed rail. Forcing everyone on the road to detour east to Palmdale to promote local economic development would be unthinkable. When the government builds a road project, it tries to make it useful for road users.

Meanwhile, transit and rail projects are saddled with Buy America requirements that ensure they will support more jobs per dollar spent but purchase less transportation benefit per dollar spent. Banning the import of foreign cars and car parts would, obviously, create far more manufacturing jobs than doing the same for rolling stock — but making cars cheap is a policy priority, while making rail projects cost-effective is not.

This sometimes reaches absurd proportions, as with the burst of Obama-era mixed-traffic streetcar projects. These streetcars superficially resemble European tram technology, but unlike European trams, they mostly lack dedicated lanes to run in. Consequently, they tend to be not just more expensive than buses but slower as well, because unlike a bus they can’t navigate around obstructions.

Essentially every American city could improve its mass transit offerings at minimal cost by creating dedicated bus lanes on the busiest routes, but doing that would antagonize drivers, and the federal government offers no support for it (because, again, it would antagonize drivers).

You end up with a choice either to not spend money (the Republican vision) or to spend it in deliberately wasteful ways to create jobs.

America needs to get serious about infrastructure

New York City recently completed the most expensive subway project in the history of the world, a brief three-station segment of the Second Avenue line.

New York’s Upper East Side is so densely populated that even at the extraordinary cost per mile involved in this project, it’s a good idea. But the exorbitant costs have made it essentially impossible to further expand the project north into East Harlem and then crosstown on 125th Street, as in the original vision.

If New York were able to build subways at the kind of per mile prices achieved in Paris — about $230 million per kilometer on recent projects rather than more than $2 billion per kilometer for the Second Avenue subway — then New York’s current mass transit spending plans would be sufficient to expand and transform the system. But under the current dynamic transit planners can’t get much done.

Some of the excess cost comes from identifiable examples of overstaffing. New York’s digs featured a dedicated elevator operator, for example, as if this job hadn’t been made obsolete by automation decades ago. Some of it comes from slightly inexplicable overbuilding, such as full-length mezzanine levels at Second Avenue Subway stations rather than making do with smaller ones.

And part of the problem is politics. The cheap way to build a station is to tear up a big hole in the ground and dig (this is called “cut and cover”), but transit officials instead chose a more expensive method that involved digging a small hole and then laboriously blasting a station-sized cavern under the street. But most fundamentally, all these specific excess costs arise because no part of the political system was focused on cost-effectiveness as a priority.

And these problems are not merely localized. From the very beginning, the Obama administration’s “vision” for high-speed rail featured bizarre technical choices driven by fairly obvious political motives.

This map, as drawn, features a number of routes that, due to the small size of the cities involved, are not very promising in ridership terms: Dallas to Little Rock, Boston to Portland, Boston to Montreal, and Birmingham to New Orleans. Meanwhile, it ignores the much more obvious potential of connecting Houston to Dallas. It’s only if you ignore information about transportation demand and think about the US Senate map as it existed in 2009 that this begins to make sense.

Adding Maine, New Hampshire, Vermont, Arkansas, and Louisiana (states that had many of the key swing senators at the time) to the route map is a lot more politically valuable than connecting America’s fourth and fifth biggest metro areas, even if it makes no sense in ridership terms.

This is a fine way to proceed if you’re just tossing around stimulus money and trying to seem vaguely forward-thinking. But if we want to reduce carbon dioxide emissions and/or improve American economic productivity, we have to do better than this.

From high-level choices about which cities to prioritizes to mid-scale decisions about routes to tiny-scale decisions about how to build stations and all the rest, if the country wants a modern transportation system it has to prioritize building useful transportation — rather than its current practice of trying to avoid any tough choices until the point where nothing gets built at all.