For nearly two years now, Project Treble has served as a source of hope and inspiration for the Android faithful — a magical-seeming fix for a long-standing source of frustration.

Unveiled in the spring of 2017, Project Treble is Google's most ambitious effort ever to rethink the structure of Android, as an operating system, and break the software down in some wild new ways — all with the aim of making it easier for phone-makers to get OS updates out to their devices in a reasonably timely manner. Android upgrades have been a major pain point on the platform practically since the start, and despite all of Google's previous efforts, manufacturers just keep getting worse at providing post-sales software support to their highest-paying customers.

The current Android 9 Pie release is particularly significant because it's the first Android version to take full advantage of all the changes Treble has introduced — and to come into a world where pretty much all of the current flagship phones were built to support it. So in theory, at least, this current Android upgrade cycle should be the one where we see the benefits of Treble in a meaningful and measurable manner.

[Get fresh tips and insight in your inbox every Friday with JR's Android Intelligence newsletter. Exclusive extras await!]

The gist of Treble, in case you haven't been paying attention, is that it takes the guts of Android — the hardware-specific stuff, like drivers that control how the software interacts with a device's processor and modem — and puts them into a totally isolated lower layer, separate from the rest of the operating system. That means manufacturers no longer have to wait for companies like Qualcomm to update all that lower-level code every time a new release comes along and can instead just jump right into their part of the updating process.

So how much of a difference should that evolution make? That's the million-dollar question. And as it turns out, we have an answer: Last year, I had the chance to interview Iliyan Malchev, a principal software engineer at Google and a key architect of the Project Treble program. One of my pressing questions for him was exactly how much of an impact Treble should have on the Android upgrade delivery process, from a real-world perspective. And his answer was pretty unambiguous.

By eliminating that initial part of the upgrade process, where device-makers had to work with silicon vendors and wait for them to update all the code related to the hardware-specific parts of the software, Treble should shave about a quarter of a year off the upgrade process, Malchev told me. That's how long the lower-level code updating typically took, in his estimation — and so without that part of the process in the picture, device-makers were essentially getting a three-month jump.

Well, guess what? I carefully track and measure Android device-makers' performance with upgrade deliveries every year (and for good reason). So now that we're past the six-month mark since Pie's release and my latest Android Upgrade Report Card is complete, we can use all of that data to see if Treble is working the way it should.

Spoiler alert: Prepare to be disappointed.

Android Pie and Project Treble: Current-gen flagship performance

Let's start by looking at how the most significant Android device-makers did when it comes to their current-gen flagship phones at the time of Pie's release. For the purpose of consistency and establishing a standard metric for comparison, my analysis focuses on the amount of time it took each company to get a finished version of Android 9 onto a model of its flagship phone that's readily available in the United States.

We can then compare that to each company's performance — under the same measurement — with the previous Android 8.0 Oreo upgrade, where Treble was not an active factor. And, in theory, we should see that the Pie rollout is a solid three months faster across the board. Right?

Yeah. About that:

JR (Click image to enlarge)

When it comes to current-gen flagships, Google was relatively consistent from Oreo to Pie — no huge surprise there and no cause for complaint, given its near-flawless starting point. HTC, meanwhile, did meaningfully worse with Pie than with Oreo: The company got the Oreo upgrade out in 99 days and to this day has not delivered Pie, meaning it's up to 191 days and counting.

LG, too, has already passed the point of meeting Treble's expected three-month window of improvement. The company took an embarrassing 274 days to get Oreo onto its current-gen U.S. flagship — so shaving three months or roughly 90 days off of that would mean that LG should have gotten Pie to owners of its current flagship a (still quite pitiful) 184 days after the software's release. That would have been last Wednesday, February 6th. And LG still hasn't done a thing.

The same can be said for Motorola, which took 124 days to get Oreo onto its current-gen flagship and has yet to deliver Pie to its highest-paying current customers.

Now, before you drown yourself in a river of virtual tears, there are a couple bits of positive news in this mountain of mediocrity: First, Samsung made a modest amount of improvement on its current-gen flagship upgrade delivery time with Pie, going from a godawful 213 days with Oreo to an only moderately awful 177 days with Pie. That's an improvement of 36 days — not nearly enough to count as a true Treble win, but at least something, right?

Still, when you zoom out and look at the bigger picture, it's tough to say Samsung accomplished much. With Nougat a year earlier, the company took 179 days to deliver the software to its then-current-gen flagship. What's more, Samsung's been on a downward spiral for quite a while now. It took 155 days for current-flagship delivery with Marshmallow and 106 days with Lollipop. So all that really happened with Pie is that Samsung basically went back to its Nougat level of performance but still failed to meet its level of performance from the previous two years before that — neither of which was even particularly impressive. So, yeah: It's hard to chalk that up as being a Treble-related victory.

The second promising nugget is a fair bit better: OnePlus cut down its current-flagship delivery time by 91 days from Oreo to Pie. That's almost exactly three months — right on target! Finally, a result that seems to indicate Treble worked as expected, at least in this one super-specific (and relatively small-scale) instance. But don't break out the bubbly and do a celebratory monkey-dance just yet: There's another side of the equation to consider.

Android Pie and Project Treble: Previous-gen flagship performance

It's easy to focus on current-gen flagships as a metric of Android manufacturers' performance with operating system upgrades — but in reality, the previous-gen phones are equally important. When you buy a flagship phone today, after all, you expect to have it supported for two years at a minimum. And yet, most device-makers treat their year-old customers as even more irrelevant afterthoughts.

To wit:

JR (Click image to enlarge)

With the sole exception of Google and OnePlus, not a single major Android device-maker (within the U.S.-specific focus of this analysis) has delivered Pie to its previous-gen flagship as of this writing — 191 days, well over half a year, since the software's release. There's simply no excuse for that, whether Project Treble is present or not.

Speaking of which, not all of these previous-gen devices were updated to support Treble ahead of the Pie rollout. But you know which ones were? The pair of flagships from OnePlus. And yet, while OnePlus did beat most of the pack by actually getting Pie onto its previous-gen products within that initial six-month period, it took longer to do that than it did with Oreo the year before — 47 days longer, so not just a rounding error. While OnePlus made appropriate improvements on its current-gen flagship, in other words, it kinda made up the difference by doing meaningfully worse with its previous-gen, Treble-supporting phones.

Pie and Treble: Putting it all in perspective

So what to make of all this? First, let me just say: I hate to be such a downer. Despite my ongoing skepticism about Treble, I had genuinely hoped the project would prove me wrong and show that with the right amount of help and motivation, we'd see some evidence that Android device-makers (beyond just Google) could treat their customers better and show us that maybe, even just a little, they care about more than making an initial hardware sale.

But looking at this data, it's hard not to reach the same conclusion I pointed out as a strong possibility at the get-go: Despite all of the sound logic behind what Google did with Project Treble — and all the validity in the time it shaves off the update process and the effect that should have on software delivery times — Treble can't address an increasingly undeniable reality within the Android ecosystem: the fact that most Android device-makers have little real motivation to make timely OS updates a priority.

The truth, of course, is that this shouldn't be entirely surprising. We've seen Google make numerous other efforts to encourage timely Android updates and make the process easier for manufacturers before, and as we've been reminded time and time again, most of the companies simply don't give a damn.

An effort like Treble can't serve as a fix for companies that just don't care

And really, at the risk of sounding callous, who can blame 'em? Aside from Google, whose business model revolves around software and services and providing an experience that encourages you to use the internet as much as possible, every Android device-maker makes its money by convincing you to upgrade your hardware at a regular interval. And on the surface, at least, providing timely and ongoing software improvements not only requires a significant amount of investment with no immediate return — it also makes your existing phone feel continually fresh and current. And that doesn't exactly inspire you to go out and buy a new model.

Ultimately, all Project Treble can do is cut the amount of time and cost required for a phone-maker to process and deliver an operating system upgrade — and in a situation where a manufacturer is already motivated, that could absolutely make a difference. But as positive of a step as it may be for Android in theory, an effort like Treble can't serve as a fix for companies that just don't care.

The only surefire solution is to educate ourselves about the various device-makers' attitudes and behaviors and then make our own future purchasing decisions with that knowledge in mind. (Screaming and vigorous desk-pounding can also be effective, from what I've heard — though in more of a short-term, stress release sort of sense. Hey, whatever works, right?)

Sign up for my weekly newsletter to get more practical tips, personal recommendations, and plain-English perspective on the news that matters.

[Android Intelligence videos at Computerworld]