With the launch of Android 8.0 last year, Google released Project Treble into the world. Treble was one of Android's biggest engineering projects ever, modularizing the Android operating system away from the hardware and greatly reducing the amount of work needed to update a device. The goal here is nothing short of fixing Android's continual fragmentation problem, and now, six months later, it seems like the plan is actually working.

At Google I/O this year, you could see signs of the Treble revolution all over the show. The Android P beta launched, but it wasn't just on Google's own Pixel devices—for the first time ever, an Android Developer Preview launched simultaneously on devices from Google, Nokia, OnePlus, Xiaomi, Essential, Vivo, Sony, and Oppo, all thanks to Project Treble compatibility. Even car makers—some of the slowest adopters of technology on Earth—were on the Project Treble train. Dodge and Volvo both had prototype cars running Android as the infotainment system, and both were running Android P.

As is becoming custom for our annual trip to Google I/O, we were able to sit down with some core members of the Android Team: Iliyan Malchev, the head of Project Treble, and Dave Burke, Android's VP of engineering. (We quoted Iliyan Malchev a million times during the Android 8.0 review, so it was nice to get information from him first hand, and Dave Burke has been through the Ars interview gauntlet several times now.) And through this lengthy chat, we got a better understanding of what life is like now that Project Treble is seeing some uptake from OEMs.

What follows is a transcript with some of the interview lightly edited for clarity. For a fuller perspective, we've also included some topical background comments in italics.

Proving out Project Treble with Android P

First up, a recap of what's going on with Treble right now.

Iliyan Malchev: With Treble, the operating system has separated to the adaptation layers that tailor down to the hardware. And that's still the case, but the devil is in the details. There's a ton of nuance that we still need to get right, and this is what we've focused on with this [Android P] release. What is the case today—and I think that gets overlooked by a lot of the press on Treble—is that any device that is preloaded with Google's apps, any device that launches with Oreo or subsequent releases, must work smoothly with a binary image of Android that we built for certification purposes.

This image isn't a product. The intent is not to launch this, but the idea is, by requiring that this "golden image" run on everything out there, we cresate a centripetal force that pushes our partners ever so gently toward not changing Android in ways that aren't really meaningful to their bottom lines. We finished that technical work with Android P this year, and we started working with silicon manufacturers.

Dave Burke: Yeah, I think this is actually one of the biggest shifts. After finishing the technical work, there was the actual process of engagement of working with the silicon vendors, which is a big change.

Malchev: We have teams in Taipei, Seoul, and San Diego that work with Mediatek, Samsung Semiconductor, and with Qualcomm, respectively. We took our work and we applied it to their BSPs [Board Support Package]. Qualcomm and everyone else will take AOSP [Android Open Source Project] as we publish it, incorporate it into that BSP, and then give that in turn to the device manufacturers. That BSP really is where devices start. They don't start with AOSP, because AOSP is, by itself, not a complete product.

Like Malchev says, the open source parts of Android (AOSP) just consist of operating-system code and won't actually run on a piece of hardware. A Board Support Package (BSP) combines AOSP with all the other code needed to make Android run on a piece of hardware. This is usually things like the Linux kernel and drivers. Like the diagram shows, Google publishes AOSP, SoC vendors like Qualcomm combine AOSP with a specific version of the Android Linux kernel and drivers to create a BSP, and OEMs load the BSP onto a phone, adding hardware and software customizations.

Malchev: It was the standing issues with the BSPs that we tackled, because if we release AOSP in August and then Qualcomm does three months of work to release the BSP, then it's already the end of the year. If you're a device manufacturer, you're basically out of luck. So we absorbed all of this work to make it simultaneous with the internal Android development.

Burke: I think one example is Telecom and Telephony. And how many changes did we upstream? There were a lot.

Malchev: Right, so in addition to all of this, we also started making AOSP more of a fleshed-out product by upstreaming 150 features that our partners had to maintain out-of-tree. That's very important to them because the ongoing costs of maintaining all of this kind of code is massive.

"Upstream" for Android is the Linux Kernel. Google maintains its own fork of Linux for Android, but the two are closer today than they have ever been. Correction: Nevermind, "Upstream" here means "upstream of OEMs," which is AOSP. Malchev is referring to including third-party phone features in AOSP so OEMs don't have to manage as much code.

Burke: And the other big shift is just our workflow. The silicon vendors—the three in this case for the chipsets that we're supporting—are actually committing code into AOSP. All the companies are working together on one repository. That's a huge change in how we operate, because we used to build the OS to a certain point; then the silicon vendor would take it to a certain point; then the device maker would take it to a certain point, all serialized. Now, we can work in parallel with the silicon vendors on the same codebase. When we have release candidate of P, they have what they call "CS," or "commercial sampling," and they're ready at the same time, which is a huge difference.

Burke and Malchev are describing the process we saw at I/O with the launch of the Android P Beta. Google, Qualcomm, and other SoC vendors and OEMs all have a hand in bringing a new build of Android to a device. And, before, the "serialized" development process meant each company had to finish before the next company could start. With a stable interface between the hardware and software parts of Android, everyone can work simultaneously to port a new version of Android to a piece of hardware.

For an idea of what this is like now, Google was even nice enough to send along this quote from Essential, which gives us a timeline of how long the Android P beta took to port.

"Making sure our Essential users have the very latest OS updates is incredibly important to us. Once we'd enforced vendor and system separation on Oreo using Treble, our small team got Android P running on PH-1 in only 3 days." --Rebecca Zavin, VP Software, Essential

The Essential Phone is probably the best case scenario for an update: a Project Treble-compatible device with stock Android, so there won't be a ton of software modification required. Three days for a port still seems like an incredible amount of progress, considering many OEMs take months to update.

Malchev: And that's really the part of the iceberg that's beneath the surface. Dave just described cooperating with these silicon manufacturers; it means we both needed to change our development practices dramatically. Qualcomm has a 6,000 person-strong engineering department that works on Android. For Project Treble, they started working jointly on our infrastructure with us. So we're both keeping their BSP up to date, and we make sure it matures as Android itself does. And so that's a massive, massive change. And we're doing the same with MediaTek and with Samsung Semiconductor.