We take a 20,000 word deep-dive on Android’s “foundational” upgrades.
Android 8.0 Oreo is the 26th version of the world’s most popular operating system. This year, Google’s mobile-and-everything-else OS hit two billion monthly active users—and that’s just counting phones and tablets. What can all those users expect from the new version? In an interview with Ars earlier this year, Android’s VP of engineering Dave Burke said that the 8.0 release would be about “foundation and fundamentals.” His team was guided by a single question: “What are we doing to Android to make sure Android is in a great place in the next 5 to 10 years?”
Take a closer look at Oreo and you really can see the focus on fundamentals. Google is revamping the notification system with a new layout, new controls, and a new color scheme. It’s taking responsibility for Android security with a Google-branded security solution. App background processing has been reined in, hopefully providing better battery life and more consistent performance. There’s even been some work done on Android’s perpetual update problem, with Project Treble allowing for easier update development and streaming updates allowing for easier installation by users. And, as with every release, more parts of Android get more modularized, with emojis and GPU driver updates now available without an OS update.Like its partnership with Nestle for Android 4.4 “KitKat,” Google is taking its alphabetical snack-themed codenames to the extreme with 8.0, and partnering with an actual snack company. This time Nabisco is sharing the “Oreo” brand with Google (We’ve yet to hear about any kind of monetary arrangement for this sort of thing). Google’s Eclipse-themed launch party was complete with custom Oreo cookies featuring an Android robot design and green filling.
Two billion users is a huge number, but with Android 8.0, Google shows that it still isn’t satisfied. A new initiative called “Android Go” targets the developing world, where cheap devices and limited access to data and power require taking a different look at how some parts of Android function.
Oreo will also serve as the base for three new Android form factors. It will be built into cars as “Android Automotive,” where Google works with car OEMs to integrate Android. Android 8.0 will also be the base OS for “Android Things,” an “Internet of things” (IoT) version of the OS designed to easily manage on embedded devices. Finally, Google’s virtual reality “Daydream” group will also launch a new form factor with Oreo—standalone VR headsets.
So, coming soon to your phone, your tablet, your watch, your TV, your car, your “things,” and your VR headset—it’s Android 8.0 Oreo. Let’s dive in.
Table of Contents
- Project Treble—Finally, real progress on the fragmentation problem
- HAL versioning and deprecation
- Working with SoC vendors
- A ROM revolution
- Isolating the media stack
- Android’s biggest re-architecture, ever
- Notifications—Android’s best feature gets better
- The new layout—and its awesome “By the Way” section
- The new colors and media notifications
- Snoozing notifications
- Notification Channels: Great for apps that have it, terrible for apps that don’t
- Icon badges and shortcuts
- The new ambient notification display
- The Great Background Processing Lockdown
- Mandatory JobScheduler
- RIP Implicit Broadcasts
- No more wakelocks, no silent background services
- (Somewhat) gracefully declining on older OSes
- Limiting scans for location and Wi-Fi
- A real API for floating apps
- Google Play Protect—Google says “please don’t install antivirus apps”
- Sideloading changes
- Security grab bag
- Emoji: New glyphs and an all-new design
- EmojiCompat and Downloadable fonts—updating emojis without a system update
- System UI improvements
- Adaptive icons—Shape shifting, animated icons
- A new widget picker
- Picture-in-Picture for phones and tablets
- Smart text selection and TensorFlow Lite
- Settings—A new theme, a new layout
- Streaming OS Updates—never fail an update due to storage space again
- Rescue Party
- Android Go—Scaling Android for the next billion users
- The OS in “Go” mode
- Google Play Services gets chopped up
- Apps get special “Go” versions and features
- Color management
- Physics-based animation and the new Easter Egg
- The new “SDCardFS” file system wrapper
- Grab Bag
- “Foundational” improvements address updates, security, speed, and battery life
- The Good
- The Bad
- The Ugly
Listing image by Google
Project Treble—Finally, real progress on the fragmentation problem
While Android’s scale is impressive, the vast majority of Android users won’t get to use new versions of the OS quickly—or at all. Google’s own Pixel phones get updates immediately, third-party flagships get the update in six months to a year, and everyone else gets the update when they throw out their existing handset and buy a new one. Android’s biggest problem is easily the ecosystem’s inability to ship updates quickly and efficiently to every device.
Google’s past “solutions” to the Android update problem have involved little action and lots of talk. In 2011, it asked OEMs to do better and announced the “Android Update Alliance“—a group of Android OEMs that totally-pinky-sweared to ship updates quicker. After the big announcement, exactly nothing changed; the group was never mentioned again. In 2016 Google reportedly ranked OEMs by update speed and showed the list around the ecosystem in an attempt to “shame” OEMs into doing a better job. Little happened.
Asking OEMs nicely has not improved things, so in Android Oreo, Google is finally implementing a technical change to Android that should help with updates. The effort—called “Project Treble”—modularizes the Android OS away from the drivers and other hardware-specific code. As Android’s Head of Engineering, Dave Burke, put it in his interview with Ars, “Today, it just costs too much to do an upgrade of Android. The amount of work and dependencies are too high.” The goal with Treble is to make it easier, faster, and—most importantly for OEMs—cheaper to pump out an Android update.
Once Google releases an Android update, getting it running on a specific device usually involves three major steps. First, SoC vendors like Qualcomm or Samsung’s Exynos division take the update and modify the release for that specific piece of hardware. Then OEMs like Samsung and LG theme the OS, rebranding Android with new icons, colors, and layouts, often changing core parts of the OS to add extra features and functionality. Finally, carriers test the new release, certify it for their networks, and send it out to users. (If you buy an unlocked device, you can skip the “carrier” step and get your update directly from the OEM.) Each step creates not only a delay, but also the opportunity for a company to end update support, leaving users with an abandoned piece of hardware.
Project Treble will help with that first step: relying on SoC vendors to get the release running on specific SoC hardware. Project Treble introduces a “Vendor Interface”—a standardized interface that sits between the OS and the hardware. As long as the SoC vendor plugs into the Vendor Interface and the OS plugs into the Vendor Interface, an upgrade to a new version of Android should “just work.” OEMs and carriers will still need to be involved in customizing the OS the rolling it out to users, but now the parties involved in an update can “parallelize” the work needed to get an update running—SoC code is no longer the “first” step that everyone else needs to wait on.
Google has a new set of tests, called the Vendor Test Suite (VTS), which ensures the Vendor Interface on a device is properly implemented and future proof. This is a hardware-focused analog to the Compatibility Test Suite (CTS), which ensures the Android app APIs are properly implemented on a device.
HAL versioning and deprecation
On the Android Developer’s Backstage Podcast (which is more-or-less Google’s “official” Android podcast) Iliyan Malchev, the head of Project Treble, dished out a good amount of detail about his pet project, saying “We haven’t been very formal about defining the boundary between the top [AOSP] and the bottom [Silicon Vendor] pieces of the OS.” With Treble, Malchev said that Google is “defining a set of interfaces, formally, in a structured and versioned manner, and committing to maintaining them.”
Treble isn’t just a single interface. HALs are broken down into “major subsystems” like the camera, audio, location, Wi-Fi, telephony, and other components. Malchev revealed there are “about 60” of these HALs in Oreo, with more to come in future releases. To allow for each subsystem to grow and evolve over time, each one gets a major and minor version number, and each release of Android will support a range of versions for each subsystem. For device types Google doesn’t officially support, there’s a “Vendor NDK” that allows vendors to make their own custom modular HALs.
The versioned subsystems mean that future device support will be partially dependant on Google and what subsystem versions it chooses to support or deprecate with each Android release. If your phone SoC only supports “Camera HAL version 1.3” and “Android 9.0” requires Camera HAL v1.4 and up, you’ll need to have your camera HAL upgraded before Android 9.0 will fully work on your device.
To help make HAL deprecation decisions like this, Google will have some sort of database containing all the HAL information for all the Treble Android devices out there. As Malchev put it on the podcast, “We will have a way to know, for every given device, exactly which HAL interface it implements at which version, and we’re going to have a way to look at the universe and see ‘Ok, what will happen if we drop support for this old version of this interface? How many devices will not be eligible to get a new upgrade?’ We’ll be able to make a data-driven decision for this sort of thing, and that’s great because previously we didn’t have that kind of data.”
Working with SoC vendors
Treble is about making lives easier for the thousands of Android OEMs out there, but nearly all the work was done with the SoC vendor community. “There are ten [silicon vendors] in the world or so.” Malchev said on the ADB Podcast, “Companies like Qualcomm, Mediatek, LSI, Spreadtrum, Intel. It is these companies that manufacture systems on a chip that underpin the actual Android devices. The fanout from these few companies to the thousands upon thousands of phones and tablets out there is massive… By working directly with these companies, we essentially solve the problem to a very, very large degree at the source, prior to that fanout.”
SoC vendors represent a much more focused group than working with OEMs. “We cannot possibly got to the hundreds upon hundreds of OEMs and say, ‘Hey, we’re doing this, can you change those things?'” Malchev explained. “We instead go to the silicon vendors of the world and work with them, so that, as the fanout happens, everyone benefits from the work we do.”
Malchev said that getting SoC vendors on board with Project Treble “took as much time as the engineering work” The work started in late 2015, and today, as part of Treble, Google has a dedicated SoC vendor outreach team. Malchev said that Google now has “teams in Taiwan, Seoul, San Diego dedicated to working with our major partners, with the silicon vendors in their own time zone and in their own back yard.” Malchev did not specify which silicon vendors Google had embedded teams with, but a quick comparison of SoC company headquarters against that list suggests that Google is in Taiwan for MediaTek, Seoul for Samsung, San Diego for Qualcomm.
A ROM revolution
While Treble is focused on official Android OEMs and making their lives easier when it comes to updates, as a side effect, Project Treble should be revolutionary for aftermarket Android ROM projects like Lineage OS (formerly CyanogenMod) and individual developers on places like the XDA Developers Forum. These groups take code from Google’s open source Android repository and make Android-based OSes themselves, usually offering upgrades to abandoned devices or conversions from skinned Android to a more stock look.
In the past, the bane of third-party ROM developers has been that AOSP has never been a complete, bootable software package for real life hardware. You’ve never been able to compile AOSP, slap it on a device, and have it work—you need all sorts of “binary blobs” from chip vendors to make things work. Usually, the ROM developers end up compiling AOSP and combining it with the official software, resulting in a janky mix of new AOSP code and old proprietary bits. Because Android never had a standardized interface between the hardware and software, updates would break all sorts of things, leading to issues with the camera, networking, and other components, or general performance problems.
Treble promises to change everything. Malchev says that Treble standardizes Android hardware support to such a degree that generic Android builds compiled from AOSP can boot and run on every Treble device. In fact, these “raw AOSP” builds are what will be used for some of the CTS testing Google requires all Android OEMs to pass in order to license the Google apps—it’s not just that they should work, they are required to work.
Treble’s modularity is enforced in Android’s partition layout too. AOSP code lives in the “System,” “Recovery,” and “Boot” partitions. The SoC vendor (usually Qualcomm) gets the “Vendor” and “Radio” partitions. The ODM/OEM (Samsung, LG, HTC, etc) gets the “Bootloader,” “ODM,” and “OEM” partitions for their code. These partitions have all existed before as options, but now things like a dedicated Vendor partition are mandatory for 8.0. Google’s documentationspells out the implications of this, saying “it should be possible to upgrade a device system.img from Android 8.0 to Android P while other images (such as vendor.img, odm.img, etc.) remain at Android 8.0. This modularity enables timely Android platform upgrades (such as monthly security updates) without requiring SoC/ODM partners to update SoC- and device-specific code.” It the ROM world, it means you should be able to flash a new system partition built from AOSP, while leaving everything else alone, and have a working system.
We’ll have to see how things exactly work in practice, but I’m expecting the ROM community to explode in popularity once Treble devices become more widespread. Custom ROMs shouldn’t need to be painstakingly hand-crafted for individual devices anymore—a single build should be able to cover multiple Treble devices from multiple manufacturers. Imagine the next time a major new version of Android is released—on Day One of the AOSP code drop, a single build (or a small handful of builds) could cover every Treble device with an unlocked bootloader, with a “download Android 9.0 here” link on XDA or some other technical website.
Users should also have more control over their devices, like being able to buy a skinned Android phone and immediately apply a fully-functional AOSP build, stripping away bloat and most of the annoying “customizations” that OEMs make. You can sort of do this today, but, again, the lack of a standard interface and needing to include binary blobs from the SoC vendor makes this a very iffy proposition. The standardization of Treble should ensure that everything will work.
Being able to compile AOSP, slap it on a device, and have it work will be a boon to home tinkerers, and on the official retail build side of things, could help to prolong the life of Android devices. If security is your goal, though, there’s still a need for involvement for the SoC vendors. There will inevitably be bugs and security flaws in the vendor code, and since that code doesn’t live in AOSP, vendor support will be required to fix those.
All of this ROM stuff depends on buying a phone with an unlocked bootloader, which lets users tinker with the software. Usually this just means buying an “unlocked” phone directly from an OEM, rather than a carrier.
Isolating the media stack
Another benefit of Project Treble: security! The separation of vendor code from the OS means the vendor code can run in a separate process. This also helps harden the media stack in Oreo. The media framework is a particularly important area after 2015’s “Stagefright” vulnerability, which exploited Android’s media player engine (which is actually called “Stagefright”) to remotely execute code.
On a Treble-enabled phone, the audio, camera, and DRM parts of the media stack have also been isolated in separate processes and sandboxes. Each part gets a new hardware abstraction layer (HAL) between the framework and kernel, too, so it should now be harder for a framework exploit in these areas to access the kernel.
Android’s biggest re-architecture, ever
Project Treble will make Android work more like the PC market, where a single version of an OS can run on multiple devices. It’s been obvious that Android needed to make a change like this for some time, but Treble was a massive project and counts for a lot of what makes Android 8.0 a full “1.0” update.
As Burke puts it, Treble “used up a huge amount of the engineering budget [for Android 8.0]. It was a lot of work and super deep surgery across every single interface of Android.” At the I/O Android Fireside Chat, Burke called Treble “probably the biggest re-architecture of Android since it started.” On the ADB Podcast, Romain Guy, Android’s Graphics lead and an Android engineer for the last 10 years, said “I don’t think there’s ever been something remotely even close to the complexity of Treble in terms of infrastructure change to the platform.” Malchev revealed Treble involved “upwards of 300 developers within Android engineering itself contributing to this, across 30 teams.”
As far as device support for Treble, the feature is a requirement for any new device that ships with Android 8.0. For upgrading devices, Treble is optional. So far the one and only upgrading device we know about is the Google Pixel, but Malchev said that “We are working with some companies to upgrade their flagship devices to O while Trebilizing them fully.”