The Android XR Operating System for Mixed Reality Headsets and Smart Glasses by Google

Last Thursday, Google was trying another approach at making headset work. Google launched Android XR, a new operating system optimised for extended reality (XR) devices, as well as compatibility for its Gemini AI assistant. It is likely to be released with upcoming mixed reality headsets and smart glasses, and Google says it will enable features based on artificial intelligence (AI), augmented reality (AR), and virtual reality. In 2023, Apple introduced visionOS, a dedicated operating system for the Apple Vision Pro, which supports both headgear and iPad apps. 

The recent introduction of the Android XR, from Google, a new operating system developed exclusively for “extended reality” devices such as headsets and glasses. It is collaborating with Samsung and a variety of other hardware makers to produce those headsets and glasses, has made the new version of Android available to developers, and intends to begin releasing XR products next year.

Rethink what is possible with Android XR. It is a new OS for headsets and glasses that blends digital info with the real world. New ways to watch, work and explore are coming soon. Discover more → https://t.co/Zw6EDtwVsH #AndroidXR pic.twitter.com/amlE3ZHT1E

— Android (@Android) December 12, 2024

The Verge’s Victoria Song had the opportunity to test a few demos and prototypes; be sure to read her tale. They still don’t know exactly how Android XR will work or how it will differ from the Android on your phone. Google is building immersive XR versions of apps such as Maps, Photos, and YouTube, as well as a version of Chrome that allows multi-window multitasking in the browser. It will also enable existing Play Store phone and tablet apps, same as Apple does with iPad apps on the Vision Pro.

Of course, the entire experience revolves around Google’s Gemini AI. Google has been attempting to crack the headset market for more than a decade — there was Glass, Cardboard, and Daydream, all of which had good ideas but didn’t go anywhere — and the company believes AI is the key to making the user experience work. “We believe a digital assistant integrated with your XR experience is the killer app for the form factor, just as email or texting were for smartphones,” said Sameer Samat, Google’s head of the Android ecosystem, in a press event ahead of the launch. As Gemini grows more multimodal, capable of capturing and creating audio and video, glasses and headsets become increasingly useful.

Perhaps the most intriguing aspect is the use of the name “XR” for the operating system. There are a million titles and acronyms for this domain, including virtual reality, augmented reality, mixed reality, extended reality, and others, all of which refer to different but overlapping concepts. XR is likely the broadest of the names, which appears to be why Google selected it. “When we say extended reality or XR,” he explained, “we’re really talking about a whole spectrum of experiences, from virtual reality to augmented reality and everything in between.”

According to Google, the first developer preview of Android XR, which was launched on Thursday, says it will allow developers to create apps and games for upcoming devices running the new operating system. It already supports Android development tools such as Android Studio, Jetpack Compose, ARCore, OpenXR, and Unity.

Users of the new Android XR operating system will be able to use Google’s Gemini AI assistant, which will include capabilities tailored to XR experiences. This implies that users will be able to talk to the assistant and ask it questions about items and locations in their field of view, as well as use the Circle to Search feature, which is available on select Android phones, to perform a visual lookup with a gesture.

Google says, in addition to these AI features, that its in-house applications, such as YouTube, Google Photos, and Google TV, will be redesigned to work on a virtual display, similar to how Apple added support for watching content on a larger, immersive screen visible while wearing the Apple Vision Pro headset.

Meanwhile, Google Maps will support a new Immersive View function, and users may use Google Chrome to explore the web on a much larger virtual screen with gesture navigation.

Google has also launched Project Moohan, the first gadget to run Android XR. Samsung will release the XR headset in 2025. It is intended to compete with the Apple Vision Pro, which was released earlier this year in certain areas, with a price tag of $3,499.

Google envisions headsets that can effortlessly transition between virtual and real worlds, similar to the Vision Pro, as well as smart glasses that serve as an always-on companion. It is also interested in audio-only gadgets, such as Ray-Ban Meta Smart Glasses. Some items may be stand-alone, while others may function as a phone accessory. We’ll see if Google builds its own XR hardware, but it’s definitely aiming to support a wide range of devices.

Android XR is still in its early phases, and most developers will only begin to receive the software and hardware required to build for the new operating system. But Google is attempting to move rapidly next year: a device dubbed Moohan, which it is developing with Samsung, is reportedly set to ship next year. In some respects, Android XR represents the conclusion of Google’s bets on AI, the larger Android ecosystem, and the wearable future of technology. All of those bets are about to be put to the test: whether or not anyone will place them.

Smart glasses (or AR glasses) are thought to be the future of XR technology, capable of providing the majority of today’s functionality without the use of heavy components. Google says it is already preparing for these technologies with Android XR and will begin real-world testing for prototype glasses running the new OS soon.

Android XR previews on these prototype devices shared by the company show some futuristic features, such as the ability to view messages in a small popup at the bottom of the wearer’s field of view, or the ability to see turn-by-turn navigation directions while using Google Maps, as well as a small circular map displayed in the same location. According to the business, these classes might also automatically deliver text translation and virtual instruction through AR technology.