Tutorial clips depicting Quest 3 automatic room scanning were discovered in the Quest firmware.
Firmware sleuth Samulia discovered the videos in the Public Test Channel build of Quest system software v55. These clips would be shown to the user when starting up Quest 3 for the first time.
Quest 3 was officially announced last week. Meta hasn’t yet revealed the full specifications yet, but said the visor is 40% thinner, the GPU is more than twice as powerful, and the color passthrough is “better and more advanced” than Quest Pro thanks to dual 4 megapixel cameras and a depth sensor.
Meta also hinted Quest 3 is capable of “intelligently understanding and responding to objects in your physical space”, but hasn’t yet gone into detail about what that actually means.
One of the leaked clips depicts a user walking around their room, looking at walls and furniture. A beam coming from the headset – likely representing the depth sensor – then generates a blue mesh around the objects.
The clip then skips forward to when the headset has seen the entire room, showing a ‘Finish Scan’ interface as all objects have been meshed.
HTC’s Vive XR Elite also has a depth sensor and a beta feature using it for automatic room setup. But XR Elite’s room scanning is limited to planar surfaces like walls, ceilings, and desks. The Quest 3 clips show precise meshing of complex 3D objects. However, we don’t yet know whether the level of detail shown in the clip is just illustrative or truly representative of Quest 3’s capabilities.
Generating a 3D room mesh is useful for both virtual and mixed reality.
To use VR on existing Quest headsets you manually draw Guardian bounds on your floor via a passthrough view of your real room. An eight foot boundary wall will then show when you get near those bounds. But this limits you to using VR only within completely empty spaces with no furniture of any height, even if you could reach over it.
A Meta product roadmap leaked to The Verge in March mentioned Quest 3 having a feature called “Smart Guardian”. One of the leaked clips appears to show this – the user in VR is kept aware of nearby furniture and made aware when approaching real walls. Compared to traditional Guardian, you could leverage more of your room for your hands in VR, including the space above beds and tables.
For mixed reality, having a room mesh allows virtual objects to interact with real objects, including being occluded by (appearing behind) furniture, being positioned on surfaces like tables, and bouncing off walls. One of the leaked clips depicts this in action.
Using room-aware mixed reality on existing Quests requires manually marking out rectangular cuboids for your walls and furniture. This arduous process means almost no Quest developers today actually leverage the scene graph features. Quest Pro was originally supposed to have a depth sensor, but it was dropped just a few months before release. Had it not been, it’s possible Quest Pro could have had these same room scanning features Quest 3 now seems set to introduce.
David Heaney
Full details of Quest 3’s room understanding and mixed reality will be revealed at Connect 2023 on September 27.
