A new report from Bloomberg’s Mark Gurman provides more details on Apple’s upcoming mixed reality headset, including information on the core interaction mechanism.
According to Gurman, Apple’s headset will use eye tracking and hand tracking technology in tandem to drive a system that “allows the wearer to control the device by looking at an on-screen item – whether it’s a button, app icon or list entry – to select it.” Once selected with their gaze, users can then “pinch their thumb and index finger together to activate the task – without the need to hold anything.”
While other headsets on the market already offer varying combinations of eye and hand tracking technology, Apple’s would deliver a system-level interaction mechanism that utilizes both technologies at the same time. Quest 2 allows users to control the user interface with hand tracking, but doesn’t include eye tracking technology. Meanwhile PSVR 2 will include eye tracking technology, which will be used for menu selection and navigation in certain games, but Sony has no announced plans to add hand tracking. Quest Pro includes both eye and hand tracking technology, but the Quest operating system does not use them in tandem to handle interactions or selections. AR systems use the input systems in various ways as well, and some developers have experimented with the combination on Quest Pro. Eye tracking is also optional on Quest Pro and PSVR 2 for privacy reasons.
Gurman’s report also indicates that Apple’s headset will include a ‘Digital Crown’, similar to the dial found on the side of the Apple Watch, which users can physically turn to switch between VR and AR passthrough mode. In terms of software, Gurman indicates that the main interface will be “nearly identical” to the iOS and iPadOS home screen, with “a grid of icons that can be reorganized” and widgets for various apps.
Previous reports from Bloomberg indicated that Apple was “working on a VR client for FaceTime with face tracking driving Memoji avatars” for the headset. In this latest report, Gurman now suggests that this FaceTime client will “realistically render a user’s face and full body in virtual reality.” This allows two Apple headset users to communicate with realistic avatars as if they are in the same room.
However, this requires “immense processing power” and so the feature will be limited to one-on-one chats – sessions with multiple users will see others displayed as an icon or a Memoji. While Meta has presented plenty of impressive research and prototypes for realistic avatars, dubbed Codec Avatars, it has yet to ship the technology to consumers.
Gurman also reports that “immersive video watching” will be integral to the headset, with Apple in discussions with “half a dozen” media companies including Disney and Dolby about developing VR content for the headset.
Previously, reports indicated that Apple was going back and forth between integrating the headset’s battery into the headband or separating it out of the headset and into a tethered battery pack worn by the user. This debate was spurred by concerns related to heat and comfort.
According to Gurman’s report this week, Apple has now “made the decision to offload the battery from inside of the headset to an external pack.” The pack will rest in the user’s pocket, connected via a cable, and is “roughly the size of two iPhone Pro Maxes stacked on top of each other.” The headset’s 6-inch tall, half-inch thick battery pack would reportedly last about two hours.
Gurman also claims that Apple expects to sell about 1 million headset units in the first year, but is not planning on making a profit from the first version. Earlier this month, reports indicate that Apple is already planning a cheaper headset model to release later down the line. Apple is also reportedly planning to create “a store within a store” at its retail outlets, where consumers can come to try out the Apple headset in-person.
