Oculus Quest 2 is heading toward mainstream adoption and Facebook’s executives in charge of VR strategy have dropped hints surrounding a “Quest Pro.”
In early 2021, Facebook’s head of VR and AR Andrew Bosworth said “Quest 2 is going to be in the market for a long while” but also referred to the idea of a “Quest Pro” as a forthcoming VR headset that’s likely to “introduce new functionality” to its platform. Quest Pro won’t be out in 2021, with Quest 2 geared to be “the best bet for the most accessible way to get into VR and have a great experience,” Bosworth said.
In a separate interview, Facebook CEO Mark Zuckerberg also said “One of the things I’m really excited about for future versions is getting eye tracking and face tracking in.” Zuckerberg also added “the biggest things that we’re very focused on now are: how do you pack more sensors, to create a better social experience, into the device?”
Zuckerberg also suggested he’s likely to continue to reinvest revenue from its VR department into lowering the entry level price of system ownership. Meanwhile, for the future, Facebook is thinking about what’s necessary for “a better social experience.”
AR, Hand Tracking And Controllers
Facebook is facing a range of options for future hardware. If the company is going to ship a “Quest Pro”, the question is exactly which of these possibilities get ranked highest by executives.
For Zuckerberg, VR and AR represents the next generation of personal computers and a way for Facebook to free itself of reliance on Apple, Google, and the app stores they maintain. It’s a big deal, building from Apple’s dominance with iPhone from 2007 onward to the 2013 failure of Facebook’s phone through to the 2014 purchase of Oculus and 2018 announcement of Oculus Quest by Zuckerberg himself. If Facebook can develop a headset that features an unmatchable price, first-to-market functionality, or some combination of the two, it could be a bit like the game Civilization when you develop a key technology like “gunpowder” before the rest of the world. With Quest 2 starting at $299, Facebook is already off to a strong start on price and Quest Pro could serve as a testing ground for more advanced features that’ll be costlier (at least at first) for Facebook to have manufactured.
“Nearly every time we might be content with the value we have provided people so far we instead risk it all to try to do even more,” Bosworth wrote in a late 2020 blog post.
Some of Facebook’s next steps — consider these low-hanging fruit — may be hinted at by features on Quest/Quest 2 that have been added or improved after initial release.
First, there’s an AR passthrough view of your physical surroundings that’s gotten new features on a consistent basis. Second, Facebook added hand tracking to its headsets as an optional input method by co-opting the four cameras designed for controller and head tracking. AR passthrough and hand tracking are essentially limited by the current hardware — right now you only get a black and white view of your physical surroundings and, while the hand tracking constantly improves, you still can’t have your hands blocking one another too much or it stops tracking.
A “Pro” version of Oculus Quest could push either of these features forward by upgrading the sensors in the headset. Critically, improved hand tracking would make it easier to get more done in the headset without controllers handy and a more realistic view of your physical environment could make it easier to wear the headset for longer periods of time. These improvements could be fundamental in making VR more appealing to use for work paired with a tracked keyboard and mouse.
Facebook Reality Labs Executive Advisor John Carmack recently commented that, over the course of multiple years, physical controllers are likely to become a “separable” feature and “we want to be able to have a controller-free SKU in the future where we rely just on hand tracking for people that want to use keyboard and mouse.” Facebook recently launched a couple of demo experiences to showcase how developers could support the same interactions with open-air gestures as they do with controllers.
“You can have a set of techniques and development that are going to put something out there that has a more featureful presence,” Facebook’s head of VR and AR Andrew Bosworth said in a conversation with Carmack. “And this is going to go at maybe lower volume in terms of the number of units, but also advances the state of the art, inspires developers, I think unlocks a lot more use cases. And then as that technology matures, finds its way into these scale units that get out to so many people.”
Will there be a controller-free version of Quest Pro? The timing likely won’t be right for it. Right now, both customers and developers expect Quests to come with controllers, and in fact, building controllers with better tracking or more satisfying haptics might also be inspirational to developers and push the state of the art forward in standalone headsets. Facebook doesn’t want to confuse buyers who expect to play Beat Saber feeling like they’re holding swords in their hands when they buy a headset with “Quest” in the name — at least not near term. Improving the controllers, while simultaneously improving hand tracking, seems perfectly in line with what Facebook would likely want to accomplish with a “Quest Pro.”
Social Connection: Eye Tracking And Face Tracking
Facebook has been researching face-sensing VR headsets for a long time. In 2019, the company revealed one approach showing a headset with additional cameras facing the user to capture intricate expressions. Meanwhile competitors, like HTC, HP, and Pico, have VR headsets on the market right now with varying kinds of eye-tracking or face-sensing capabilities.
Separate lines of research at Facebook investigate hyper-realistic”codec” avatars — but even animating current cartoon-like avatars with more realistic expressions and eye contact would make a major difference to the sense of actually occupying the same space as a friend or colleague even if you are physically distant.
Facebook offered to reimburse tens of thousands of its employees for the purchase of Oculus Quest 2, its head of VR takes meetings in headset on a weekly basis, and the company allows its employees to work from home. Taken altogether, Facebook is effectively “dogfooding” VR to staff such that the team will become intimately aware of exactly what improvements are needed to make this experience better.
Put another way, any technology that makes Facebook’s employees work more effectively with one another, no matter their physical distance, is exactly the same technology executives want to sell to consumers. The ability to convey nuanced expression to your friends or colleagues anywhere they are in the world while wearing a headset might be a critical part of that story. Does average time in the headset increase if you are able to smile naturally or make eye contact with your friends or colleagues who are also in a headset? Facebook’s employees are likely going to be among the first to know the answer to that question.
More than that, the addition of accurate, dependable, system-level eye tracking in a standalone VR headset may also offer updated tools for developers that could dramatically improve some game mechanics. Throwing objects or pulling faraway objects toward you, for example, might feel far more satisfying and reliable when the software is informed about where you are looking.
Foveated rendering could also be the key to unlocking other improvements. Eye tracking that’s accurate and fast enough for foveated rendering can allow a VR headset to manage resources more efficiently by drawing the greatest visual detail directly where you’re looking.
Field Of View, Optics And Weight
Last year, UploadVR’s David Heaney wrote an editorial that correctly outlined Facebook focusing in on a single headset — Quest 2. What we expect Facebook to do in a Quest Pro, then, is take everything this platform does and add to it.
Does that mean a wider field of view? What about a higher resolution?
While eye-tracking could be fundamental for enabling realistic eye contact in VR, it could also be key to enabling foveated rendering wherein the area of the screen that you’re directly looking at could feature enhanced detail. As for field of view, though, in a March 2021 question and answer session on Instagram, Bosworth outlined the challenging trade-offs.
“I actually think a taller field of view could be more impactful than even a wider field of view, kind of counter-intuitively, in terms of immersion. However, resolution also feels like a huge opportunity for things like reading text and expanding use cases of VR,” Bosworth said. “You’ve got field of view which trades off against weight, which trades off against acuity, which trades off against the amount of brightness and the power of the display system that you’re using, so it’s really tightly constrained so we’re just trying to balance all of those together.”
Notably, a reliable supply chain analyst indicated in June this year that Facebook ordered new, more expensive, lenses for use in a new headset.
Long Term Technologies And Multiple Facebook Devices
Research into new optical architectures at Facebook could one day add field of view or sharper objects viewed within your arm’s length, while other research considers super flat and hyper lightweight optics. Can either path transfer into efficient manufacturing? Can they survive if you drop your headset on a hard surface? What other priorities are more important?
“If you’re building a pair of glasses that need to look like normal glasses, you need to have the system be so tightly optimized so you can basically do all the computation that you would expect from a modern computer, but do it on someone’s face within a thermal envelope and a power envelope that can last all day long. So that’s a very big challenge,” Facebook CEO Mark Zuckerberg said in a recent Q&A session.
CTRL Labs was bought by Facebook and offers a wrist-based approach to input. Facebook also offers a “Hey Facebook” voice assistant and the company is pursuing a smartwatch. Could such a device also enhance a Quest Pro?
For comparison, Apple’s wrist-based computer received a software update recently with a feature that uses “built-in motion sensors like the gyroscope and accelerometer, along with the optical heart rate sensor and on-device machine learning, Apple Watch can detect subtle differences in muscle movement and tendon activity, which lets users navigate a cursor on the display through a series of hand gestures, like a pinch or a clench.” Apple’s iPad, meanwhile, gained support for “third-party eye-tracking devices, making it possible for people to control iPad using just their eyes.”
Facebook’s Portal devices offer video calling from any room but don’t integrate with Oculus headsets in any way, at least not as of this writing. Over time, cross-over functionality between Facebook’s family of devices might offer added convenience and compelling features when used with Oculus headsets. In the case of Portal, for example, perhaps it could enable full-body tracking or seamless mixed reality video?
Oculus Quest Pro Release Timing
All Facebook has said is that its ambitions for a “Quest Pro” headset won’t be available to consumers in 2021. Near term, the company is looking to build out its software platform on the Quest 2 while assuring potential buyers that the device won’t be replaced soon.
We’d be surprised, though, if Facebook didn’t try to get such a higher end standalone headset out to buyers by the end of 2022. That’s absolute speculation on our part — the chip shortage, pandemic, quality control, and problematic manufacturing could all knock back expectations. But Facebook started defining this standalone product category in 2018 with Go, found the sweet spot with Quest in 2019, and improved on price and weight with Quest 2 in 2020. In 2021, Facebook is focusing on building out the software platform with new game releases and monthly updates to the Oculus Quest system software. So far, that’s meant the addition of first-party wireless PC VR support, improved hand tracking, and the first pieces of an Infinite Office that will enable working from anywhere. And not to mention competitors like Sony and Apple may be ready to jump in with dramatic new features in the near future.
Facebook is building toward something greater than Quest 2 and Quest Pro may be the place where we see the next pieces of that journey.
