Meta showed research which turns any flat surface into a virtual keyboard.
Mark Zuckerberg posted a video to his personal Instagram profile showing clips of himself and Meta CTO Andrew Bosworth trying a surface-locked virtual keyboard in a Quest 2 headset.
Zuckerberg claims he was able to achieve around 100 words per minute, while Bosworth says he reached 119 words per minute.
The average person types at around 40 words per minute on a traditional keyboard, whereas professional typists reach between 70 and 120 words per minute depending on their skill level.
Text input in VR and AR today is cumbersome, and significantly slower than on PCs and smartphones. Floating virtual keyboards require awkwardly holding your hands up and pressing one key at a time, providing no haptic feedback and preventing you from resting your wrist. Quest headsets support pairing a physical bluetooth keyboard for full speed typing, and some models are tracked, but this means you have to carry around a separate device larger than the headset itself.
If future headsets could turn any flat surface into a virtual keyboard, it would bring partial haptic feedback and allow you to rest your wrists as with physical keyboards, without the need to carry around a physical keyboard.
Developers can technically already build surface-locked virtual keyboards on Quest today, by using hand tracking and getting the user to tap the surface to calibrate its position. But in practice, even the slightest deviation of the virtual surface height from the real surface results in false key presses.
Meta didn’t share yet exactly how its research solves this issue. However, fiducial markers can be seen on the desk in the clip. If the system is preprogrammed with the exact dimensions of these markers, this may act as a robust dynamic calibration system.
Quest 3 will add a depth sensor to a Meta headset for the first time, and a leaked setup clip shows the headset generating a 3D mesh of its environment. If this mesh is precise enough, Quest 3 could potentially eventually support this kind of virtual keyboard. For now though, Meta is solely describing this as research, not a demo of a near term product experience.
Meta will likely show off more of its VR and AR research at Meta Connect, which starts September 27 this year.
