Quest Pro supports Eye Tracked Foveated Rendering, but exactly how much does it improve performance?
If you’re not familiar with the term, eye tracked foveated rendering (ETFR) is a technique where only the region of the display you’re currently looking at is rendered in full resolution, thus freeing up performance since the rest is lower resolution. That extra performance can be used to improve the graphical fidelity of apps, or for higher base resolution.
You don’t notice the lower resolution in the periphery because the human eye itself can only see high resolution in the very center – the fovea. This is why you can’t read a page of text without moving your gaze. Believe it or not, that foveal area is only around 3 degrees wide.
ETFR has long been considered a “holy grail” for VR, because if your GPU really only had to render 3 degrees of your field of view in full resolution the performance benefit could be on the order of 20x. That would enable ultra-high resolution displays or incredibly detailed graphics. But in reality, achieving this would require perfect eye tracking with zero latency, an absurdly high display refresh rate, and a high-quality reconstruction algorithm so you wouldn’t notice flickering & shimmering.
Quest Pro is Meta’s first shipping headset with eye tracking. The end-to-end latency of this first-generation eye tracking tech is on the order of 50 milliseconds, and the display’s refresh rate maxes out at 90Hz. As such, the actual savings from its foveated rendering are nowhere near 20x.
Meta’s headsets have supported Fixed Foveated Rendering (FFR) – rendering the edges of the lens in lower resolution – since Oculus Go six years ago. In a talk this week given to developers, Meta detailed the exact performance benefits of Eye Tracked Foveated Rendering (ETFR) and compared it to FFR.
Both types of foveated rendering are enabled by developers on a per-app basis (though obviously both can’t be used at once). Developers are given three choices for the resolution reduction of the periphery: Level 1, Level 2, and Level 3. With ETFR Level 1 the periphery is rendered with 4x less pixels, while at Level 3 mostly with 16x less pixels.
The exact performance benefit of foveated rendering also depends on the app’s base resolution. The higher the resolution, the greater the savings.
In Meta’s performance test app, they found at default resolution FFR saves between 26% and 36% performance depending on the foveation level, while the new ETFR saves between 33% and 45%.
But at 1.5x the default resolution the savings were greater, with FFR delivering 34% to 43% and ETFR delivering 36% to 52%. That’s an up to 2x boost over using no foveation at all – but only a tiny benefit over FFR.
Of course, what really matters is what we don’t know yet: how noticeable are each of these ETFR levels? And how does this compare to how noticeable FFR is? This is what needs to be compared – not a given level of FFR to the same level of ETFR. This is something we’ll be testing in detail for our Quest Pro review.
On Quest 2 FFR Level 1 isn’t noticeable at all, but Level 3 definitely is. And since Quest Pro has lenses that are sharper both in the center and edges, FFR may be more noticeable than ever before, making ETFR even more advantageous.
On PlayStation VR2 the claimed performance benefit of foveated rendering is greater. Sony claims its FFR saves around 60%, while its ETFR saves around 72%. This is probably down to the vastly different GPU architectures of consoles and PC GPUs compared to mobile GPUs, as well as the higher resolution. It could also be down to differences in the eye tracking tech – Meta’s is internal while Sony uses Tobii’s.
Eye Tracking on both Quest Pro and PlayStation VR2 is optional for privacy reasons. But disabling it will obviously disable ETFR too, so apps will have to fall back to FFR.
