Professional sports is one potent sector of the entertainment industry that virtual and augmented reality has yet to truly crack.
Much of the focus has been on watching existing sports by placing you next to the action, like sitting court-side at a basketball game, with varying degrees of success.
Last weekend, Psychic VR debuted a new project called Air Race X, representing a bold vision for what sports might look like in immersive mixed reality formats.
The project isn’t centered around viewing an existing sport in VR, nor creating an entirely new sport for VR/AR mediums either. Instead, Air Race X attempts to take an existing professional sport and partially digitize it, merging real world data and athletic performance into a mixed reality immersive presentation.
While a fascinating concept, the results were ultimately more mixed in execution than reality. Here’s how it went down.
Getting Psychic
Air Race X was just one facet of a dazzling display of technology, concepts and in-development projects presented to me by Psychic VR Lab during a three-day media tour last week – our second visit to the company.
In 2019 we noted that Psychic was a incredibly bullish company when it came to the future of VR and AR. Four years later, that remains entirely true.
Jamie Feltham
During that previous visit, Psychic demonstrated the early capabilities of Styly, the company’s all-in-one creation and viewing platform for immersive content.
Conceptually akin to other creation tools like Unity – though infinitely less advanced in terms of features, infrastructure and scale – Styly is a platform where users can create scenes and experiences that can then be published to the platform’s public library and viewed by others on AR/VR devices. Creators can share their experiences via QR codes, which can be displayed at physical events like Air Race X for easy access on compatible devices.
Four years on, Psychic VR remains committed to developing and growing the Styly platform. It forms the basis for almost everything the company has in the works, aiming to support a wealth of different experiences at a variety of scales.
Standard VR/AR scenes created in Styly can be viewed anywhere, but the app also supports location-specific content that uses virtual positioning systems to anchor an experience around a physical location, such as Tokyo’s iconic Shibuya Crossing.
This was the plan for Air Race X, Psychic’s first iteration of a mixed reality air race event built using the Styly platform.
Bringing In Yoshi
Yoshi Muroya is a world championship-winning air pilot, previously positioned as high as 6th place in global rankings. On the second day of our media tour, Psychic takes us to Fukushima Sky Park to meet Yoshi and watch him complete an air race course, the data of which would be used the following day for the Air Race X event.
Traditional air racing competitions see pilots set times one-by-one across progressive runs of the same course, competing for the best time with as few penalties as possible. Traditionally, the pilots will all get together in one place to compete in global events such as the Red Bull World Championship.
However, Red Bull pulled out of the sport in 2019, discontinuing its World Championship series and stating that it “did not attract the level of outside interest as many other Red Bull events across the world.”
Four years on, Air Race X is Psychic’s bid to revive competitive air racing with a new format that also happens to reduce some of the logistical and economic commitments involved. Instead of gathering pilots in one place for an air race event, the pilots will be able to fly the proposed course – in this case mapped to fit over Shibuya – before the event, whenever and wherever they want. The data from their performances will be compiled and sent over to Psychic for use in Air Race X, which will see their flight paths digitally recreated and visible in mixed reality over Tokyo.
0:00
Yoshi Muroya takes to the skies at Fukushima Air Park. The data from the flight is used the next day in Air Race X, where his flight path is recreated and displayed over Tokyo’s Shibuya district.
We watched Yoshi complete an impressive display of twists, turns and various other maneuvers in the sky above Fukushima Sky Park. He was just one of several competitive pilots who flew the same course remotely from various cities around the globe. Their times and flight path data (copied to a USB plugged into the plane’s black box) are then converted for use in Air Race X – no international travel required.
On Your Marks… Get Set… Air Race?
After watching Yoshi complete his course at Fukushima on Saturday, the stage was set for Air Race X to take place in Tokyo the following day.
The competition was presented as a live event, available to watch online or across three physical locations in the Shibuya district. The race course itself was centered around the famous Shibuya Crossing, with the respective runs of four pilots recreated and displayed in mixed reality simultaneously across VR and AR devices. The digital nature of the planes also allows for previously impossible scenarios, such as two planes running the course at the same time without the risk of crashing into each other.
We observed the race from an in-person event held for media, investors, local government officials and other guests on the fifteenth floor of a building overlooking Shibuya Crossing. Yoshi Muroya was in attendance and spoke on stage before and after the event, alongside the three other competing international pilots who joined via a Zoom call.
0:00
One of the viewing options was a traditional sports feed simulcast to YouTube, which used a mixture of environmental cameras, digital overlays of the planes, and cockpit footage captured from pilots’ runs.
As the race began, prospective viewers had a few options. The physical screens around the room displayed a live feed (simulcast publicly on YouTube) overlaying the digital planes onto camera views positioned around the city at key points on the virtual course. For those seeking something more immersive, Pico 4E headsets were available for viewing the race in mixed reality while gazing out of the room’s windows via the headset’s passthrough camera.
A similar presentation was available in augmented reality through phones and tablets in the Styly app, allowing users to move their device around to track the digital planes and watch them fly in mobile AR.
Digital content displayed over the city included the planes themselves, course markers, plane path trails, digital billboards and celebratory fireworks. There were speakers that projected audio across the room as planes flew by and commentators provided analysis alongside the action, just like a live sport event.
Feeling Occluded
It was an impressive array of content, but there was one major problem evident from the minute the race started: the experience had absolutely no occlusion on any platform.
0:00
Screen recording capture of Air Race X running in augmented reality on an iPhone 12, showing planes and the digital course overlaid on Shibuya without occlusion.
This meant that walls, window frames and other buildings in the Shibuya surroundings were all displayed underneath the AR content. The planes, race course and all virtual objects appeared overlaid on the camera or passthrough view.
While Pico 4E headsets were available at the event for some guests, I wasn’t able to view the main race in VR (only AR). However, earlier in the day I tried out a demo of the race on Quest 3 in mixed reality at one of the other event locations.
Admittedly, Quest 3’s improved and balanced passthrough image provided for the most striking demonstration of the concept. However it nonetheless ran into the same problems with occlusion. Observing in mixed reality from an open rooftop, there were brief moments – when looking directly at the race area with nothing in the way – where the real promise of Air Race X came to life. But the illusion didn’t last long – the planes twisted around a building and should have disappeared behind it. Instead, they appeared overlaid in front of it.
I’m told that the Pico 4 version of the experience would have been equivalent in terms of the lack of occlusion.
While a single problem amid an admittedly impressive technical and logistical feat, the lack of any occlusion in Air Race X was ultimately difficult to overcome. It’s hard to feel excited about a virtual air race when the planes aren’t actually weaving around buildings. It simply doesn’t feel like the race is happening in physical space, which is the entire point. Without occlusion, the entire concept fails to come to fruition.
0:00
Air Race X displayed in AR on an iPad.
Part of the problem stems from hardware and software limitations. Most devices used for the experience, like standard model iPhones, iPads and Pico headsets, don’t include depth sensors, which can help devices and developers occlude nearby objects in mixed reality experiences. That said, experiences would likely still rely on system-level APIs to achieve this. As an example of how far out this technology is, Meta only just shipped an experimental depth API for mixed reality occlusion on Quest 3, a headset marketed entirely around mixed reality.
Even then, the level of occlusion and world understanding needed to pull off a concept like Air Race X goes far beyond just nearby objects – it would also need to occlude far away buildings and architecture well outside the range of a depth sensor.
Psychic is using geometric data of the cityscape, made available for use by the Japanese government, to correctly plan and position the content among the streets of Shibuya, but it’s unclear whether that same data could be used to help solve the large-scale occlusion problem.
There’s a strong argument to be made that, on a conceptual level, an experience like Air Race X shouldn’t even be enticed at the moment, given the lack of essential features needed to properly bring it to life. Instead, we were left with a fantastic concept executed on technology that is not nearly ready for something of this scale and complexity.
I reached out to Psychic after the event to inquire more about the lack of occlusion. Though occlusion is supported on the Styly platform, it was intentionally disabled for Air Race X, even on devices equipped with depth/LiDAR sensors (like the Pro series iPhones and iPads. Psychic found it caused wider misalignment issues with the overall experience when enabled for objects like window frames.
Psychic also stated that implementing such occlusion might restrict audience visibility and “create ‘hard-to-see areas’ and ‘non-visible areas’ for the airplane,” which I’d argue is the whole point.
Other Reservations
Psychic hopes this is the first of many Air Race X events, with dreams of the Shibuya Crossing course in Tokyo becoming a staple of the sport akin to iconic Formula 1 locations around the world. However, occlusion issues aside, there remain some other reservations about the experience as a whole.
This first Air Race X event was made possible through a combination of large investment, sponsorships and other forms of funding. However, future iterations will need to find an audience – and ways to monetize that audience – in order to remain viable.
The obvious path will be paid access, advertisements and sponsorships. The foundation for those is already in place – during the finals event, sponsors like Lexus had their logos displayed on the virtual billboards across the immersive course, for example.
However, it remains to be seen whether there’s genuine public interest in such an immersive experience, especially since a lack of interest is what Red Bull cited when it pulled support for the real thing four years ago.
Even so, the best part of any competitive sport is that it takes places live, in real time. Anything can happen in live sport, which fuels tension, excitement and authenticity. While the audience doesn’t know the result of an Air Race X event ahead of time, it is still by nature a pre-recorded event.
A betting system was in place for last weekend’s finals race, though there was no real money involved. Most forms of gambling are illegal in Japan, so users were just betting with free tokens and competing for prizes instead of cash. However, actual gambling could be a potential revenue stream for Air Race X events in other countries, even if it raises ethical questions when the competition is ultimately pre-recorded.
Getting Over The Line
At the end of the day, Yoshi Muroya took away first place in the Air Race X finals, awarded the trophy by the major of Shibuya. The result wasn’t a huge surprise, given I had heard raucous cheers from Yoshi’s team downstairs at Fukushima Air Park the day before, about 20 minutes after he completed his course for us.
Air Race X was just one of many in-development projects shown to me by Psychic VR, all of which seemingly funded by large swathes of continuing investment from local corporations and media companies.
While a interesting concept, it’s hard to see the bet paying off while the race isn’t properly displayed in the environment with full occlusion – and it might be a long time until that’s actually feasible.
However, Air Race X is one side of a coin – the other is CityXR. Another of Psychic’s ongoing projects, CityXR aims for Styly to transform city space into virtual immersive playgrounds for developers and users alike. While another bold vision from an unusual company, the project comes with its own set of difficulties that run parallel to those of Air Race X.
We’ll discuss CityXR more next week in a second piece covering our weekend in Tokyo exploring the future of augmented and virtual reality with Psychic VR Lab.
Disclosure: Flights, accommodation and food were paid by Psychic VR for the three-day media tour.
