A new app available for iOS and Android places a virtual pianist at your real life piano, allowing you to watch them play a full piece from start to finish as if they’re in the room with you.
Back in February, I wrote about VRTuos, an Oculus Quest app that used hand tracking to sync up a virtual piano in VR with your real life piano. While very basic and still somewhat buggy, it essentially allowed you to learn a simple piano on a real piano using VR and hand tracking. While it wasn’t perfect, I managed to teach myself the Tetris theme using the app and came away quite impressed.
Now, many months later, an AR equivalent has arrived — AR Pianist is an astonishing app that uses augmented reality to stage a virtual AR concert on a real life piano, and it’s pretty convincing too.
The setup process is simple — as you can see in the video below, you calibrate the camera for AR and the press the ‘Place’ button once you have your piano in view and all the keys visible. Like magic, a pianist avatar will be sitting at your piano, and with the press of a button they will run through an entire piece.
Nothing is actually happening in real lie — the app does not connect to electronic pianos and no actual playing is taking place. It’s entirely virtual — the AR pianist creates the (almost) perfect illusion someone is sitting at your piano and playing a piece for you.
You can move around and view the pianist from different angles as they play, and even get right up close to the keys, watching from the perspective of the piano stool as if you were playing the piece yourself. Of course, it’s not perfect — the more keys are out of view, the less aligned the AR keys become — but it’s close enough not to matter.
The app is free to download — there are some free tracks included and others that are locked behind a subscription paywall. The track featured in the video above, ‘Fur Elise Jam’, is a new track available on the app, and is a collaboration with the well-known musicians and YouTube channel, The Piano Guys. The standard avatar is replaced by one of Jon Schmidt, which is convincingly lifelike when viewed up close.
If you don’t own a piano in real life, there is also an option to place a full AR piano in your space as well. The app also has non-AR functionality, such as displaying songs in a synthesia-type manner and using your microphone to listen for the notes as you play them.
