Virtual Art Sessions is a mixed reality experiment that explores the different creative possibilities introduced by Tilt Brush, an entirely new category of creative storytelling. The experiment recreates the experiences of six world renowned artists using Tilt Brush for the very first time. Explorable on any device, visitors to the site can observe the artists develop their virtual reality creations from start to finish from any angle or perspective, literally putting them into the shoes of the artist.

The six artists came from diverse artistic backgrounds and had no prior VR experience. Their unique approaches to this medium became apparent when watching them work. To bring that perspective to the web, we decided to recreate not just the artwork, but also the rendering of the artist inside their Tilt Brush creation, giving users a unique look into their process.

We recorded RGB and volumetric data of the artists’ physical forms, positional data of their headset and controllers, and 3D geometric data of their artwork. Using that data, we recreated their sessions on the web, giving users 360° viewing freedom.

Virtual Art Sessions runs entirely on your phone or desktop web browser. It makes use of Google Chrome’s V8 Javascript engine to render large volumes of data in real time. It relies on WebM video and WebGL to produce the 360° representations of the artists and artwork – the artist portrayals alone require the browser to draw over 200,000 points at 30 times a second.

For HTC Vive users, we created an experimental WebVR mode letting them step directly inside the experience.

We released the code open source, giving developers insight into how we built the project and letting them create their own remixes with the data.