As the lead singer in the Smashing Pumpkins, Billy Corgan won seven MTV Video Music Awards in 1996. He had always been intrigued by the visual side of music from album covers to iconography to stage sets and music videos. Virtual reality had sparked Corgan’s curiosity when we first met with him at the end of 2016. He was fueled by an interest in using the new medium to connect with his fans and capture performances in ways that had never been done before.

As he began work on his first solo album, we partnered with co-directors Rob Ruffler and David Liu and Viacom to begin to make Corgan’s vision a reality with “Aeronaut,” a three-minute-and forty-five-second journey to the center of Corgan’s world via an up-close and personal performance by him at the piano.

“Aeronaut VR” is a three-and-a-half-minute room-scale experience designed for Smashing Pumpkins’ William Patrick Corgan’s single, “Aeronaut.” The unique content was not only designed as a fully immersive and participatory VR experience, but a 2D video was filmed within the VR experience using Cinemachine, a new software tool from Unity. The 2D video filmed inside the 3D VR world was imagined by San Francisco artist and filmmaker Danny Bittman using Tilt Brush and Blocks from Google, and brought to life by the joint Viacom and Isobar team. By capturing Corgan’s three-and-a-half-minute performance in volumetric video at Microsoft’s Mixed Reality Capture Studios, Isobar, Viacom, and Bittman were able to place Corgan inside of Bittman’s world, and use Unity to take viewers inside that world as well.

At the Mixed Reality Capture Studios, we filmed the performance on a stage using 106 cameras, then used computer vision algorithms to create textured 3D surfaces of everything in view. We further processed the resulting holographic video to provide some consistency in the meshes over time as well as compressing it for easier transmission and viewing. The team then worked inside of the VR experience by collaborating, reviewing designs, and discussing interactions. The two experiences were created over the course of a year. In November 2017, the 2D video was launched, coinciding with an event that shared a preview of the VR experience. It was also previewed at Business Insider’s Ignition 2017 Conference and at CES 2018 in the DAN Clubhouse. The world premiere of “Aeronaut VR” took place in March, at SXSW's Virtual Cinema and will be available to the public later this year. “Aeronaut VR” been described as the highest quality representation of volumetric video capture and playback in the marketplace.