Now is a time when we are limited in our ability to feel the physical presence of other people or to even go outdoors. YAKUSHIMA TREASURE was looking for a way to deliver its music.

However, they thought it would be difficult to surpass the quality of a live performance by simply distributing video.

We were required to create a completely new shared experience of live performance that was different from the real live experience, and different from the streaming live video experience of the past.

Now is a time when we are limited in our ability to feel the physical presence of other people or to even go outdoors. YAKUSHIMA TREASURE was looking for a way to deliver its music.

However, they thought it would be difficult to surpass the quality of a live performance by simply distributing video.

We were required to create a completely new shared experience of live performance that was different from the real live experience, and different from the streaming live video experience of the past.

This work allowed audiences to experience an entirely new way of taking in a concert that is different from both regular video captured with a camera and from a scene observed with the human eye.

Rather than recording video of this musical performance about the cycle of rebirth, we captured a world "seen by spirits" in the form of data.

The goal was to create a new way to experience live performance, and we decided to choose the technology necessary to make this happen.

We thoroughly discussed the definition of "live performance"

-A live performance must be well established.

-It should be a one-time live performance without re-recording.

-Interactions that enhance simultaneous viewing.

For these purposes, we prepared surround sound, musical instruments, and costumes in a banyan forest, and placed importance on performing a high-quality live performance.

We adopted a scanning method that allows us to record the entire live performance in one go, using nine depth-sensing cameras to capture the movement and a 360° camera to scan the space.

The scanned data is then reconstructed to create a viewing experience that makes you feel as if you and others are present in the live space.

By rethinking the essence of live performance and using technology, we have succeeded in

360°LiDAR Scan

The entire forest was scanned as 3D point cloud data with a LiDAR sensor.

Photogrammetry

Photogrammetry employing over 1,500 high-resolution photographs enabled expression of colors and texture as though from a painting.

Volumetric Video

Nine depth cameras were used. They recorded 3D data that captured absolutely all of the movement and expression from a one-time-only performance.

Spatial Audio

360°microphones in a surround sound system using eight speakers recorded the audio.

Space and colors momentarily cut out.

Motion and sound captured as time progresses.

A live performance reconstructed in a combination of different perspectives on the world.

Free from the physical constraints of camera angles, the audience could approach very close to the artist, fly throughout the forest, and view the concert from any perspective.

The data on the audience members' perspectives was recorded as a human presence and shared, thus enabling them to sense each other's presence.

This work offering an entirely new live experience attracted coverage by numerous media outlets.

GIZMODE

"A new form of live concert that inquires into the post-COVID world."

NEWREEL

"A contemplative, interactive concert occurring on the web."

Qetic

"The 15-minute live performance began with the 'Mogaribune,' a song about the journey to the afterlife, and reached its climax with 'Higashi,' a song portraying the rebirth of the soul."

In addition, many viewers were talking about this new live viewing experience on social networking sites, and YAKUSHIMA TREASURE was successful in bringing the performance to them.