Electromyography signals of people living with ALS has been overlooked for a long time. The discovery that electromyography signals can be detected from the body of them and the invention of a digital avatar as a second body using this finding are major breakthroughs. It also has great potential to support the future participation of ALS patients in society in today's ever-developing digital world of VR, XR and the Metaverse.

ALS is an incurable disease in which motor neurons that move the body age, and the whole body gradually becomes immobile. No effective treatment has yet been established, and the average life expectancy after the onset of the disease is said to be 3 to 5 years. However, consciousness, the five senses, and intelligence remain healthy.

One of their greatest wishes is to regain the physicality they had before the onset of ALS.

This project aims to fulfill thier wishes by utilizing technologies to create a new tool to help people with ALS to communicate with others by physical expressions and achive the future where they will not have to give up their physicality anymore to increase their wellbeing.

ALS is a disease first described in 1869 by French neurologist Jean-Martin Charcot (1825-1893). Charcot gave the name ALS to a disease in which pathological changes were found only in the spinal cord area related to movement among patients who had difficulty exercising and lost muscle mass. Currently, about 350,000 people are battling this disease worldwide. No effective treatment has yet been established.

The human body generates minute electric currents when muscles are moved. This is called an electromyography(EMG) signal, and although MASA cannot raise his arms because he is a patient with ALS, we found that this EMG signal occurs inside his body as well. We then developed a UI and system that connects the EMG signals in the neck, arms, and legs to the digital avatar. This allowed MASA to regain its physicality in the digital world.

March 31, 2023 -First Electromyography Signal Measurement Experiment

At the NTT Research Institute in Yokosuka City, Japan, the experiment was conducted to detect EMG signals from MASA's body. Succeeded in detecting EMG signals in accordance with his will from both hands, legs and neck.

April 15, 2023 - Operation System Prototype Development

Started developing operating systems in a metaverse space using an EMG sensor. While exchanging opinions with MASA, we produced several prototypes and verified the easiest system for ALS users to use.

July 12, 2023 - Prototype Verification

Verified the avatar operation in a metaverse space by Masa's EMG signals using the completed prototype. A UI for a metaverse that can be freely operated by ALS patients was completed.

July 20, 2023 - DJ System Development

The first place to show this system was set in the Ars Electronica Festival 2023, and we started development of DJ performance systems using EMG signals. Produces Avatar, motion, and space for DJ performance while discussing with MASA.

September 8, 2023 - The First DJ Performance

At the Ars Electronica Festival 2023, the world's first DJ performance with the digital avatar operated by an ALS patient using EMG signals was achieved.

November 22, 2023 - The Second DJ Performance

At Tate Modern in the UK, we had the second DJ performance.

This system measures the EMG signals of both arms, both legs, and two positions of the neck of ALS patients and allows them to manipulate an avatar in a metaverse space according to the signals. The EMG signals on the user's right and left arms are linked to the movements of the avatar's right and left arms. The user's right and left legs are linked to the movements associated with walking and jumping. The neck EMG signals are measured at two locations on the left and right side and are linked to the orientation of the avatar. By combining these six EMG signal measurements and eye gaze input, we have developed a system capable of performing eight patterns of avatar movement. These patterns are selected by eye gaze input and can be added without limit. The DJ metaverse space for the performance with this avatar was developed on Unity and can be integrated into various existing Unity projects. This makes it possible to customize the performance to the artist and the event. In the future, we will continue to develop the system with the goal of extending it to more general platforms such as Fortnite, Roblox, Decentraland, etc.

The project received approx. 30 million JPY of investment from NTT, and they comitted to co-development of the avatar system. The usage of electromyography signal is also expanded to gaming where people with disabilities can play with other people regardless of disabilities.

The system we have developed helps people with disabilities express themselves and communicate freely with others. Since there are millions of people with physical disabilities in the world, there are millions of potential users of the system. By unleashing untapped creativity, we can make the world a better place. To make this a reality, the project should be expanded to involve more players. This would even have the potential to create a new market where people with disabilities can work regardless of their disabilities.