People with physical handicaps who have inconveniences with their hands or feet have various limitations with regards to“individual expression.”This is especially the case when it comes to playing a musical instrument in which case proves challenging to the normal person.

The “Eye Play the Piano” system that we developed is the world’s first innovative system that enables a user to perform music without using their hands, arms or feet, but only with their eyes.

We began by utilizing an eye-tracking system to recognize the direction of eyes through the virtual reality headset.

The system detects the user’s eye movements to select the keys, and the blinks to play the corresponding note.

About the system:

1. The virtual reality headset (FOVE) perceives the users eye movement through the mounted eye tracking technology.

2. The preferred note can be triggered by looking at the panel within the interface, which has been developed to be played by sight.

3. The user blinks on the preferred panels within the interface to trigger the selected note, which is then conveyed to the connected piano via MIDI signal.

4. Furthermore, tilting down of the head plays the role of the piano pedal, which lengthens the selected note.

The “Eye Play the Piano” system can be taught to the user anywhere as long as there is the exclusive application, headset display and a piano. The concept model of the application was developed in December of 2014 and is scheduled to become an open source application in December of 2015.

*FOVE refers to the eye tracking head mount display device. FOVE is the technology support member of this project. http://fove-inc.com/

Eye Play the Piano started as a concept model put together by The University of Tsukuba’s Special Needs-School and the technology team but currently The Ministry of Education is considering the introduction into Special Needs Education.

If the application is approved, the open source application of Eye Play the Piano will be completed by December of 2015 and is scheduled to be included in the curriculum in Japanese Special Needs-Schools from April of 2016.

In addition, the charity funding that was put into effect during the project has already collected over one million (1,000,000) yen from countries all across the world. These funds are expected be used to donate the tracking head mount display device to Special Needs-Schools and also fund the updating of the application.