The world’s 200 million visually impaired people have been excluded from the excitement of live sports. Visual impairment makes it difficult to enjoy the action live with others. This stems from the information gap with regular spectators. Even as more people with disabilities participate in sports through the Paralympics, accessibility in sports viewing is still lacking. Envisioning a barrier-free future where people of all abilities can enjoy the same live sports events, we launched this project to assist the visually impaired in enjoying motorsport. From seeing the game, to hearing the game. We developed Voice Watch, the world’s first play-by-play commentary AI, to enable visually impaired people to follow live races. The technology converts various kinds of race data into audio, providing commentary for visually impaired spectators at circuits without live commentary. Voice Watch automatically generates live commentary to create a new motorsport spectator experience: “hearing” the race.

We introduced Voice Watch into Japan’s leading endurance racing series, Super Taikyu, during events held at the Suzuka Circuit and Okayama International Circuit, both of which host F1 races. Using an object identification model trained on images of team cars, and iPhones among the spectator seats as network cameras, Voice Watch analyzed the unfolding scenes in real-time. It also gathered numerical information from the driving data monitor available on the event website, which is updated with each team’s times and positions. By recording all stats throughout the race, comparing them over time, and calculating the differences between teams, Voice Watch forecasted over 150 potential future outcomes, such as overtaking and changes in the standings and included them in the commentary. The result was a highly accurate live commentary that allowed visually impaired spectators to enjoy races at venues without live commentary.

Of the visually impaired individuals who tried Voice Watch, 80% said they were able to enjoy the race and would like to attend other sporting events if Voice Watch was available. These results indicate that we were successful in creating a new spectator experience for the visually impaired. Media exposure reached over 2 million. The positive response led to Voice Watch also being adopted at Kodomo Suporinpikku, a children’s sporting event in Tokyo attended by some 1,000 people. In the 50m sprint for elementary school children, a visually impaired father was able to enjoy his child’s run via Voice Watch’s commentary. Many other sporting events, from minor sports tournaments to regional qualifiers, have a passionate following but lack live commentary. In the future, we aim to move beyond motorsport to create a world where everyone can enjoy all kinds of live sporting events, regardless of physical ability.