From seeing the game, to hearing the game. We developed Voice Watch, the world’s first play-by-play commentary AI, to enable visually impaired people to follow live sports. The technology converts various kinds of stats data into audio, providing commentary for visually impaired spectators at stadiums without live commentary. Voice Watch automatically generates live commentary to create a new sports spectator experience: “hearing” the game.

Voice Watch consists of 3 different AIs: an object recognition AI, a sign detection AI, and a speech frame AI. The object recognition AI serves as the visually impaired person’s eyes, using cameras to distinguish different teams and athletes, and grasp the game unfolding before the audience. The sign detection AI analyzes real-time stats data to swiftly identify signs of major changes in the game and predict what happens next. For the speech frame AI, we analyzed play-by-play audio from Japan’s most renowned professional commentator, distilling this expertise into unique speech frames capable of reproducing the commentary phrases that excite spectators. By bringing these elements together, Voice Watch generates live commentary. In 2023, we also developed the new feature: “personalized commentary”. Voice Watch is now capable of delivering different patterns of live commentaries for each athlete and team, enabling the audience to focus on their favorite team. Voice Watch is keep evolving to realize a new spectator experience.