For the approximately 300 million people worldwide who are living with blindness or low vision, the experience of broadcast sport is severely compromised.

Televised coverage relies overwhelmingly on visual information, while radio commentary is often too slow to give fans a true sense of the live action.

Our objective was to invent and establish a new standard for broadcast sports accessibility.

Action Audio is an innovative accessibility audio service for sports broadcasts that allows people with blindness and low vision to experience the sports they love, in real time, alongside their sighted friends and family.

After receiving world-wide acclaim from the blind sports community since its pilot at the 2021 Australian Open finals, the service was launched in 2022 to cover all 58 centre court matches with an additional Google assistant voice feature to further improve ease of accessibility.

While exploring spatialized sound in mixed reality, we discovered an opportunity to pair 3D sound with data to create soundscapes that held experiential and informational meaning.

Applying some of Neuroscientist David Eagleman’s ideas of ‘sensory replacement’ – that is, counteracting the loss of one sense by ‘feeding in’ additional information through another – we developed a concept to use ball and player tracking technology and sound design to give visually impaired audiences a ‘sense’ for the progress of a live sporting match.

Connecting with the Blind Tennis Association and Blind Sports & Recreation Victoria, we developed a data gathering and co-design process to create a set of sound design principles that could be applied to a tennis match that would enhance the spectating experience for blind and low vision sports fans.

For many blind and low vision tennis fans, Action Audio’s implementation at the Australian Open represented the first time they were able to follow a live broadcast tennis match with friends, without assistance. To further improve ease of accessibility in 2022, listeners can now experience Action Audio together with the live radio commentary of each game through Google Assistant.

Action Audio is a live ball tracking sound design system, synced with the live radio broadcast - AO Radio, available via the website and via google voice commands for ease of access.

Over a two year process of co-design and prototyping, Action Audio was piloted in 2021. The service was scaled and officially launched in 2022 with additional voice assistant features.

Before Action Audio the experience was a series of sneaker squeaks, ball sounds and the occasional grunt. Feedback on Action Audio from a blind person confirms, “This is the final piece that i’m missing”.

Using existing technology was an important part of the creation of Action Audio, and an important part of this was utilising the Hawkeye ball-tracking data already in use at major tennis tournaments.

Hawkeye data is a raw stream of location information of both the ball and players. The technical development of Action Audio involved creating a fast and intuitive data processing algorithm to translate this stream of raw positional data to information-rich audio, specifically targeted at sports fans with blindness or low vision.

This was done using scalable compute clusters on Amazon AWS, and custom Python code running on our production machines in the central broadcast compound at the event.

For the first time, blind fans could experience tennis independently.

Audience impact 22:

- 58 live games

- 11,967 total listeners 111 different countries

- 5,663.8 total listener hours

- 28m35s average session length

Media impact 22:

- 115 pieces of media coverage, The New York Times, AOL UK News & BBC

Click News

- 4.47 million estimated social media views

- 4.73 billion combined total of publication-wide audience figures