Surfing isn’t just a notoriously difficult sport to learn, but also to judge. Waves are highly variable, the flow or style of a surfer’s manoeuvre is subjective, and bias from judges towards particular surfers can occur too.

Artificial intelligence can solve this, but is often perceived as a threat – something that will replace humans. Instead of replacing judges, Huey supplements their work by providing accurate and fast metrics on things that are hard to see for humans: a surfer’s speed, height, rotation, etc. Only a person can be a judge, but Huey can see what they might have missed.

Machine Learning can produce incredible insights, but often the stories around its value are prosaic and unrelatable. Huey is about a unique human story – ML working in tandem with people to remove biases and improve a sport.

Huey composed of existing ML tools, like human pose estimation and pattern recognition, as well as traditional computer vision tasks, like optical flow for speed, and depth perception for distance.

The initial working prototype – built with Tensorflow – could predict surfers’ maneuvers and speed from archival footage of past surf competitions. To that limited dataset, hours of surfing footage from a wave pool were added, to create robust quality for the ML model.

As COVID19 prevented Huey’s first live judging event from happening, that has become a live demo instead. The dataset keeps being added to over time, which makes Huey progressively even more robust.

Huey aims to remove unconscious bias from judges in surfing. Surfers of a particular age, gender, status looks or even nationality might be judged beyond their performance. This tool creates metrics for evaluating the skill of a surfer on aspects that are inarguable. Judges can compare exact values on how high a surfer rose from a wave, how vertical they were and how much they rotated – and close the gaps of uncertainty our unconscious biases can hide in.

The tool itself is the first of its kind, and has been built on the Tensorflow platform with training data gathered specifically for this project.

Huey is the result of a small group of passionate creatives, engineers and surfers with a limited budget and resources. It ultimately seeks to erase bias from sports, one wave at a time.

19th July 2019 - Proof of concept complete

21st Sept 2019 - Money secured from stakeholders to carry out Feasibility study

14th of November 2019 - Feasibility and testing

21st of March 2020 - Wave Pool Event

August - December 2020 - UI design and GFX exploration

December 2021- March 2022 - Develop and test judges’ interface and app

22nd of March 2022 - Stab High Surf Competition wave pool launch

Our ML model has been trained from hundreds of hours of surfing footage. The training footage combined cameras from three fixed positions, allowing us to gather data and predict a surfer’s position in 3D. The footage was annotated with keypoints across a surfer’s body and surfboard as part of the training process. This allows us to predict the pose and position of both the surfer and their surfboard.

Our tool can take a camera feed from a live event, and in real-time detect a surfer’s height from the wave, their verticalness, rotation, projection and their completion and type of maneuver.

Project Huey is built on top of TensorFlow with training and processing run on Google Cloud.

The initial proof of concept, training data library, the feasibility study, and prototype has been completed. We are currently working on the MVP and exploring the tool’s UI and visualisations for the judges and viewers for the Stab High Surfing event in March 2022.

While Project Huey was developed for surf competition judging, the scalability into other, and mostly less complex sports, is the logical next step. Besides the implied long-term outcome of ridding sports judging from biases, Huey aims to start a bigger conversation. A conversation about what humans are good at, what machines are good at - and how we can maximise these strengths.

We found ourselves in an advertising landscape where robots dance, machines paint and algorithms write poems. Project Huey looks to innovate on where we choose to innovate and how we use technology for what. We hope that Project Huey will become a role model on ethical, productive and truly beneficial use of technology that does what humans can’t - instead of replacing their subjectivity.

Investment received: USD 398k