Project Convey is relevant to creative data because our technology uses data gathering, processing and interpretation along with the creative design and use of emojis to better a communication and connection experience for those on the autism spectrum. We have found a way to utilize unique pieces of data in a new way so that a previously excluded group of people can have a better experience in video chat platforms that have become a day to day part of life since the pandemic.

Cox’s brand purpose is to help people have more meaningful connections. But to many of its customers, Cox is just the regional cable and internet provider they’re stuck with – a utility bill at best – not a brand that shares their values. We wanted to change that.

The objective: To strengthen Cox's relationship with its customers by improving overall brand sentiment and trust in a highly commoditized and negatively perceived space.

The challenge: Show people that Cox is not just a number you call when internet connection is down, but a brand you can count on when human connection is down.

The brief: Create a bold, real-world solution that brings Cox’s brand purpose to life by providing real, human connection wherever it’s needed.

Every day, millions of people connect through video chat. But for people on the spectrum, this connection is impossible, because many can’t read non-verbal cues, like facial expressions, which are key to connecting on a deeper level.

In response, Cox created Project Convey, a video chat prototype that helps people on the spectrum connect better with the person on the other side of the screen. Using AI, the prototype analyzes three inputs: facial expressions, speech, and tone, and in real-time, translates the emotion into an emoji. Cox worked with members of the autism community to design the tool, from UX and UI, to color, sound, and the emojis themselves. Those who trialed our prototype experienced more engaging and meaningful video calls than they had on other video chat platforms. The experience was not just improved for the person with ASD, the technology created a better experience for both parties.

When translating emotions into emojis, several different types of data are gathered. Facial expression data is gathered from the user’s facial 'skeleton' and is processed to read the user's facial expression. The words said are gathered and processed in two ways. The Audio data is sent through an AI that will extract the actual words and then send them to another AI that will then return an emotion/sentiment. That data is sent to our custom AI that analyzes the tone/frequency. Doing so returns additional tone data that provides an emotion.

The key to the data interpretation is the code that filters for confidence levels so that we display the correct emotion. Using weights and confidence levels for the speech, tone, and facial expressions combined, we then interpret and display the most likely emotion based on a custom algorithm that returns the end result.

In our case, data does not simply enhance the experience, it makes it possible. When a user joins a call we are getting data from his/her facial expressions, the words they say, and how they say those words, relying on three core data sources for a better experience. Data is gathered through the webcam and microphone. This data is then processed through our AI system to process the emotions. Once processed, the data from all sources is further processed from our custom algorithm that will return a single emotion for any given moment. That emotion is returned in the form of an emoji allowing users on the spectrum to better understand the emotion conveyed by the other user on the call. Analyzing facial expression and text data from a single 'non ASD user' allows us to provide meaningful and fun Emoji animations for users on the spectrum.

This is the highest viewed campaign at launch for Cox Communications. Within the first two weeks of the campaign we received:

3,570,343 video views

1,308,289 social engagements

4,520,432 impressions

The autism community also showed tremendous support for the prototype, expressing their happiness for this unique experience and talking about the exciting possibility of a future where people with autism can better connect with their loved ones on video chat. The client's interaction with this project and the resulting films have inspired them to possibly find a way to integrate this technology into their products and services in the future, in order to better include those with diverse abilities.