Bias is insidious by nature. Our brief was to use the latest AI technology to make inequities not only visible, but actionable, for marketers. With the knowledge that audits are more effective when they leverage reportable metrics, we set out to build a tool to help marketers benchmark based on gender and age representation in their market and vertical. We then leveraged the same criteria to expose not only diversity representation gaps, but also troubling portrayals of women in Super Bowl ads. Next, we aimed to raise the profile of data bias as a research domain for the marketing practitioner audience with an Introduction to Machine Learning Fairness. Finally, we produced three visual stories: an intersectional analysis of race representation in advertising, a showcase of standout inclusive campaign work, and a guide to metrics-driven inclusive employment practices, all of which included actionable steps for directly encoding inclusion into marketing plans.

Telling a compelling data-driven story was key to our mission. Our interactive tool begins with a video highlighting the biggest takeaway: that despite a persistent global gap in gender representation, more people watch ads that feature at least as many women as men. In the first interactive module, users can compare Gender & Screen Time, Gender & Speaking Time, and Age & Screen time across 11 verticals and 51 markets. They can also choose to weight the videos by how many times they were watched, making it possible to account for real brand experiences. Next, they can compare 4.25 years of representation data across up to five markets to find out who was seen and heard in each. Three bar charts offer a global overview of content watched and created, followed by a five-year comparison of men and women’s ages on screen paired with insights about portrayals of gender and

The interactive report analyzed 2.7M videos uploaded by advertisers on YouTube between January 2015 and March 2019. Videos received 550B+ views during that period, helping us determine who was featured most often in the most-watched videos. To measure representation, we used a series of deep neural net models to detect and classify the appearance of human faces in video frames, outputting three classification models: perceived gender expression, age, and visual speech detection. The values are summarized at the video, industry, and market levels. Supplementing the automated analysis, we worked with a GDI research team to identify trends and assess portrayals in the most-watched videos from the report, along with Super Bowl advertising from 2014-2018. Before the study, the team conducted codebook development, training, and tests for reliability. Inter-rater reliability was achieved in terms of both absolute agreement and Cohen’s Kappa measures. Participatory design workshops formed the ML fairness guide.

Since it was published on the homepage of Think with Google, this series helped more than 50K unique users consider inclusion in their campaigns. The site has millions of subscribers in 78 countries across 12 business verticals. The YouTube Ads Gender Representation Report helped more than 23K unique users learn about the quality of gender representation in their market and vertical. Our Marketer’s Guide to Machine Learning Fairness helped nearly 50K unique users confront the dangers of data bias and monitor their models accordingly. Meanwhile, the data feature covering gender representation in Super Bowl ads saw a 70.6% engagement rate, testifying to the will of advertisers to educate themselves about this issue. Our research partner, GDI, regularly uses the interactive report to onboard new clients. We are in the process of rolling out a pilot to bring our global data-driven visual stories to markets in APAC and EMEA.