From the typewriter to recorded sound, accessibility researchers often end up making technology better for everyone. Together, with creators and allies in the accessibility community, we made Creatability, a set of 8 experiments that explore how creative tools – drawing, music and more – can be made more accessible using web and AI technology. They’re just a start, we hope these experiments inspire others to unleash their inner artist regardless of ability. That’s why we’re open sourcing the code and creating helpful guides as starting points for people to create their own projects.

There are many new advances in AI technology available to developers now. Tensorflow.js and Posenet, for example, let you interact with your computer using your entire body and more. Meanwhile, more than 56 million people are living with a disability. It’s critical that we design uses of AI technology in an inclusive way, together with collaborators from the accessibility community.

Our goal was to create inspiring and useful applications of the latest AI technology to demonstrate how the benefits of AI could be brought to everyone. To do this we would work with creators and allies in the accessibility community to give a glimpse of what’s possible when applying the latest AI advances like Posenet and Tensorflow.js. We would demonstrate how you can interact with your computer using all kind of inputs and outputs - voice, body, keyboard, and more, to make creative tools as accessible and universal as possible.

From the typewriter to recorded sound, accessibility researchers often end up making technology better for everyone. Together, with creators and allies in the accessibility community, we made Creatability, a set of 8 experiments that explore how creative tools – drawing, music and more – can be made more accessible using web and AI technology. They’re just a start, we hope these experiments inspire others to unleash their inner artist regardless of ability. That’s why we’re open sourcing the code and creating helpful guides as starting points for people to create their own projects.

February 17 2018 - Development kickoff for Posenet

April 12 2018 - Design and prototype kickoff for experiments

May 7 2018 - Posenet launched with a blog post on Tensorflow.js medium and open sourced github repo made available on Tensorflow.js github

July 10 2018 - Production/full development kick off

August 1 2018 - Sound design kick off

August 28 2018 - Video production kick off

October 25 2018 - Website with 8 Experiments, github open source repo and video is launched

Creatability uses posenet, a lightweight AI model running on TensorFlow.js, developed and open sourced by The Creative Lab that lets anyone with a decent webcam-equipped desktop or phone experience pose estimation technology right from within a web browser. Posenet runs in all browsers except for IE and we’ve open sourced the model so Javascript developers can tinker and use this technology with just a few lines of code. But ore importantly, Posenet helps preserve user privacy since Tensorflow.js runs in the browser, no pose data ever leaves a user’s computer. Posenet is continuously being improved for more accuracy and speed and has influenced similar web based pose technologies such as BodyPix: real-time person segmentation in the browser using Tensorfliw.js.

The outcome and goal of Creatability was to highlight our co-creators from the accessibility community, how important accessible design is for all products and how AI affects everyone.

By open-sourcing the pose estimation model and code from the project, developers are able to take the work we’ve done and use it for their own projects, allowing the project to scale in many different ways. Creatability showcases how important it is to work with people from the community you are building for, allowing them to have a voice and decision making power in the design and development process. By leading the way with this type of thinking and design, we hope the greater tech industry will start designing with communities rather than designing for communities and that this project will influence the tech industry to keep all communities at top of mind when creating products, not just primary users.