There are many new advances in AI technology available to developers now. Tensorflow.js and Posenet, for example, let you interact with your computer using your entire body and more. Meanwhile, more than 56 million people are living with a disability. It’s critical that we design uses of AI technology in an inclusive way, together with collaborators from the accessibility community.

Our goal was to create inspiring and useful applications of the latest AI technology to demonstrate how the benefits of AI could be brought to everyone. To do this we would work with creators and allies in the accessibility community to give a glimpse of what’s possible when applying the latest AI advances like Posenet and Tensorflow.js. We would demonstrate how you can interact with your computer using all kind of inputs and outputs - voice, body, keyboard, and more, to make creative tools as accessible as possible.

From the typewriter to recorded sound, accessibility researchers often end up making technology better for everyone. Together, with creators and allies in the accessibility community, we made Creatability, a set of 8 experiments that explore how creative tools – drawing, music and more – can be made more accessible using web and AI technology. They’re just a start, we hope these experiments inspire others to unleash their inner artist regardless of ability. That’s why we’re open sourcing the code and creating helpful guides as starting points for people to create their own projects.

An exciting design challenge, as an accessibility-focused project, was to make the design as universal as possible, accommodating all inputs and outputs. So anyone can interact with these experiments in lots of ways: voice, body, keyboard, switch devices, and more. And output was equally important, for example making sure spoken and sound outputs were thoughtfully designed for visually impaired users.

What’s really exciting is that even though it made the design process even more challenging and difficult, thinking carefully about all users from the beginning yielded even better, simpler designs. And that’s one of our key design learnings that we hope can extend to all of our future projects and inspire other projects.

The project was received really successfully by the public. For example, Venture Beat praised its engaging interactivity: “a music-composing tool that lets you create tunes by moving your face to a digital canvas that translates sights and sounds into sketches and a music visualizer tool that mimics the effects of synesthesia.”

One of the most important parts of our launch strategies was highlighting our partners, not Google. For example, Claire Kearney-Volpe who leads the NYU Ability Project, and Jay Alan Zimmerman, a deaf composer who works in New York, were ecstatic about the visibility that this launch brought to their critical work in designing projects with accessibility in mind.