Innovation > Innovation

DOT GO. THE FIRST CUSTOMIZABLE OBJECT RECOGNITION APP FOR THE VISUALLY IMPAIRED

SERVICEPLAN GERMANY, Munchen / DOT / 2021

Awards:

Shortlisted Cannes Lions
CampaignCampaignLayout(opens in a new tab)
Supporting Images
Supporting Images
Supporting Images

Overview

Credits

Overview

Why is this work relevant for Innovation?

Until now, camera-based assistive technology for the visually impaired relied on proprietary data, making them limited, expensive and fragmented. Dot Go closes this gap leveraging the combined power of open-source object recognition models with the latest LiDAR integration in iPhone models.

For the first time, the visually impaired can not only detect any object in their space but also they have the freedom to interact with them in any way they desire. Dot Go is the first in a new category of accessible, adaptable and hands-free solutions for everyday use.

Background

As a blind and visually impaired person it’s challenging to live an active, independent lifestyle. Camera-based assistive technology can help, however existing solutions are limiting: They all rely on proprietary data and closed systems, designed for fixed use cases rather than increasing user needs. This leads to fragmentation and makes solutions increasingly expensive, forcing users to choose between adaptability and their individual needs.

Dot Go is designed to work as an accessibility tool kit. It empowers users to create custom presets without coding knowledge. It allows them to connect any object – from people and vehicles to signage and equipment – with unique sound, haptic and visual feedback, and trigger automations on external devices, smart home applications and online.

Describe the idea

Traditionally, users would rely on one or more apps for different needs. What other solutions are still restricted to fixed use cases limiting the user and leaving fragmentation open.

Dot Go solves this by leveraging open-source object recognition models, including those of other solutions, together with new LiDAR integration in smartphones. This allows anyone to customize presets for specific objects, locations and needs, creating a growing library of use cases curated by users, institutions, and brands.

Most importantly, it enables the visually impaired community to engage more with the world around them and experience it in their own way. This community-driven approach solves the issue of siloed solutions and their associated costs.

What were the key dates in the development process?

March 2020 – iPad LiDAR integration and development begins

October 2020 – iPhone LiDAR integration and Closed Beta Testing begins

January 2021 – Apple TestFlight Launch

March 2021 – First Beta Feedback

Describe the innovation / technology

Dot Go uses the latest LiDAR integration in iPhone to detect and measure precise distance to any object with an accuracy of 1mm and trigger an action upon detection. Following the simple material conditional logic from programming – “if this then that” – users can create on a basic principle: any object can connect with any action.

Unlike previous solutions that rely on proprietary databases, Dot Go leverages open-source object recognition models, including those of other solutions, making the app as accurate as all existing solutions combined, while enabling customization for specific user needs.

The Dot Go shirt enables hands-free use for enhanced performance: An integrated NFC tag can launch the app automatically, while Pocket Mode allows energy saving and back-tap to switch easily between presets.

A beta version of Dot Go is underway of testing by users worldwide through the Apple TestFlight program.

Describe the expectations / outcome

Dot Go is set to be the first in a new category of accessible, adaptable and hands-free solutions for the blind and visually impaired. For the first time, an assistive application is leveraging the power of open-source object recognition models together with LiDAR integration in iPhone.

This open-source approach allows a growing library of use cases curated by users, public institutions, and brands. Most importantly, it enables the visually impaired community to engage more with the world around them and experience it in their own way.

In March 2020, prototype development of Dot Go was started. Since January 2021, a beta version of Dot Go is underway of testing by users worldwide through the Apple TestFlight program.

More Entries from Early Stage Technology in Innovation

24 items

Grand Prix Cannes Lions
DEGREE INCLUSIVE

Product Innovation

DEGREE INCLUSIVE

UNILEVER, WUNDERMAN THOMPSON ARGENTINA

(opens in a new tab)

More Entries from SERVICEPLAN GERMANY

24 items

Gold Cannes Lions
PARKDISTANCECONTROL

Automotive

PARKDISTANCECONTROL

MINI, SERVICEPLAN GERMANY

(opens in a new tab)