It all started with a naive question: we’d been bringing the voice back for thousands of people every year but, what if you don't have a voice to start with?

Voice for All. A system that captures physical gestures through motion sensing technology and translates them into auditory experiences - a voice.

We were building an app that can recognize sign language. After testing and considering multiple possibilities, we went with Kinect.

The first step to make it work was to build a dictionary that contained the sign language movements. All future signs would be referenced against this dictionary to identify the sign being performed.

To do this, the user stands in front of the Kinect device and performs the signs, one by one. The users movements are recorded and saved as the template for each word. Using the KinectSDK, we tracked 6 out of a possible 20 joints to capture the movements involved in the sign: Head, Center Hip, Left Hand, Left Elbow, Righ Hand, Righ Elbow. All the recorded joint information is saved as JSON data formats.

As people are all different (height, body shape, etc) their movements in performing a sign would all have slight variations. To eliminate these variations, the application implements many algorithms to normalize the position and size of the user. To normalize the position of the user, we needed to normalize all the joints with respect to the Hip joint. For the size of the user, the distances between the joints are normalized according to the distance between the Head and Hip.

The ‘Dynamic Time Warping’ Algorithm is then used to compare the two series of data. All signs that the user performs are converted to vector M and compared with signs in the dictionary, thus determining what sign is being performed.

As a leading brand with a clear promise, Hexos has probably given their voice back to every Indonesian that had one. It was time to go farther, even way beyond our product. We were able to affect people’s lives in a more meaningful way, while creating something out of the ordinary still strongly connected with our overall brand message.

'Voice for All' was introduced in April at Intercontinental Hotel, Jakarta. Thanks to institutions like Sehjira Foundation that collaborated during the development and tests, we will be able to help thousands of people.

As you can see, the beta release launch is pretty recent, but people have started ‘talking’ online already. And with the finished video as a main tool, our pr campaign will do this even bigger.

We are currently working to build a more robust and complete sign language 'dictionary'. With the help from countless people and organizations, this only takes us closer to a public release version available in the near future.

Thanks to 'Voice for All', Hexos is going a step farther, always working under its very same essence: giving a voice to those who needed one.