This project was a collaboration between the Google Assistant and Google Translate teams. One challenge faced was converting the visual only experience of Google Translate's conversation feature into a voice experience. Additionally, we had to ensure that this feature would work well across many Assistant surfaces, all while managing different interaction types. An important and measured consideration was to create a consistent, predictable experience, regardless of surface. For our launch, we focused on voice-only interactions, such as on smart speakers, and voice-forward interactions, as with smart displays.

The fascination with breaking language barriers is as old as history itself, and devices that magically translate spoken words have been the stuff of dreams, legends, and movies. Google Assistant’s Interpreter Mode brings humanity one step closer to achieving that dream.

Using voice technology to make communication more natural, we wanted to create an experience that not only saved time, but is as simple as talking to a human translator. To achieve that vision, our team at Google created Interpreter Mode, which translates spoken conversations in real-time. If the device has a screen, those translations are shown in order to aid the conversation.

Moving forward, we want this translation technology to be accessible to anyone, anywhere.

Moving forward, we envision a translation technology that’s accessible to anyone, anywhere.

This technology has the potential to serve as a valuable translation companion in locations with high tourist traffic, like airports and hotels, as well as places where translation services can play a critical role, such as in hospitals, or for first responders.

Translation itself has the ability to enable the exchange of knowledge and information, regardless of where you come from, or what language you speak. It is our belief that this technology, when democratized, has the capacity to do tremendous social good.

Interpreter Mode launched in February of 2019. It’s available on most Assistant-enabled smart speakers and smart displays, and can translate across 29 languages. Users trigger the feature by asking the Assistant, “Hey Google, be my interpreter”. Once in Interpreter Mode, the Assistant will automatically identify the language being spoken, and translate accordingly.

Interpreter Mode is unique among voice assistants because of its ability to moderate a conversation. If the Assistant is unsure of what a user said, it will ask them to repeat themselves, just like a human would. Similarly, if the Assistant is unsure which language was just spoken, it will ask the user to clarify.

We designed the voice interaction to be as contextual and natural as possible. We accomplished this by investing in a couple key interaction models: surface-based and usage-based. The former allows us to specify the Assistant’s response depending on which surface the user is using. For example, on a smart speaker, the Assistant’s responses in Interpreter Mode are more verbose than they are on Smart displays, as speakers lack the visual feedback given by the display. The usage-based interaction model allows the Assistant to have a variety of responses based on how much a feature has been used. For example, an inexperienced user will receive a more detailed response compared to an experienced user. As the usage of the feature progresses, so do the responses the user hears.

As part of our launch strategy, Interpreter Mode ran as a pilot during CES 2019 in locations with a high potential for the need for translation. It operated for a few weeks at the concierge and front desks of Caesars Palace (Las Vegas), the Hyatt Regency (San Francisco) and Dream Downtown (New York City).