User interface for realtime language translation

Patent metadata

Field	Value
Publication number	US-10496759-B2
Application number	US-201815973722-A
Country	US
Kind code	B2
Filing date	May 8, 2018
Priority date	Nov 8, 2013
Publication date	Dec 3, 2019
Grant date	Dec 3, 2019

Abstract

A language translation application on a user device includes a user interface that provides relevant textual and graphical feedback mechanisms associated with various states of voice input and translated speech.

First claim

What is claimed is:

1. A computer-implemented method comprising:
providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes:
a first language region for displaying transcriptions of the spoken inputs in the first language,
a second language region for displaying translations of the spoken inputs in the second language, and
an indicator region for displaying an icon that, when an outline of the icon is animated, indicates that a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language, wherein the icon comprises a speaker icon whose outline is animated in a spinning pattern only for a period of time before the speaker is prepared to output synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and
in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to animate the outline of the icon to indicate that the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

2. The method of claim 1 , comprising:
receiving a particular spoken input; and
updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

3. The method of claim 1 , wherein the mobile device includes an automated speech recognizer, and an automated language translator.

4. The method of claim 3 , wherein the automated speech recognizer comprises a continuous speech recognizer.

5. The method of claim 1 , wherein the icon is user-selectable to manually activate or deactivate the speaker or a microphone.

6. The method of claim 1 , wherein the first language region is further for displaying translations of spoken inputs in the second language, in the first language.

7. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes:
a first language region for displaying transcriptions of the spoken inputs in the first language,
a second language region for displaying translations of the spoken inputs in the second language, and
an indicator region for displaying an icon that, when an outline of the icon is animated, indicates that a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language, wherein the icon comprises a speaker icon whose outline is animated in a spinning pattern only for a period of time before the speaker is prepared to output synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and
in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to animate the outline of the icon to indicate that the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

8. The system of claim 7 , wherein the operations comprise:
receiving a particular spoken input; and
updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

9. The system of claim 7 , wherein the mobile device includes an automated speech recognizer, and an automated language translator.

10. The system of claim 9 , wherein the automated speech recognizer comprises a continuous speech recognizer.

11. The system of claim 7 , wherein the icon is user-selectable to manually activate or deactivate the speaker or a microphone.

12. The system of claim 7 , wherein the first language region is further for displaying translations of spoken inputs in the second language, in the first language.

13. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes:
a first language region for displaying transcriptions of the spoken inputs in the first language,
a second language region for displaying translations of the spoken inputs in the second language, and
an indicator region for displaying an icon that, when an outline of the icon is animated, indicates that a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language, wherein the icon comprises a speaker icon whose outline is animated in a spinning pattern only for a period of time before the speaker is prepared to output synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and
in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to animate the outline of the icon to indicate that the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

14. The device of claim 13 , wherein the operations comprise:
receiving a particular spoken input; and
updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

15. The device of claim 13 , wherein the mobile device includes an automated speech recognizer, and an automated language translator.

16. The device of claim 15 , wherein the automated speech recognizer comprises a continuous speech recognizer.

17. The device of claim 13 , wherein the icon is user-selectable to manually activate or deactivate the speaker or a microphone.

Assignees

• Google Llc

Inventors

• Cuthbert Alexander Jay
• Goyal Sunny
• Gaba Matthew Morton
• Estelle Joshua J
• Seno Masakazu

Classifications

• G10L25/48 Primary

  specially adapted for particular use · CPC title

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G06F17/289 Primary

  Physics · mapped topic