Motion responsive user interface for realtime language translation

Patent metadata

Field	Value
Publication number	US-9355094-B2
Application number	US-201314077411-A
Country	US
Kind code	B2
Filing date	Nov 12, 2013
Priority date	Aug 14, 2013
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

A data processing apparatus receives data indicating a movement of a client device by a first user. The apparatus determines that the movement of the client device is a delimiter motion for switching between a first mode, in which the client device is configured to (i) provide a first interface for a first user speaking in a first language and (ii) perform speech recognition of the first language, and a second mode, in which the client device is configured to (i) provide a second interface for a second user speaking in a second language and (ii) perform speech recognition of the second language, the second interface being different from the first interface. Based on determining that the movement is a delimiter motion, the apparatus switches between the first mode and the second mode without the second user physically interacting with the client device.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving data indicating a movement of a client device by a first user;
determining that the movement of the client device is classified as a delimiter motion for; and
based on determining that the movement is classified as a delimiter motion, switching between a first mode, in which the client device (i) configures a language setting of an automated speech recognition engine to transcribe the speech of the first user into text that is in the first language, and (ii) provides a first interface that includes the transcribed text of the first user that is in the first language, and a translation of the transcribed text of the first user that is in a second language, and a second mode, in which the client device (i) configures the language setting of the automated speech recognition engine to transcribe the speech of a second user into text that is in the second language, and (ii) provides a second interface that includes the transcribed text of the second user that is in the second language, and a translation of the transcribed text of the second user that is in the first language, wherein the second interface is different from the first interface, and wherein switching between the first mode and the second mode occurs without the second user physically interacting with the client device.

2. The method of claim 1 , wherein the movement of the client device by the first user comprises rotation of the client device by the first user.

3. The method of claim 1 , further comprising, while the client device is in the second mode,
receiving an utterance of the second user in the second language; and
outputting an audio signal encoding the translation of the transcribed text of the second user that is in the first language.

4. The method of claim 1 , wherein the client device configures the language setting of the automated speech recognition engine to transcribe the speech of the first user into text that is in the first language by providing an identifier indicating to the automated speech recognition engine that the speech is in the first language.

5. The method of claim 1 , wherein the automated speech recognition engine executes on a server that is remote from the client device.

6. The method of claim 1 , further comprising, while the client device is in the first mode,
receiving an utterance of the first user in the first language; and
outputting an audio signal encoding the translation of the transcribed text of the first user that is in the second language.

7. The method of claim 6 , further comprising, while the client device is in the first mode,
translating, at the client device, the utterance of the first user into the second language.

8. A client device comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving data indicating a movement of a client device by a first user;
determining that the movement of the client device is classified as a delimiter motion for; and
based on determining that the movement is classified as a delimiter motion, switching between a first mode, in which the client device (i) configures a language setting of an automated speech recognition engine to transcribe the speech of the first user into text that is in the first language, and (ii) provides a first interface that includes the transcribed text of the first user that is in the first language, and a translation of the transcribed text of the first user that is in a second language, and a second mode, in which the client device (i) configures the language setting of the automated speech recognition engine to transcribe the speech of a second user into text that is in the second language, and (ii) provides a second interface that includes the transcribed text of the second user that is in the second language, and a translation of the transcribed text of the second user that is in the first language, wherein the second interface is different from the first interface, and wherein switching between the first mode and the second mode occurs without the second user physically interacting with the client device.

9. The client device of claim 8 , wherein the movement of the client device by the first user comprises rotation of the client device by the first user.

10. The client device of claim 8 , wherein the operations further comprise, while the client device is in the second mode,
receiving an utterance of the second user in the second language; and
outputting an audio signal encoding the translation of the transcribed text of the second user that is in the first language.

11. The client device of claim 8 , wherein the operations further comprise, while the client device is in the first mode,
receiving an utterance of the first user in the first language; and
outputting an audio signal encoding the translation of the transcribed text of the first user that is in the second language.

12. The client device of claim 11 , wherein the operations further comprise, while the client device is in the first mode,
translating, at the client device, the utterance of the first user into the second language.

13. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
receiving data indicating a movement of a client device by a first user;
determining that the movement of the client device is classified as a delimiter motion for; and
based on determining that the movement is classified as a delimiter motion, switching between a first mode, in which the client device (i) configures a language setting of an automated speech recognition engine to transcribe the speech of the first user into text that is in the first language, and (ii) provides a first interface that includes the transcribed text of the first user that is in the first language, and a translation of the transcribed text of the first user that is in a second language, and a second mode, in which the client device (i) configures the language setting of the automated speech recognition engine to transcribe the speech of a second user into text that is in the second language, and (ii) provides a second interface that includes the transcribed text of the second user that is in the second language, and a translation of the transcribed text of the second user that is in the first language, wherein the second interface is different from the first interface, and wherein switching between the first mode and the second mode occurs without the second user physically interacting with the client device.

14. The computer-readable medium of claim 13 , wherein the movement of the client device by the first user comprises rotation of the client device by the first user.

15. The computer-readable medium of claim 13 , wherein the operations further comprise, while the client device is in the first mode,
receiving an utterance of the first user in the first language; and
outputting an audio signal encoding the translation of the transcribed text of the first user that is in the second language.

16. The computer-readable medium of claim 15 , wherein the operations further comprise, while the client device is in the first mode,
translating, at the client device, the utterance of the first user into the second language.

Assignees

• Google Inc

Inventors

• Cuthbert Alexander J
• Estelle Joshua J
• Hughes Macduff Richard
• Goyal Sunny
• Jiang Minqi Sebastian

Classifications

• G10L15/005

  Language recognition · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F40/263

  Language identification · CPC title

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G10L13/086

  Detection of language · CPC title