Auto-translation for multi user audio and video

Patent metadata

Field	Value
Publication number	US-9720909-B2
Application number	US-201514827826-A
Country	US
Kind code	B2
Filing date	Aug 17, 2015
Priority date	Dec 12, 2011
Publication date	Aug 1, 2017
Grant date	Aug 1, 2017

Abstract

The disclosed subject matter provides a system, computer readable storage medium, and a method providing an audio and textual transcript of a communication. A conferencing services may receive audio or audio visual signals from a plurality of different devices that receive voice communications from participants in a communication, such as a chat or teleconference. The audio signals representing voice (speech) communications input into respective different devices by the participants. A translation services server may receive over a separate communication channel the audio signals for translation into a second language. As managed by the translation services server, the audio signals may be converted into textual data. The textual data may be translated into text of different languages based the language preferences of the end user devices in the teleconference. The translated text may be further translated into audio signals.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, at a first end user device associated with a user, an audio data signal from a conferencing services server via a first communication channel, the audio data signal representing a communication in a first spoken language received at a second end user device that is intended for the user;
determining, at the first end user device, the first spoken language of the communication represented by the audio data signal;
determining, at the first end user device, language preferences of the user of the first end user device;
comparing, at the first end user device, the determined first spoken language with the language preferences of the user of the at the first end user device; and
when the determined first spoken language does not match the language preferences:
establishing, at the first end user device, a second communication channel with a translation services server, wherein the second communication channel with the translation services server is separate from first communication channel with the conferencing services server,
providing, from the first end user device, the audio data signal to the translation services server,
providing, from the first end user device, the language preferences of the user to the translation services server, and
receiving, at the first end user device, a translated audio data signal from the translation services server, the translated audio data signal representing the communication in a second spoken language corresponding to the language preferences and different from the first spoken language.

2. The method of claim 1 , wherein determining the first spoken language of the communication represented by the audio data signal comprises recognizing the first spoken language of the communication.

3. The method of claim 1 , wherein determining the first spoken language of the communication represented by the audio data signal comprises receiving a user preferences signal from the first end user device.

4. The method of claim 3 , wherein the user preferences signal is an identifier embedded in the audio data signal received from the conferencing services server.

5. The method of claim 1 , further comprising, when the determined first spoken language does not match the language preferences, outputting, at the first end user device, an audio output corresponding to the translated audio data signal.

6. The method of claim 1 , further comprising, when the determined first spoken language does match the language preferences, outputting, at the first end user device, an audio output corresponding to the audio data signal.

7. The method of claim 1 , further comprising receiving, at the first end user device, a transcript of the audio data signal or the translated audio data signal.

8. A computer program product having computer program code containing instructions embodied in a non-transitory machine readable storage medium, a processor executes the instructions to provide a method, the method comprising:
receiving, at a first end user device associated with a user, an audio data signal from a conferencing services server via a first communication channel, the audio data signal representing a communication in a first spoken language received at a second end user device that is intended for the user of the first end user device;
determining, at the first end user device, the first spoken language of the communication represented by the audio data signal;
determining, at the first end user device, language preferences of the user of the first end user device;
comparing, at the first end user device, the determined first spoken language with the language preferences of the user of the at the first end user device; and
when the determined first spoken language does not match the language preferences:
establishing, at the first end user device, a second communication channel with a translation services, wherein the second communication channel with the translation services server is separate from first communication channel with the conferencing services server,
providing, from the first end user device, the audio data signal to the translation services server,
providing, from the first end user device, the language preferences of the user to the translation services server, and
receiving, at the first end user device, a translated audio data signal from the translation services server, the translated audio data signal representing the communication in a second spoken language corresponding to the language preferences and different from the first spoken language.

9. The computer program product of claim 8 , wherein determining the first spoken language of the communication represented by the audio data signal comprises recognizing the first spoken language of the communication.

10. The computer program product of claim 8 , wherein determining the first spoken language of the communication represented by the audio data signal comprises receiving a user preferences signal from the first end user device.

11. The computer program product of claim 8 , wherein the method further comprises, when the determined first spoken language does not match the language preferences, outputting an audio output corresponding to the translated audio data signal.

12. The computer program product of claim 8 , wherein the method further comprises, when the determined first spoken language does match the language preferences, outputting an audio output corresponding to the audio data signal.

13. The computer program product of claim 8 , wherein the method further comprises receiving, at the first end user device, a transcript of the audio data signal or the translated audio data signal.

14. A first end user device associated with a user, comprising:
one or more processors; and
a non-transitory machine readable storage medium storing code containing instructions that, when executed by the one or more processors, provide a method comprising:
receiving, at the first end user device, an audio data signal from a conferencing services server via a first communication channel, the audio data signal representing a communication in a first spoken language received at a second end user device that is intended for the user of the first end user device;
determining, at the first end user device, the first spoken language of the communication represented by the audio data signal;
determining, at the first end user device, language preferences of the user of the first end user device;
comparing, at the first end user device, the determined first spoken language with the language preferences of the user of the at the first end user device; and
when the determined first spoken language does not match the language preferences:
establishing, at the first end user device, a second communication channel with a translation services server, wherein the second communication channel with the translation services server is separate from first communication channel with the conferencing services server,
providing, from the first end user device, the audio data signal to the translation services server,
providing, from the first end user device, the language preferences of the user to the translation services server, and
receiving, at the first end user device, a translated audio data signal from the translation services server, the translated audio data signal representing the communication in a second spoken language corresponding to the language preferences and different from the first spoken language.

15. The first end user device of claim 14 ,

Assignees

• Google Inc

Inventors

• Kristjansson Trausti
• Huang John
• Lin Yu-Kuan
• Tyan Hung-Ying
• Uszkoreit Jakob David
• Estelle Joshua James
• Wang Chung-Yi
• Buryak Kirill
• Konishi Yusuke

Classifications

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G10L13/00 Primary

  Speech synthesis; Text to speech systems · CPC title

• H04M2203/2061

  Language aspects · CPC title

• H04N7/152

  Multipoint control units therefor · CPC title

• H04M3/568

  audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants (echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30) · CPC title