Enhanced voice conferencing with history, language translation and identification

Patent metadata

Field	Value
Publication number	US-9245254-B2
Application number	US-201213397289-A
Country	US
Kind code	B2
Filing date	Feb 15, 2012
Priority date	Dec 1, 2011
Publication date	Jan 26, 2016
Grant date	Jan 26, 2016

Abstract

Techniques for ability enhancement are described. Some embodiments provide an ability enhancement facilitator system (“AEFS”) configured to enhance voice conferencing among multiple speakers. Some embodiments of the AEFS enhance voice conferencing by recording, translating and presenting voice conference history information based on speaker-related information, wherein the translation is based on language identification using multiple speech recognizers and GPS information. The AEFS receives data that represents utterances of multiple speakers who are engaging in a voice conference with one another. The AEFS then determines speaker-related information, such as by identifying a current speaker, locating an information item (e.g., an email message, document) associated with the speaker, or the like. The AEFS records conference history information (e.g., a transcript) based on the determined speaker-related information. The AEFS then informs a user of the conference history information, such as by presenting a transcript of the voice conference and/or related information items on a display of a conferencing device associated with the user.

First claim

The invention claimed is:

1. A method for ability enhancement, the method comprising: by a computer system,
receiving data representing speech signals from a voice conference amongst multiple speakers, wherein the multiple speakers are remotely located from one another, wherein each of the multiple speakers uses a separate conferencing device to participate in the voice conference;
determining speaker-related information associated with the multiple speakers, based on the data representing speech signals from the voice conference;
recording conference history information based on the speaker-related information, by recording indications of topics discussed during the voice conference by:
performing speech recognition to convert the data representing speech signals into text;
analyzing the text to identify frequently used terms or phrases; and
determining the topics discussed during the voice conference based on the frequently used terms or phrases;
audibly notifying a user to view the conference history information on a display device, wherein the user is notified in a manner that is not audible to at least some of the multiple speakers; and
presenting, on the display device, at least some of the conference history information to the user;
translating an utterance of one of the multiple speakers in a first language into a message in a second language, based on the speaker-related information,
wherein the speaker related information is determined by automatically determining the second and the first language comprising steps of:
concurrently or simultaneously applying multiple speech recognizers and using GPS information indicating the speakers' locations; and
recording the message in the second language as part of the conference history information.

2. The method of claim 1 , wherein the recording conference history information based on the speaker-related information includes: recording a transcription of utterances made by speakers during the voice conference.

3. The method of claim 2 , wherein the recording a transcription includes:
performing speech recognition to convert data representing a speech signal from one of the multiple speakers into text; and
storing the text in association with an indicator of the one speaker.

4. The method of claim 1 , further comprising: performing voice identification based on the data representing the speech signals from the voice conference.

5. The method of claim 4 , wherein the performing voice identification includes: in a conference call system, matching a portion of the data representing the speech signals with an identity of one of the multiple speakers, based on a communication channel that is associated with the one speaker and over which the portion of the data is transmitted.

6. The method of claim 4 , the semantic content including a name, event, or entity mentioned by a speaker.

7. The method of claim 6 , further comprising:
processing voice messages from the multiple persons to generate voice print data for each of the multiple persons, wherein each of the voice messages is a telephone voice mail message stored by a voice mail service in association with a sender telephone number; and
performing reverse directory lookups using the sender telephone numbers to determine names of speakers associated with the voice messages.

8. The method of claim 1 , wherein the recording conference history information based on the speaker-related information includes: recording indications of information items related to subject matter of the voice conference.

9. The method of claim 8 , wherein the recording indications of information items related to subject matter of the voice conference includes:
performing speech recognition to convert the data representing speech signals into text; and
analyzing the text to identify information items mentioned by the speakers.

10. The method of claim 1 , wherein the presenting at least some of the conference history information includes: presenting the conference history information to a participant in the voice conference, the participant having rejoined the voice conference after having not participated in the voice conference for a period of time.

11. The method of claim 10 , wherein the participant rejoins the voice conference after at least one of: pausing the voice conference, muting the voice conference, holding the voice conference, voluntarily leaving the voice conference, and/or involuntarily leaving the voice conference.

12. The method of claim 1 , wherein the recording conference history information based on the speaker-related information includes: recording the data representing speech signals from the voice conference.

13. The method of claim 1 , wherein the recording conference history information based on the speaker-related information includes: as each of the multiple speakers takes a turn speaking during the voice conference, recording speaker-related information associated with the speaker.

14. The method of claim 1 , wherein the recording conference history information based on the speaker-related information includes: recording conference history information based on the speaker-related information during a telephone conference call amongst the multiple speakers.

15. The method of claim 1 , wherein the presenting at least some of the conference history information includes: presenting the conference history information to a new participant in the voice conference, the new participant having joined the voice conference while the voice conference was already in progress.

16. The method of claim 1 , wherein the presenting at least some of the conference history information includes: presenting the conference history information to a user after conclusion of the voice conference.

17. The method of claim 1 , wherein the presenting at least some of the conference history information includes: providing a user interface configured to access the conference history information by scrolling through a temporal record of the voice conference.

18. The method of claim 1 , wherein the presenting at least some of the conference history information includes: presenting a transcription of utterances made by speakers during the voice conference.

19. The method of claim 1 , wherein the presenting at least some of the conference history information includes: presenting indications of topics discussed during the voice conference.

20. The method of claim 1 , wherein the presenting at least some of the conference history information includes: presenting indications of information items related to subject matter of the voice conference.

21. The method of claim 1 , wherein the presenting at least some of the conference history information includes: presenting, while a current speaker is speaking, conference history information on a display device of the user, the displayed conference history information providing information related to previous statements made by the current speaker.

22. The method of claim 1 , further comprising:
retrieving information items that reference the text data; and
informing the user of the retrieved information items.

23. The method of claim 1 , wherein the performing speech recognition includes:
performing speech recognition based at least in part on a language model associated with the one speaker, wherein the language model represents word transition likelihoods; and
generating the language model based on information items generated by

Assignees

• Lord Richard T
• Lord Robert W
• Myhrvold Nathan P
• Tegreene Clarence T
• Hyde Roderick A
• Wood Jr Lowell L
• Ishikawa Muriel Y
• Wood Victoria Y H
• Whitmer Charles
• Bahl Paramvir
• Burger Douglas C
• Chandra Ranveer
• Gates Iii William H
• Holman Paul
• Kare Jordin T
• Mundie Craig J
• Paek Tim
• Tan Desney S
• Zhong Lin
• Dyor Matthew G
• Elwha Llc

Inventors

• Lord Richard T
• Lord Robert W
• Myhrvold Nathan P
• Tegreene Clarence T
• Hyde Roderick A
• Wood Jr Lowell L
• Ishikawa Muriel Y
• Wood Victoria Y H
• Whitmer Charles
• Bahl Paramvir
• Burger Douglas C
• Chandra Ranveer
• Gates Iii William H
• Holman Paul
• Kare Jordin T
• Mundie Craig J
• Paek Tim
• Tan Desney S
• Zhong Lin
• Dyor Matthew G

Classifications

• G06F40/279

  Recognition of textual entities · CPC title

• G06F40/58

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06F3/165

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title

• G06Q10/10 Primary

  Office automation; Time management · CPC title

• H04L67/306

  User profiles · CPC title