Multisensory speech detection

Patent metadata

Field	Value
Publication number	US-9570094-B2
Application number	US-201514753904-A
Country	US
Kind code	B2
Filing date	Jun 29, 2015
Priority date	Nov 10, 2008
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

A computer-implemented method of multisensory speech detection is disclosed. The method comprises determining an orientation of a mobile device and determining an operating mode of the mobile device based on the orientation of the mobile device. The method further includes identifying speech detection parameters that specify when speech detection begins or ends based on the determined operating mode and detecting speech from a user of the mobile device based on the speech detection parameters.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
identifying, by a mobile computing device, an identity of a person that is speaking into the mobile computing device;
selecting stored biometric information associated with the identified person;
determining, by the mobile computing device, that the identified person has stopped speaking into the mobile computing device based at least on the selected biometric information associated with the identified person; and
generating a transcription of the person's speech into the mobile device in response to determining that, based at least on the selected biometric information associated with the identified person, the identified person has stopped speaking into the mobile computing device.

2. The method of claim 1 , wherein identifying the identity of the person that is speaking into the mobile computing device comprises comparing auditory information received by a microphone of the mobile computing device to stored auditory profile information for the person that is speaking into the mobile computing device.

3. The method of claim 2 , wherein identifying the identity of the person that is speaking into the mobile computing device further comprises comparing the auditory information received by the microphone of the mobile computing device to stored auditory profile information for one or more persons other than the person speaking that is into the mobile computing device.

4. The method of claim 1 , wherein identifying the identity of the person that is speaking into the mobile computing device comprises using additional biometric information obtained by a camera of the mobile computing device to identify the identity of the person that is speaking into the mobile computing device.

5. The method of claim 1 , wherein determining that the identified person has stopped speaking into the mobile computing device is based further on information that is specific to the identified person.

6. The method of claim 1 , wherein identifying an identity of a person that is speaking into the mobile computing device determining that the person's speech is not background noise.

7. The method of claim 6 , wherein determining that the person's speech is not background noise includes comparing auditory information received by a microphone of the mobile computing device to a speech energy threshold for the person.

8. The method of claim 1 , wherein determining that the identified person has stopped speaking into the mobile computing device is based further on an input provided by a pose identifier, an input provided by a speech detector, and an input provided by a speaker identifier.

9. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
identifying, by a mobile computing device, an identity of a person that is speaking into the mobile computing device;
selecting stored biometric information associated with the identified person;
determining, by the mobile computing device, that the identified person has stopped speaking into the mobile computing device based at least on the selected biometric information associated with the identified person; and
generating a transcription of the person's speech into the mobile device in response to determining that, based at least on the selected biometric information associated with the identified person, the identified person has stopped speaking into the mobile computing device.

10. The computer storage medium of claim 9 , wherein identifying the identity of the person that is speaking into the mobile computing device comprises comparing auditory information received by a microphone of the mobile computing device to stored auditory profile information for the person that is speaking into the mobile computing device.

11. The computer storage medium of claim 10 , wherein identifying the identity of the person that is speaking into the mobile computing device further comprises comparing the auditory information received by the microphone of the mobile computing device to stored auditory profile information for one or more persons other than the person speaking that is into the mobile computing device.

12. The computer storage medium of claim 9 , wherein identifying the identity of the person that is speaking into the mobile computing device comprises using additional biometric information obtained by a camera of the mobile computing device to identify the identity of the person that is speaking into the mobile computing device.

13. The computer storage medium of claim 9 , wherein determining that the identified person has stopped speaking into the mobile computing device is based further on information that is specific to the identified person.

14. The computer storage medium of claim 9 , wherein identifying an identity of a person that is speaking into the mobile computing device determining that the person's speech is not background noise.

15. The computer storage medium of claim 14 , wherein determining that the person's speech is not background noise includes comparing auditory information received by a microphone of the mobile computing device to a speech energy threshold for the person.

16. The computer storage medium of claim 9 , wherein determining that the identified person has stopped speaking into the mobile computing device is based further on an input provided by a pose identifier, an input provided by a speech detector, and an input provided by a speaker identifier.

17. The method of claim 1 , wherein the stored biometric information includes face appearance, ear shape, or hand print.

18. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
identifying, by a mobile computing device, an identity of a person that is speaking into the mobile computing device;
selecting stored biometric information associated with the identified person;
determining, by the mobile computing device, that the identified person has stopped speaking into the mobile computing device based at least on the selected biometric information associated with the identified person; and
generating a transcription of the person's speech into the mobile device in response to determining that, based at least on the selected biometric information associated with the identified person, the identified person has stopped speaking into the mobile computing device.

19. The system of claim 17 , wherein identifying the identity of the person that is speaking into the mobile computing device comprises comparing auditory information received by a microphone of the mobile computing device to stored auditory profile information for the person that is speaking into the mobile computing device.

20. The system of claim 17 , wherein determining that the identified person has stopped speaking into the mobile computing device is based further on an input provided by a pose identifier, an input provided by a speech detector, and an input provided by a speaker identifier.

Assignees

• Google Inc

Inventors

• Burke Dave
• Lebeau Michael J
• Gianno Konrad
• Kristjansson Trausti T
• Jitkoff John Nicholas
• Senior Andrew W

Classifications

• H04R1/08

  Mouthpieces; {Microphones;} Attachments therefor · CPC title

• G10L25/21

  the extracted parameters being power information · CPC title

• G10L15/24

  Speech recognition using non-acoustical features · CPC title

• H04W4/026

  using orientation information, e.g. compass · CPC title

• H04M2250/74

  with voice recognition means · CPC title