Speech recognition system adaptation based on non-acoustic attributes

Patent metadata

Field	Value
Publication number	US-2016140964-A1
Application number	US-201514790142-A
Country	US
Kind code	A1
Filing date	Jul 2, 2015
Priority date	Nov 13, 2014
Publication date	May 19, 2016
Grant date	—

Abstract

A method includes the following steps. A vicinity from which speech input to a speech recognition system originates is determined. Non-acoustic data from the vicinity of the speech is obtained using one or more non-acoustic sensors. A subject speaker is identified as the source of the speech input from the obtained non-acoustic data. One or more non-acoustic attributes of the subject speaker is analyzed. A speech recognition system is adjusted based on the one or more analyzed non-acoustic attributes.

First claim

What is claimed is:

1 . A method, comprising the steps of:
determining a vicinity from which speech input to a speech recognition system originates; obtaining non-acoustic data from the vicinity of the speech using one or more non-acoustic sensors; identifying a subject speaker as the source of the speech input from the obtained non-acoustic data; analyzing one or more non-acoustic attributes of the subject speaker; and adjusting a speech recognition system based on the one or more analyzed non-acoustic attributes; wherein the steps are performed by at least one processor device coupled to a memory.

2 . The method of claim 1 , wherein determining the vicinity comprises locating a sound direction of the speech input.

3 . The method of claim 1 , wherein determining the vicinity comprises locating one or more head regions using at least one of the one or more non-acoustic sensors.

4 . The method of claim 1 , wherein obtaining non-acoustic data comprises capturing visual data of the vicinity of the speech input.

5 . The method of claim 4 , wherein identifying the subject speaker comprises segmenting one or more faces from the captured visual data.

6 . The method of claim 5 , wherein identifying the subject speaker further comprises:
detecting mouth motion on the one or more faces; and selecting a face corresponding to the subject speaker based on the detected mouth motion.

7 . The method of claim 5 , wherein analyzing the one or more non-acoustic attributes comprises extracting one or more facial features of the subject speaker.

8 . The method of claim 4 , wherein analyzing the one or more non-acoustic attributes comprises extracting at least one of a hair color and an age.

9 . The method of claim 5 , wherein analyzing the one or more non-acoustic attributes comprises mapping the subject speaker to a cluster.

10 . The method of claim 9 , wherein the cluster comprises at least one of a gender cluster and an ethnicity cluster.

11 . The method of claim 1 , wherein adjusting the speech recognition system comprises selecting an acoustic model for use by the speech recognition system.

12 . The method of claim 1 , wherein adjusting the speech recognition system comprises selecting a language model for use by the speech recognition system.

Assignees

• IBM

Inventors

• Connell Ii Jonathan H
• Marcheret Etienne

Classifications

• G10L15/07 Primary

  to the speaker · CPC title

• G10L15/24

  Speech recognition using non-acoustical features · CPC title

• G10L15/25 Primary

  using position of the lips, movement of the lips or face analysis · CPC title

• G10L2015/227

  of the speaker; Human-factor methodology · CPC title

• G06N3/008

  based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour · CPC title