Operation assisting method and operation assisting device

Patent metadata

Field	Value
Publication number	US-9418653-B2
Application number	US-201514715573-A
Country	US
Kind code	B2
Filing date	May 18, 2015
Priority date	May 20, 2014
Publication date	Aug 16, 2016
Grant date	Aug 16, 2016

Abstract

An operation assisting method comprising comparing input spoken voices with a preliminarily stored keyword associated with an operation target and determining whether or not the keyword is spoken, determining whether or not similarity between or among the input spoken voices falls within a predetermined range. In a case where it is determined that the keyword is not spoken, determining whether or not eyes of a user are directed at the operation target, and in a case of the similarity falling within the predetermined range, determining that the keyword is spoken, in a case of being determined that the eyes of the user are directed at the operation target.

First claim

What is claimed is:

1. An operation assisting method comprising:
comparing voice data with preliminarily stored keywords associated with an operation target and determining whether the voice data match with the preliminarily stored keywords or not;
determining whether or not one or more of the preliminarily stored keywords are included in the voice data, and whether or not a similarity between or among the voice data falls within a predetermined range;
detecting whether or not sight lines of a speaker are directed at the operation target only when it is determined that the voice data do not match with the preliminarily stored keywords, and the similarity between or among the voice data falls within the predetermined range; and
determining that the one or more of the preliminarily stored keywords are included in the voice data only when the similarity between or among the voice data are within the predetermined range during a predetermined time period before a timing when the sight lines of the speaker are directed at the operation target.

2. The operation assisting method according to claim 1 , wherein the voice data are determined to match with the preliminarily stored keywords when the voice data includes one or more words that are same as the preliminarily stored keywords.

3. An operation assisting device comprising:
a keyword detector that compares voice data with preliminarily stored keywords associated with an operation target and determines whether the voice data match with the preliminarily stored keywords or not; and
a sight-line detector that detects whether or not sight lines of a speaker are directed at the operation target only when it is determined that the voice data to not include the preliminarily stored keywords, an the similarity between or among the voice data falls within a predetermined range,
wherein the keyword detector determines that the voice data match with the preliminarily stored keywords in a case where the similarity between or among the voice data is within the predetermined range during a predetermined time period before a timing when the sight-line detector detects that the sight lines of the speaker are directed at the operation target.

4. The operation assisting device according to claim 3 , wherein
the keyword detector starts a predetermined operation for the operation target in a case where it is determined that the voice data match with the preliminarily stored keywords.

5. The operation assisting device according to claim 3 , wherein
the keyword detector activates a voice operation function for the operation target in a case where it is determined that the voice data match with the preliminarily stored keywords.

6. The operation assisting device according to claim 3 , wherein
the keyword detector further calculates an evaluation value indicating a likelihood that the voice data match with the preliminarily stored keywords, and determines that the voice data match with the preliminarily stored keywords, in a case where the calculated evaluation value is greater than or equal to a predetermined threshold value.

7. The operation assisting device according to claim 3 , wherein the keyword detector determines that the voice data match with the preliminarily stored keywords when the voice data include one or more words that are same as the preliminarily stored keywords.

8. The operation assisting device according to claim 3 , wherein
the keyword detector includes
a keyword determiner that calculates an evaluation value indicating a likelihood that the voice data match with the preliminarily stored keywords, and determines that the voice data match with the preliminarily stored keywords, in a case where the calculated evaluation value is greater than or equal to a first threshold value;
an information storage that stores therein the voice data corresponding to the evaluation value, in a case where the evaluation value is less than the first threshold value and greater than or equal to a second threshold value; and
an utterance determiner that calculates the similarity between or among the voice data in a case where a predetermined number of the voice data is, input during a predetermined time period before a timing when detecting that the sight lines of the speaker are directed at the operation target, are stored in the information storage, the utterance determiner determining that the voice data match with the preliminarily stored keywords, under a condition that the calculated similarity falls within the predetermined range.

9. The operation assisting device according to claim 8 , wherein
the utterance determiner inquires of the speaker about whether or not the voice data match with the preliminarily stored keywords, and determines that the voice data match with the preliminarily stored keywords, in a case where an asnswer informing that the voice data match with the preliminarily stored keywords is input and the similarity falls within the predetermined range.

10. The operation assisting device according to claim 8 , wherein
the utterance determiner notifies the speaker of the preliminarily stored keywords in a case where the similarity falls within the predetermined range.

11. The operation assisting device according to claim 8 , wherein
the utterance determiner inquires of the speaker about predetermined information, and notifies the speaker of the preliminarily stored keywords in a case where a correct answer is input and the similarity falls within the predetermined range.

12. An operation assisting device comprising:
a transmitter that performs transmission to a keyword detection device that compares voice data with preliminarily stored keywords associated with an operation target and determines whether or not the voice data match with the preliminarily stored keywords;
a client side receiver that receives utterance detection information indicating that the voice data match with the preliminarily stored keywords is transmitted from the keyword detection device; and
a sight-line detector that detects whether or not sight lines of a speaker are directed at the operation target only when it is determined that the voice data do not include the preliminarily stored keywords and the similarity between or among the voice data falls within a predetermined range, wherein
the transmitter
transmits, to the keyword detection device, information indicating whether or not the sight lines of the speaker are directed at the operation target, and
determines that the voice data match with the preliminarily stored keywords, only when the similarity between or among the voice data is within the predetermined range during a predetermined time period before a timing when the sight-line detector detects that the sight lines of the speaker are directed at the operation target.

13. The operation assisting device according to claim 12 , wherein the voice data are determined to match with the preliminarily stored keywords when the voice data includes one or more words that are same as the preliminarily stored keywords.

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Sekiguchi Takeshi
• Kunieda Nobuyuki

Classifications

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L2015/088

  Word spotting · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/08 Primary

  Speech classification or search · CPC title

• G10L15/25

  using position of the lips, movement of the lips or face analysis · CPC title