Interactive robot and human-robot interaction method

Patent metadata

Field	Value
Publication number	US-10482886-B2
Application number	US-201715853918-A
Country	US
Kind code	B2
Filing date	Dec 25, 2017
Priority date	Aug 5, 2017
Publication date	Nov 19, 2019
Grant date	Nov 19, 2019

Abstract

An interactive robot includes an image capturing device, an audio capturing device, an output device, and a processor. The processor is configured to obtain audio information captured by the audio capturing device and image information captured by the image capturing device, recognize a target from the audio information and the image information, confirm basic information and event information of the target and link the basic information with the event information, obtain key information from the event information of the target, implement a neural network analysis algorithm on the key information to confirm an emotion type of the target, search a preset public knowledge database according to the key information to obtain a relevant result, apply a deep learning algorithm on the relevant result and the emotion type of the target to determine a response, and execute the response through the output device.

First claim

What is claimed is:

1. An interactive robot comprising:
an image capturing device;
an audio capturing device;
at least one output device; and
at least one processor configured to:
obtain audio information captured by the audio capturing device and image information captured by the image capturing device;
recognize a target from the audio information and the image information;
confirm basic information and event information of the target and link the basic information with the event information;
obtain key information from the event information of the target;
implement a neural network analysis algorithm on the key information to confirm an emotion type of the target, search a preset public knowledge database according to the key information to obtain a relevant result, apply a deep learning algorithm on the relevant result and the emotion type of the target to determine a response; and
execute the response through the output device;
wherein the response is a series of instructions for controlling the interactive robot to interact with the target.

2. The interactive robot of claim 1 , wherein the processor recognizes a voiceprint from the audio information and recognizes a faceprint from the image information, and recognizes the target from the voiceprint and the faceprint.

3. The interactive robot of claim 1 , wherein the processor controls the audio capturing device to set the audio information of the target as the event information of the target.

4. The interactive robot of claim 3 , wherein the processor recognizes the audio information, converts the audio information into text data, and obtains key information from the text data; the key information from the text data is the key information of the event information.

5. The interactive robot of claim 1 , wherein the processor controls the image capturing device to set the image information of the target as the event information of the target.

6. The interactive robot of claim 5 , wherein the processor obtains facial expression information and limb movement information from the image information, obtains facial expression parameters from the facial expression information, obtains limb movement parameters from the limb movement information, and sets the facial expression parameters and the limb movement parameters as the key information of the event information.

7. The interactive robot of claim 1 , wherein the processor is further configured to set emotions of the interactive robot; the processor implements a neural network analysis algorithm on the key information to confirm an emotion type of the target, searches a preset public knowledge database according to the key information to obtain a relevant result, applies a deep learning algorithm on the relevant result, the emotion type of the target, and the set emotions of the interactive robot to determine the response.

8. The interactive robot of claim 1 , wherein the output device includes an audio output device and an expression output device; the processor controls the audio output device to output audio information as the response and controls the expression output device to output an expression to execute the response.

9. A human-robot interacting method implemented in an interactive robot, the method comprising:
obtaining audio information captured by an audio capturing device and image information captured by an image capturing device;
recognizing a target from the audio information and the image information;
confirming basic information and event information of the target and linking the basic information with the event information;
obtaining key information from the event information of the target;
implementing a neural network analysis algorithm on the key information to confirm an emotion type of the target, searching a preset public knowledge database according to the key information to obtain a relevant result, applying a deep learning algorithm on the relevant result and the emotion type of the target to determine a response; and
executing the response through the output device;
wherein the response is a series of instructions for controlling the interactive robot to interact with the target.

10. The method of claim 9 , wherein the target is recognized from the audio information and the image information by recognizing a voiceprint from the audio information and recognizing a faceprint from the image information.

11. The method of claim 9 , wherein the basic information and the event information are confirmed and linked together by setting the audio information captured by the audio capturing device as the event information.

12. The method of claim 11 , wherein the audio information is recognized and converted into text data, key information is obtained from the text data, and the key information from the text data is set as the key information of the event information.

13. The method of claim 9 , wherein the basic information and the event information are confirmed and linked together by setting the image information captured by the image capturing device as the event information.

14. The method of claim 13 , wherein facial expression information and limb movement information are obtained from the image information, facial expression parameters are obtained from the facial expression information, limb movement parameters are obtained from the limb movement information, and the facial expression parameters and the limb movement parameters are set as the key information of the event information.

15. The method of claim 9 , wherein an audio output device is controlled to output audio information as the response, and an expression output device is controlled to output an expression to execute the response.

16. A non-transitory storage medium having stored thereon instructions that, when executed by at least one processor of an interactive robot, causes the least one processor to execute instructions of a method for human-robot interaction, the method comprising:
obtaining audio information captured by an audio capturing device and image information captured by an image capturing device;
recognizing a target from the audio information and the image information;
confirming basic information and event information of the target and linking the basic information with the event information;
obtaining key information from the event information of the target;
implementing a neural network analysis algorithm on the key information to confirm an emotion type of the target, searching a preset public knowledge database according to the key information to obtain a relevant result, applying a deep learning algorithm on the relevant result and the emotion type of the target to determine a response; and
executing the response through the output device;
wherein the response is a series of instructions for controlling the interactive robot to interact with the target.

Assignees

• Fu Tai Hua Ind Shenzhen Co Ltd
• Hon Hai Prec Ind Co Ltd

Inventors

• Zhang xue-qin

Classifications

• G06F40/30

  Semantic analysis · CPC title

• G10L17/00

  Speaker identification or verification techniques · CPC title

• G10L15/1822

  Parsing for meaning understanding · CPC title

• G10L25/63

  for estimating an emotional state · CPC title

• B25J9/161

  Hardware, e.g. neural networks, fuzzy logic, interfaces, processor · CPC title