Robot capable of incorporating natural dialogues with a user into the behaviour of same, and methods of programming and using said robot

Patent metadata

Field	Value
Publication number	US-10052769-B2
Application number	US-201314390746-A
Country	US
Kind code	B2
Filing date	Apr 3, 2013
Priority date	Apr 4, 2012
Publication date	Aug 21, 2018
Grant date	Aug 21, 2018

Abstract

A humanoid robot is provided, the robot being capable of holding a dialog with at least one user, the dialog using two modes of voice recognition, one open and the other closed, the closed mode being defined by a concept characterizing a dialog sequence. The dialog may also be influenced by events that are neither speech nor a text. The robot is capable of executing behaviors and generating expressions and emotions. It has the advantage of considerably reducing programming time and latency of execution of dialog sequences, providing a fluency and naturalness close to human dialogs.

First claim

The invention claimed is:

1. A humanoid robot, comprising:
i) at least one sensor selected from a group comprising audio sensors and visual sensors of events generated by at least one user of said robot,
ii) at least one event recognition module at the output of said at least one sensor and,
iii) at least one module for generating events towards said at least one user, a module for dialog with said at least one user, said dialog module receiving as input the outputs of said at least one recognition module and producing outputs to said event generation module selected from a group including speech, movements, expressions and emotions, wherein said robot further includes an artificial intelligence engine configured for controlling the outputs of the event generation module according to a context of dialog and variables defining emotions of the robot as a series of expressions between a neutral state and a terminal state, an evolution between said neutral state and said terminal state being triggered by a pulse generated as a function of time and of one or more of the outputs of the at least a sensor, the event recognition module, the dialog module and the event generation module.

2. The humanoid robot of claim 1 ,
wherein said at least one event recognition module receives inputs originating from at least two sensors belonging to at least two different types, and said at least one event generation module at the output of said dialog module is capable of producing as outputs events taking into account said inputs originating from said at least two sensors.

3. The humanoid robot of claim 1 ,
wherein said at least one recognition module is capable of structuring the inputs into concepts according to a dynamic hierarchical tree.

4. The humanoid robot of claim 1 ,
wherein an input in said at least one recognition module applies to text or voice inputs and activates a grammar in said dialog module.

5. The humanoid robot of claim 4 ,
wherein an input in said at least one recognition module activates/deactivates recognition of said input.

6. The humanoid robot of claim 4 ,
wherein said at least one recognition module includes a first and a second submodules, the first submodule operating on a closed list of words linked to at least one concept and the second submodule operating on an open list of words.

7. The humanoid robot of claim 6 ,
wherein only one output from the first submodule is supplied to the dialog module.

8. The humanoid robot of claim 6 ,
wherein only an output from the second submodule is supplied to the dialog module.

9. The humanoid robot of claim 6 ,
wherein an output from the first submodule and an output from the second submodule are supplied together to the dialog module.

10. The humanoid robot of claim 6 ,
wherein only an output from the first submodule is first supplied to the dialog module, said output from the first submodule being confirmed in the dialog module by an output from the second submodule.

11. The humanoid robot of claim 6 ,
wherein none of the outputs from the first and second sub-modules generates output from the dialog module and said robot provides at least one output to said at least one user.

12. The humanoid robot of claim 1 ,
wherein the dialog module further receives as input dynamic elements originating from an application.

13. The humanoid robot of claim 1 ,
wherein at least one output from the dialog module is supplied to a module capable of performing a function selected from a group of functions including a function for generating at least one expression of said robot, a function for deciding to generate at least one behavior of said robot, and a function for generating at least one emotion of said robot.

14. The humanoid robot of claim 13 ,
wherein said function selected from the group of functions is said function for generating at least one behavior, and
wherein said function for generating at least one behavior takes into account the system constraints of said robot.

15. The humanoid robot of claim 13 ,
wherein said function selected from the group of functions is said function for generating at least one emotion, and
wherein said function for generating at least one emotion is capable of generating a series of predefined expressions between a neutral state and a predefined state in response to input events.

16. The humanoid robot of claim 1 ,
further comprising a visual recognition module,
wherein said visual recognition module is capable of interpreting at least one sign from said at least one user as a beginning or an end of a dialog sequence.

17. The humanoid robot of claim 3 ,
wherein said dialog module includes a submodule for lexical analysis and a submodule for interpreting the outputs of said lexical analysis submodule capable of generating concepts to which the words of the current dialog are linked.

18. The humanoid robot of claim 1 ,
wherein said dialog module is capable of processing questions and commands from said at least one user relating to the state of the physical and/or logical system of same.

19. A method of dialog between a humanoid robot and at least one user, including:
i) at least one step of recognizing inputs originating from at least one sensor selected from a group including audio sensors and visual sensors of events generated by said at least one user,
ii) a step of generating events to said at least one user, and,
iii) a step of dialog with said dialog with said at least one user, said dialog step receiving as input outputs from said at least one recognition step and producing outputs to said step of generating events selected from a group including speech, movements, expressions and emotions, wherein said method further includes a step of controlling the outputs of the event generation module via an artificial intelligence engine according to a context of dialog and variables defining emotions of the robot and as a series of expressions between a neutral state and a terminal state, an evolution between said neutral state and said terminal state being triggered by a pulse generated as a function of time and of one or more of the outputs of the at least a sensor, the event recognition module, the dialog module and the event generation module.

20. The dialog method of claim 19 ,
wherein variables defining the one or more of resources available to the robot or system constraints applied to the robot include battery level and temperature of said robot.

21. The dialog method of claim 19 ,
wherein said robot holds a dialog with at least two users, parameters characterizing said at least two users being stored in a memory of said robot for being used when said robot recognizes one of the at least two users.

22. A computer program embedded in a humanoid robot including program code instructions suitable, when executed on a computer, for managing a dialog between said humanoid robot and at least a user, said computer program including:
i) at least one module for recognizing events at the output of at least one sensor selected from a group including audio sensors and visual sensors of events generated by said at least one user,
ii) at least one module for generating events to said at least one user, and,
iii) a module for dialog with said at least one user, said dialog module receiving as input outputs from said at least one recognition module and producing outputs to said module for generating events selected from a group including speech, movements, expre

Assignees

• Softbank Robotics Europe

Inventors

• Houssin David
• Gate Gwennael

Classifications

• Y10S901/46

  Sensing device · CPC title

• G10L15/19

  Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F16/3329

  Natural language query formulation · CPC title

• G06N3/008

  based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour · CPC title