Robotic physical movement adjustment based on user emotional state

Patent metadata

Field	Value
Publication number	US-11110608-B2
Application number	US-201715858400-A
Country	US
Kind code	B2
Filing date	Dec 29, 2017
Priority date	Dec 29, 2017
Publication date	Sep 7, 2021
Grant date	Sep 7, 2021

Abstract

A method controls a robot. One or more processors receive sensor readings from one or more sensors that are monitoring a human in real time, where the human is currently observing a robotic action by a robot, and where the robotic action is a physical movement performed by the robot. The processor(s) determine a cognitive state of the human while the human is observing the robotic action by the robot, and then adjust the robotic action being performed by the robot based on the cognitive state of the human.

First claim

What is claimed is:

1. A processor-implemented method comprising:
receiving, by one or more processors, sensor readings from one or more sensors, wherein the one or more sensors are monitoring a human in real time, wherein the human is currently observing a robotic action by a robot, wherein the robotic action is a physical movement performed by the robot;
determining, by one or more processors and based on the sensor readings, an emotional state of the human caused by the human observing the robotic action by the robot, wherein the emotional state of the human is fear; and
in response to determining, by one or more processors and based on the sensor readings, that the emotional state of the human caused by the human observing the robotic action by the robot is fear, reducing, by one or more processors, a velocity of movements made by the robot while performing a physical task.

2. The processor-implemented method of claim 1 , wherein the robotic action is a physical locomotion of the robot, and wherein the processor-implemented method further comprises:
adjusting, by one or more processors, the physical locomotion of the robot to increase a physical distance between the robot and the human based on the emotional state of the human.

3. The processor-implemented method of claim 1 , wherein the robotic action further comprises a generation of a visual image on a display on the robot, wherein the human is viewing the visual display on the robot, and wherein the processor-implemented method further comprises:
adjusting, by one or more processors, the visual image on the display on the robot based on the emotional state of the human that is viewing the visual display on the robot.

4. The processor-implemented method of claim 1 , wherein the robotic action further comprises a generation of a sound from a speaker on the robot, and wherein the processor-implemented method further comprises:
adjusting, by one or more processors, a volume level of the sound from the speaker based on the emotional state of the human.

5. The processor-implemented method of claim 1 , further comprising:
detecting, by a chemical detector on the robot, a characteristic scent of the human; and
identifying, by one or more processors, a presence of the human within a predefined distance of the robot based on the characteristic scent of the human that is detected by the chemical detector on the robot.

6. The processor-implemented method of claim 1 , further comprising:
determining, by one or more processors, the emotional state of the human based on an analysis of a speech pattern of the human, wherein the analysis of the speech pattern is performed using advanced natural language analytics, and wherein the speech pattern is indicative of the emotional state of the human.

7. The processor-implemented method of claim 1 , further comprising:
determining, by one or more processors, the emotional state of the human based on biometric sensor readings from a set of biometric sensors directed towards the human.

8. The processor-implemented method of claim 1 , further comprising:
determining, by one or more processors, the emotional state of the human based on an analysis of thermal imaging of a face of the human, wherein the thermal imaging detects a blood flow level in the human that is indicative of the emotional state of the human.

9. The processor-implemented method of claim 1 , further comprising:
modifying, by one or more processors, the robotic action based on heuristic machine learning by the robot, wherein the heuristic machine learning causes the robot to learn about an environment of the robot in order to adjust the robotic action.

10. The processor-implemented method of claim 1 , further comprising:
identifying, by one or more processors, objects within an area;
identifying, by one or more processors, a type of area of the area based on the objects identified within the area;
determining, by one or more processors, that the robot is prohibited from operating in the identified type of area; and
in response to determining that the robot is prohibited from operating in the identified type of area, blocking, by one or more processors, the robot from being in the area.

11. The processor-implemented method of claim 1 , further comprising:
receiving, by one or more processors, location sensor readings from one or more location sensors, wherein the location sensor readings describe a type of physical location in which the human is located;
detecting, by one or more processors, that the robot is within the type of physical location in which the human is located;
determining, by one or more processors, that a presence of the robot within the type of physical location in which the human is located is affecting the emotional state of the human being; and
in response to determining that the robot is within the type of physical location in which the human is located and in response to determining that the presence of the robot within the type of physical location in which the human is located is affecting the emotional state of the human being, directing, by one or more processors, the robot to leave the type of physical location in which the human is located.

12. The processor-implemented method of claim 1 , further comprising:
detecting, by one or more processors and based on positioning sensor readings from one or more positioning sensors, that the robot is within a predefined distance of an object that has been predetermined to be delicate; and
directing, by one or more processors, the robot to move away from the object by more than the predefined distance.

13. The processor-implemented method of claim 1 , wherein the robot includes a mechanical manipulator arm, wherein the mechanical manipulator arm is engineered to lift an object that is away from the robot, and wherein the processor-implemented method further comprises:
detecting, by one or more processors and based on positioning sensor readings from one or more positioning sensors, that the robot is within a predefined distance of an object that has been predetermined to be delicate; and
preventing, by one or more processors, the robot from moving the mechanical manipulator arm while the robot is within the predefined distance from the object.

14. The processor-implemented method of claim 1 , wherein the human is observing, via a remote video feed, the robotic action of the robot at a location that is remote from a location of the robot.

15. The processor-implemented method of claim 1 , wherein the physical movement performed by the robot is an initial movement, wherein the initial movement is a two step movement along two different directions that causes the robot to move indirectly towards an object, and wherein the adjusted robotic action is a one step movement along a single direction that causes the robot to move directly towards the object.

16. A robot comprising:
a sensor receiver for receiving sensor readings from a set of one or more sensors that detect an emotional state of a human who is currently observing a robotic action by a robot, wherein the robotic action is a physical movement performed by the robot, wherein the emotional state of the human is fear, and wherein the emotional state of fear in the human is caused by the human observing the robotic action by the robot;
an instruction receiver for receiving a computer-executable program for causing the robot to modify a velocity of the robotic action based on the sensor readings that describe the emotional state of the human; and
a robotic controller processor for:
determining, base

Assignees

• IBM

Inventors

• Aaron Andrew
• Deligianni Hariklia
• Melville David O S
• Podlaseck Mark E
• Seo Hyun Kyu

Classifications

• B25J13/00 Primary

  Controls for manipulators (programme controls B25J9/16) · CPC title

• G10L25/63

  for estimating an emotional state · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• H04R1/028

  associated with devices performing functions other than acoustics, e.g. electric candles · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title