System and method for controlling a robot

Patent metadata

Field	Value
Publication number	US-12459115-B2
Application number	US-202318178882-A
Country	US
Kind code	B2
Filing date	Mar 6, 2023
Priority date	Mar 6, 2023
Publication date	Nov 4, 2025
Grant date	Nov 4, 2025

Abstract

A controller for controlling a robot is provided. The controller comprises a hierarchical multimodal reinforcement learning (RL) neural network including a first level controller and three second level controllers. The second level controllers comprise a first sub level controller to receive input data based on predefined questions, a second sub level controller to receive the input data by generating a validation question based on state of the RL neural network and a third sub level controller to determine the input data based on state of the RL neural network. The controller is configured to select one of the second level controllers using the first level controller to perform a first interaction relating to a task based on the state of the RL neural network; generate a control command using the selected second level controller based on the corresponding input data; and control operation of the robot by executing control command.

First claim

We claim:

1. A controller for controlling a robot, comprising:
a memory configured to store a hierarchical multimodal reinforcement learning (RL) neural network, wherein
the hierarchical multimodal RL neural network includes a first level controller and at least three second level controllers, the at least three second level controllers comprising a first sub level controller, a second sub level controller and a third sub level controller, wherein the hierarchical multimodal RL neural network is configured to receive input data via an input interface of the robot such that
the first sub level controller is configured to receive the input data based on a set of predefined questions,
the second sub level controller is configured to receive the input data by generating a validation question based on a state of the hierarchical multimodal RL neural network, and
the third sub level controller is configured to obtain the input data based on the state of the hierarchical multimodal RL neural network; and
a processor configured to:
select, using the first level controller, one of the at least three second level controllers to perform a first interaction relating to a task, based at least on the state of the hierarchical multimodal RL neural network;
generate, using the selected second level controller, a control command based on the corresponding input data; and
control an operation of the robot by executing the control command.

2. The controller of claim 1 , wherein when the selected second level controller is the first sub level controller, the processor is further configured to:
cause the first sub level controller to select a first predefined question from the set of predefined questions based on the state of the hierarchical multimodal RL neural network;
cause the first sub level controller to receive a first input data comprising a natural language instruction; and
cause the first sub level controller to generate the control command based on the instruction.

3. The controller of claim 1 , wherein when the selected second level controller is the second sub level controller, the processor is further configured to:
receive a set of image frames, the set of image frames indicating at least depth occupancy map, or a panoramic view surrounding the robot;
cause the second sub level controller to generate the validation question based on the state of the hierarchical multimodal RL neural network;
cause the second sub level controller to receive a second input data comprising a natural language validation response; and
cause the second sub level controller to generate the control command based on the validation response.

4. The controller of claim 3 , wherein when the second input data comprises the validation response and natural language data, the processor is further configured to:
cause the second sub level controller to generate the control command based on the validation response; and
update the state of the hierarchical multimodal RL neural network based on the natural language data and the execution of the control command.

5. The controller of claim 3 , wherein when the second input data comprises a negative validation response or the second sub level controller fails to receive the second input data, the processor is further configured to:
cause to terminate operation of the second sub level controller; and
cause to execute at least one of: the first sub level controller or the third sub level controller for performing the first interaction, based on an external input.

6. The controller of claim 3 , wherein when the second input data comprises a negative validation response or the second sub level controller fails to receive the second input data, the processor is further configured to:
cause the second sub level controller to generate a first control command based on the state of the hierarchical multimodal RL neural network;
cause the second sub level controller to control the robot based on the first control command;
cause the second sub level controller to update the state of the hierarchical multimodal RL neural network based on the execution of the first control command; and
cause the second sub level controller to generate a second validation question based on the updated state of the hierarchical multimodal RL neural network.

7. The controller of claim 1 , wherein the hierarchical multimodal RL neural network is trained end-to-end using the reinforcement learning without incurring a penalty when the validation question generated by the second sub level controller forms an answer to one of the set of predefined question used by the first sub level controller to generate the control command.

8. The controller of claim 1 , wherein the input data received by at least one of the first sub level controller or the second sub level controller is generated by an oracle.

9. The controller of claim 8 , wherein the processor is further configured to:
receive a validation response for the validation question from the oracle, based on the oracle decoding the validation question and providing the validation response based on a matching between a shortest path between a current pose of the robot and a goal pose of the task, and a path based on the decoded validation question; and
train the hierarchical multimodal RL neural network based on the validation response.

10. The controller of claim 1 , wherein when the selected second level controller is the third sub level controller, the processor is further configured to:
cause the third sub level controller to generate the control command based on the state of the hierarchical multimodal RL neural network.

11. The controller of claim 1 , wherein the corresponding input data received by at least one of the first sub level controller or the second sub level controller is in natural language.

12. The controller of claim 1 , wherein the processor is further configured to:
update, using the selected second level controller, the state of the hierarchical multimodal RL neural network based on the execution of the first interaction, wherein the first interaction is performed between the controller and at least one of an environment, or an entity associated with the task; and
select, using the first level controller, one of the at least three second level controllers to perform a second interaction relating to the task, based at least on the input data and the updated state of the hierarchical multimodal RL neural network, wherein the second interaction is performed after the first interaction for completing the task.

13. The controller of claim 1 , wherein the processor is further configured to:
determine a resource constraint associated with selection of each of the at least three second level controllers; and
select, using the first level controller, one of the at least three second level controllers, based on the determined resource constraint.

14. The controller of claim 1 , wherein at least one of: the input data or the state of the hierarchical multimodal RL neural network, as used by the first sub level controller, the second sub level controller and the third sub level controller are different.

15. The controller of claim 1 , wherein
the first sub level controller is configured to determine the control command based on a natural language instruction received in response to a first predefined question from the set of predefined questions;
the second sub level controller is configured to determine the control command based on a validation response received in response to the generated validation question; and
the third sub level contr

Assignees

• Mitsubishi Electric Res Laboratories Inc

Inventors

• Cherian Anoop
• Liu Xiulong
• Paul Sudipta
• Chatterjee Moitreya

Classifications

• B25J13/003

  by means of an audio-responsive input (audible safety signals B25J19/061) · CPC title

• B25J9/161 Primary

  Hardware, e.g. neural networks, fuzzy logic, interfaces, processor · CPC title

• B25J9/1664

  characterised by motion, path, trajectory planning · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• B25J9/163 Primary

  learning, adaptive, model based, rule based expert control · CPC title