Apparatus and method for controlling robotic manipulators

Patent metadata

Field	Value
Publication number	US-12544931-B2
Application number	US-202318506672-A
Country	US
Kind code	B2
Filing date	Nov 10, 2023
Priority date	Mar 14, 2023
Publication date	Feb 10, 2026
Grant date	Feb 10, 2026

Abstract

A method for controlling a robot including a manipulator is provided. The method may include: acquiring an image of a scene including a target object; inputting the image into a control policy model to obtain an object placement pose of the manipulator, as an output of the control policy model, wherein the control policy model is generated using data collected by a data collection system that is configured to acquire an object retrieval trajectory by observing a robot movement for object retrieval, and reverse the object retrieval trajectory into an object placement trajectory; and commanding the robot to position the manipulator according to the object placement pose, to place the target object at a designated location.

First claim

What is claimed is:

1 . An electronic device for controlling a robot including a manipulator, the electronic device comprising:
one or more processors configured to: acquire an image of a scene including a target object; input the image into a control policy model to obtain an object placement pose of the manipulator, as an output of the control policy model, wherein the control policy model is generated using data collected by a data collection system that is configured to acquire an object retrieval trajectory by observing a robot movement for object retrieval, and reverse the object retrieval trajectory into an object placement trajectory; and command the robot to position the manipulator according to the object placement pose, to place the target object at a designated location, wherein the data collection system is configured to obtain a first set of manipulator grasping poses, obtain a second set of manipulator grasping poses by pruning grasping poses that do not belong to an object class of interest, from the first set of grasping poses, determine a grasping pose from the second set of manipulator grasping poses, and obtain the object retrieval trajectory that extends from the determined grasping pose to a clearance pose.

2 . The electronic device of claim 1 , further comprising a vision sensor configured to capture the image of the scene including the target object.

3 . The electronic device of claim 1 , wherein the object placement pose of the manipulator that is output from the control policy model comprises:
information of rotational positions comprising a yaw angle, a pitch angle, and a roll angle of the manipulator; and information of transitional positions comprising an x-direction position, a y-direction position, and a z-direction position of the manipulator.

4 . The electronic device of claim 1 , wherein the data collection system is configured to:
obtain the first set of manipulator grasping poses via a grasping planning neural network.

5 . The electronic device of claim 1 , wherein the data collection system comprises a tactile sensor mounted on the manipulator and is configured to:
identify a center position of a plurality of contact points in a contact area between the tactile sensor and the target object; determine a position vector for changing the center position of the plurality of contact points to a predetermined stable position; and output a robot control command based on the position vector.

6 . The electronic device of claim 5 , wherein the data collection system is configured to command the robot to re-grasp the target object based on the robot control command including the position vector to acquire the object retrieval trajectory.

7 . The electronic device of claim 1 , wherein the data collection system is further configured to:
determine a retrieval pose of the manipulator based on a movement path extending from the grasping pose to the clearance pose of the manipulator; position the manipulator according to the retrieval pose and store the retrieval pose to generate the object retrieval trajectory based on the retrieval pose; downsample the object retrieval trajectory to match a predetermined control policy frequency; reverse the downsampled object retrieval trajectory into the object placement trajectory; and generate the control policy model by training a policy learning model based on the object placement trajectory and the image of the scene including the target object.

8 . The electronic device of claim 1 , wherein the data collection system is configured to command the robot to position the manipulator at a grasping position, change a rotational stiffness level and a translational stiffness level of the manipulator from original values to predetermined minimum values, command the robot to close a gripper of the manipulator, and acquire the object retrieval trajectory while the rotational stiffness level and the translational stiffness level are set to the predetermined minimum values.

9 . The electronic device of claim 1 , the control policy model comprises one or more convolutional neural networks (CNNs), followed by a multilayer perceptron (MLP) layer that maps an output of the CNNs into parameters of a distribution of robot control actions.

10 . A method for controlling a robot including a manipulator, the method comprising:
acquiring an image of a scene including a target object; inputting the image into a control policy model to obtain an object placement pose of the manipulator, as an output of the control policy model, wherein the control policy model is generated using data collected by a data collection system that is configured to acquire an object retrieval trajectory by observing a robot movement for object retrieval, and reverse the object retrieval trajectory into an object placement trajectory; and commanding the robot to position the manipulator according to the object placement pose, to place the target object at a designated location, wherein the data collection system is configured to obtain a first set of manipulator grasping poses, obtain a second set of manipulator grasping poses by pruning grasping poses that do not belong to an object class of interest, from the first set of grasping poses, determine a grasping pose from the second set of manipulator grasping poses, and obtain the object retrieval trajectory that extends from the determined grasping pose to a clearance pose.

11 . The method of claim 10 , wherein the object placement pose of the manipulator comprises:
information of rotational positions comprising a yaw angle, a pitch angle, and a roll angle of the manipulator; and information of transitional positions comprising an x-direction position, a y-direction position, and a z-direction position of the manipulator.

12 . The method of claim 10 , further comprising acquiring the object retrieval trajectory by:
obtaining the first set of manipulator grasping poses via a grasping planning neural network.

13 . The method of claim 10 , wherein a tactile sensor mounted on the manipulator, and the method further comprises acquiring the object retrieval trajectory by:
identifying a center position of a plurality of contact points in a contact area between the tactile sensor and the target object; determining a position vector for changing the center position of the plurality of contact points to a predetermined stable position; outputting a robot control command based on the position vector to place the manipulator at the predetermined stable position; and acquiring the object retrieval trajectory by executing an object retrieval task from the predetermined stable position.

14 . The method of claim 10 , further comprising acquiring the object placement trajectory by:
downsampling the object retrieval trajectory to match a predetermined control policy frequency; reversing the downsampled object retrieval trajectory into the object placement trajectory; and generating the control policy model by training a policy learning model based on the object placement trajectory and the image of the scene including the target object.

15 . The method of claim 10 , further comprising acquiring the object retrieval trajectory by:
commanding the robot to position the manipulator at a grasping position; changing a rotational stiffness level and a translational stiffness level of the manipulator from original values to predetermined minimum values; commanding the robot to close a gripper of the manipulator, and acquiring the object retrieval trajectory while the rotational stiffness level and the t

Assignees

• Samsung Electronics Co Ltd

Inventors

• Konar Abhisek
• Limoyo Oliver
• Hogan Francois
• Dudek Gregory Lewis
• Ablett Trevor

Classifications

• B25J9/1669

  characterised by special application, e.g. multi-arm co-operation, assembly, grasping · CPC title

• B25J9/163

  learning, adaptive, model based, rule based expert control · CPC title

• B25J9/1612

  characterised by the hand, wrist, grip control · CPC title

• B25J9/161

  Hardware, e.g. neural networks, fuzzy logic, interfaces, processor · CPC title

• B25J9/1664 Primary

  characterised by motion, path, trajectory planning · CPC title