Parameterized waypoint generation on dynamically parented non-static objects for robotic autonomous tasks

Patent metadata

Field	Value
Publication number	US-12103185-B2
Application number	US-202117198152-A
Country	US
Kind code	B2
Filing date	Mar 10, 2021
Priority date	Mar 10, 2021
Publication date	Oct 1, 2024
Grant date	Oct 1, 2024

Abstract

In one embodiment, a method includes generating a trajectory plan to complete a task to be executed by a robotic system, identifying objects in the environment required for completing the task, determining attributes for each of the identified objects, determining trajectory-parameters for the trajectory plan based on the determined attributes for each identified object and operational conditions in an environment associated with the robotic system, and executing the task based on the determined trajectory-parameters for the trajectory plan.

First claim

What is claimed is:

1. A method comprising, by a robotic system:
accessing a natural-language input regarding a task to be executed by the robotic system;
determining a semantic meaning of the natural-language input to identify, from the natural-language input, (1) the task and (2) one or more objects for the robotic system to use to complete the task;
generating, for the task to be executed by the robotic system, a trajectory plan to complete the task by selecting a predetermined, parameterized trajectory based on the identified task, wherein the parameterized trajectory comprises at least one group of waypoints for which modifications apply collectively to the waypoints in the group, and wherein the parameterized trajectory is associated with one or more changeable trajectory parameters, each changeable trajectory parameter associated with a predetermined allowable value range;
identifying one or more objects in an environment of robotic system that correspond to the one or more objects identified from the natural-language input to use for completing the task;
identifying, for each of the one or more objects, an object type from a set of a plurality of predetermined object types, wherein each object type is distinct from a shape;
determining, for each of the one or more objects and based on the respective identified object type, one or more predetermined trajectory-influencing attributes of the respective identified object type, wherein the one or more predetermined trajectory-influencing attributes comprises one or more of an object size, an object shape, an object color, an object texture, and an object orientation;
determining, for each of the one or more objects, whether each of the changeable trajectory parameters corresponds to at least one determined trajectory-influencing attribute;
determining, for each of the one or more objects, a respective object-specific value for each of the one or more predetermined trajectory-influencing attributes;
determining, for each of the one or more objects, whether each object-specific value is within the predetermined allowable value range associated with the corresponding changeable trajectory parameter;
in response to a determination that (1) each changeable trajectory parameter corresponds to at least one determined trajectory-influencing attribute and (2) each object-specific value is within the predetermined allowable value range associated with the corresponding changeable trajectory parameter, then updating the trajectory plan by applying the same transformation to each waypoint of one or more of the at least one group of waypoints of the selected predetermined, parameterized trajectory, wherein the transformation is based on the object-specific values for the one or more predetermined trajectory-influencing attributes; and
executing the task based on the updated trajectory plan.

2. The method of claim 1 , wherein the at least one group of waypoints are a subset of a series of waypoints in a three-dimensional (3D) space in the environment associated with the robotic system.

3. The method of claim 2 , wherein updating the trajectory plan based on the object-specific values for the one or more predetermined trajectory-influencing attributes comprises one or more of: scaling the subset of waypoints, rotating the subset of waypoints, or moving the subset of waypoints.

4. The method of claim 1 , wherein the one or more objects are identified based on sensor data captured by one or more sensors associated with the robotic system.

5. The method of claim 1 , wherein one or more of the object-specific values are determined based on sensor data captured by one or more sensors associated with the robotic system.

6. The method of claim 1 , further comprising:
accessing a scene database comprising a plurality of data entries, wherein each data entry comprises an object identifier (ID), an object type associated with the object ID, and the one or more predetermined trajectory-influencing attributes associated with that object type.

7. The method of claim 6 , wherein determining, for each of the one or more objects and based on the respective identified object type, one or more predetermined trajectory-influencing attributes of the respective identified object comprises:
comparing the one or more objects with the plurality of data entries in the scene database to identify one or more object IDs matching one or more of the one or more objects; and
retrieving, from the scene database, the one or more predetermined trajectory-influencing attributes corresponding to the identified one or more object IDs.

8. A robotic system comprising:
a computing system with control software;
a robot controller;
one or more robotic limbs;
one or more non-transitory computer-readable storage media including instructions; and
one or more processors coupled to the storage media, the one or more processors configured to execute the instructions to:
access a natural-language input regarding a task to be executed by the robotic system;
determine a semantic meaning of the natural-language input to identify, from the natural-language input, (1) the task and (2) one or more objects for the robotic system to use to complete the task;
generate, for the task to be executed by the robotic system, a trajectory plan to complete the task by selecting a predetermined, parameterized trajectory based on the identified task, wherein the parameterized trajectory comprises at least one group of waypoints for which modifications apply collectively to the waypoints in the group, and wherein the parameterized trajectory is associated with one or more changeable trajectory parameters, each changeable trajectory parameter associated with a predetermined allowable value range;
identify one or more objects in an environment of the robotic system that correspond to the one or more objects identified from the natural-language input to use for completing the task;
identify, for each of the one or more objects, an object type from a set of a plurality of predetermined object types, wherein each object type is distinct from a shape;
determine, for each of the one or more objects and based on the respective identified object type, one or more predetermined trajectory-influencing attributes of the respective identified object type, wherein the one or more predetermined trajectory-influencing attributes comprises one or more of an object size, an object shape, an object color, an object texture, and an object orientation;
determine, for each of the one or more objects, whether each of the changeable trajectory parameters corresponds to at least one determined trajectory-influencing attribute;
determine, for each of the one or more objects, a respective object-specific value for each of the one or more predetermined trajectory-influencing attributes;
determine, for each of the one or more objects, whether each object-specific value is within the predetermined allowable value range associated with the corresponding changeable trajectory parameter;
in response to a determination that (1) each changeable trajectory parameter corresponds to at least one determined trajectory-influencing attribute and (2) each object-specific value is within the predetermined allowable value range associated with the corresponding changeable trajectory parameter, then update the trajectory plan by applying the same transformation to each waypoint of one or more of the at least one group of waypoints of the selected predetermined, parameterized trajectory, wherein the transformation is defined by the object-specific values for the one or more predetermined trajectory-influencing attributes; and
execute the task based on the updated trajectory plan.

9.

Assignees

• Samsung Electronics Co Ltd
• Samsung Electronics Co Ltd

Inventors

• Harms Brian
• Hajash Kathleen Sofia
• Schoessler Philipp
• Mason Dane

Classifications

• G05B2219/40564

  Recognize shape, contour of object, extract position and orientation · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• B25J11/0045

  Manipulators used in the food industry · CPC title

• G05B2219/40395

  Compose movement with primitive movement segments from database · CPC title

• B25J9/1661

  characterised by task planning, object-oriented languages · CPC title