Motion planning

Patent metadata

Field	Value
Publication number	US-12533806-B2
Application number	US-202318243467-A
Country	US
Kind code	B2
Filing date	Sep 7, 2023
Priority date	Oct 28, 2022
Publication date	Jan 27, 2026
Grant date	Jan 27, 2026

Abstract

Apparatuses, systems, and techniques to generate a motion plan. In at least one embodiment, a motion plan is generated using a neural network based, at least in part, on a demonstration of a task.

First claim

What is claimed is:

1 . A method, comprising:
a computing device,
observing a demonstration depicting a first object having a first geometry,
calculating states comprising information related to movement of the first object during the demonstration,
generating at least one contact mode prediction based, at least in part, on the states of the first object and data representing a different second geometry of a second object, and
determining a motion plan based, at least in part, on the at least one contact mode prediction; and
a robot, performing a task using the second object and in accordance with the motion plan.

2 . The method of claim 1 , wherein the demonstration depicts at least the task being performing using the first object.

3 . The method of claim 1 , wherein the at least one contact mode prediction is generated in connection with one or more neural networks.

4 . A system, comprising:
at least one processor; and at least one memory comprising instructions that, in response to being performed by the at least one processor, cause the system to at least:
obtain a demonstration of a task involving at least a first object having a first geometry;
determine states comprising information related to movement of the first object during the demonstration;
use a neural network to generate a set of contact modes based, at least in part, on the states of the first object and data representing a different second geometry of a second object; and
generate a motion plan based, at least in part, on the set of contact modes, wherein the motion plan is to cause at least a robot to perform the task in connection with the second object.

5 . The system of claim 4 , wherein the at least one memory comprises further instructions that, in response to being performed by the at least one processor, cause the system to at least:
provide the motion plan to the robot, wherein the robot is in an environment comprising the second object; and cause the robot to utilize the motion plan to perform the task in the environment using at least the second object.

6 . The system of claim 4 , wherein the set of contact modes indicates at least one or more points of contact associated with the second object.

7 . The system of claim 4 , wherein the at least one memory comprises further instructions that, in response to being performed by the at least one processor, cause the system to at least:
generate a first portion of the motion plan based, at least in part, on a first contact mode of the set of contact modes; and generate a second portion of the motion plan based, at least in part, on a second contact mode of the set of contact modes.

8 . The system of claim 4 , wherein the at least one memory comprises further instructions that, in response to being performed by the at least one processor, cause the system to at least:
generate the motion plan in connection with one or more optimization processes.

9 . The system of claim 4 , wherein the demonstration includes point cloud data.

10 . The system of claim 4 , further comprising:
at least one image capture device, obtaining the demonstration of the task comprising causing the at least one image capture device to capture at least one image of a real-world demonstration; and at least one of an autonomous machine or a semi-autonomous machine to perform the task.

11 . The system of claim 4 , wherein the system is comprised in at least one of an autonomous machine or a semi-autonomous machine.

12 . A method, comprising:
calculating states comprising information related to movement of a first object having a first geometry based, at least in part, on a demonstration depicting at least the first object; generating one or more contact mode predictions based, at least in part, on the states of the first object and data representing a different second geometry of a second object; and determining a motion plan based, at least in part, on the one or more contact mode predictions, wherein the motion plan causes a robot to perform a task using the second object and in accordance with the motion plan.

13 . The method of claim 12 , further comprising:
causing the robot, comprising at least one of a real-world or virtual device, to perform one or more actions based, at least in part, on the motion plan to perform the task using the second object in another environment.

14 . The method of claim 12 , wherein the demonstration depicts at least one or more robots performing the task using the object.

15 . The method of claim 12 , wherein the one or more contact mode predictions are generated using a neural network.

16 . The method of claim 15 , wherein the neural network is a feedforward neural network.

17 . The method of claim 15 , further comprising:
generating one or more sets of the states and contact modes for one or more objects; and training the neural network based, at least in part, on the one or more sets of the states and contact modes.

18 . The method of claim 12 , wherein the demonstration depicts at least the task being performing using the first object.

19 . The method of claim 12 , wherein the demonstration is performed in a first environment and the at least one real-world or virtual device is to perform the task in a different second environment.

20 . The method of claim 12 , further comprising:
performing a simulation comprising the demonstration.

21 . A non-transitory computer-readable medium comprising instructions that, when performed by at least one processor of a computing device, cause the computing device to at least:
determine states comprising information related to movement of a first object, having a first geometry, during a demonstration of a performance of a task involving the first object; use one or more machine learning models to generate a set of contact modes based, at least in part, on the states of the first object and data representing a different second geometry of a second object; and generate a motion plan based, at least in part, on the set of contact modes and data related to at least one robot, wherein the motion plan causes the at least one robot to perform the task using the second object.

22 . The non-transitory computer-readable medium of claim 21 , comprising further instructions that when performed by the at least one processor of the computing device, cause the computing device to at least:
use the one or more machine learning models to generate a second set of contact modes based, at least in part, on the states and the second object; and generate a second motion plan based, at least in part, on the second set of contact modes, wherein the second motion plan causes the at least one robot to perform the task using the second object.

23 . The non-transitory computer-readable medium of claim 21 , wherein the set of contact modes comprises at least a first contact mode that is based, at least in part, on a first start state of the first object and a first end state of the first object of the states.

24 . The non-transitory computer-readable medium of claim 21 ,
wherein the motion plan includes at least one or more instructions executable by the at least one robot.

25 . The non-transitory computer-readable medium of claim 21 , comprising further instructions that when performed by the at least one processor of the computing device, cause the computing device to at least:
d

Assignees

• Nvidia Corp

Inventors

• Wu Albert
• Eppner Clemens
• Fox Dieter

Classifications

• B25J9/1697

  Vision controlled systems · CPC title

• B25J9/1671

  characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems · CPC title

• B25J9/1653

  parameters identification, estimation, stiffness, accuracy, error analysis · CPC title

• B25J9/163

  learning, adaptive, model based, rule based expert control · CPC title

• B25J9/161

  Hardware, e.g. neural networks, fuzzy logic, interfaces, processor · CPC title