Data selection based on uncertainty quantification

Patent metadata

Field	Value
Publication number	US-11941899-B2
Application number	US-202117331451-A
Country	US
Kind code	B2
Filing date	May 26, 2021
Priority date	May 26, 2021
Publication date	Mar 26, 2024
Grant date	Mar 26, 2024

Abstract

Apparatuses, systems, and techniques generate poses of an object based on image data of the object obtained from a first viewpoint of the object and a second viewpoint of the object. The poses can be evaluated to determine a portion of the image data usable by an estimator to generate a pose of the object.

First claim

What is claimed is:

1. A computer system comprising one or more processors and computer-readable memory storing instructions executable by the one or more processors to cause the computer system to at least:
perform a first comparison of first results of a first estimator and a second estimator, the first results obtained by inputting, into the first estimator and the second estimator, image data representing an object captured at least from a first viewpoint and a second viewpoint, the first results of the first estimator and the second estimator based on the image data representing the object captured from the first viewpoint;
perform a second comparison of second results of the first estimator and the second estimator, the second results obtained by inputting, into the first estimator and the second estimator, the image data representing the object captured from the first viewpoint and the second viewpoint, the second results of the first estimator and the second estimator based on the image data representing the object captured from the second viewpoint; and
based at least in part on the first comparison and the second comparison, select the image data representing the object captured at least from the first viewpoint or the image data representing the object captured at least from the second viewpoint to be obtained by the first estimator or the second estimator.

2. The computer system according to claim 1 , wherein the first estimator is a first neural network and the second estimator is a second neural network.

3. The computer system according to claim 2 , wherein the first neural network and the second neural network are equivalent neural networks, the first neural network trained using a first data set and the second neural network trained using a second data set, the first data set being different than the second data set.

4. The computer system according to claim 2 , wherein the first neural network and the second neural network are different neural networks, the first neural network and the second neural network trained using a common data set.

5. The computer system according to claim 1 , wherein performing the first comparison generates a first value representing a first uncertainty quantification and performing the second comparison generates a second value representing a second uncertainty quantification.

6. The computer system according to claim 1 , wherein the first results comprise a first bounding box and a second bounding box and the second results comprise a third bounding box and a fourth bounding box, the first and third bounding boxes generated by the first estimator and the second and fourth bounding boxes generated by the second estimator.

7. The computer system according to claim 1 , wherein the instructions executable by the one or more processors are further to cause the computer system to at least:
cause the first estimator or the second estimator to generate a grasp pose corresponding to the object; and
based on the grasp pose, control a robot to grasp the object using a robotic manipulator of the robot.

8. The computer system according to claim 1 , wherein the instructions executable by the one or more processors are further to cause the computer system to at least:
cause the first estimator or the second estimator to generate a pose corresponding to the object; and
based on the pose, control a computer-implemented device.

9. A device comprising:
one or more processors and memory storing executable instructions that, as a result of being executed by the one or more processors, cause the device to:
generate a first set of image data based on an object viewed from a first viewpoint;
generate a second set of image data based on the object viewed from a second viewpoint;
determine a first value based on the first set of image data and a second value based on the second set of image data; and
based on the first and second values, select the first set of image data or the second set of image data to be processed.

10. The device according to claim 9 , wherein the first viewpoint is different from the second viewpoint.

11. The device according to claim 9 , wherein generating the first set of image data based on the object viewed from the first viewpoint comprises:
processing the image data of the object viewed from the first viewpoint with a first neural network to generate a first portion of image data in the first set of image data; and
processing the image data of the object viewed from the first viewpoint with a second neural network to generate a second portion of image data in the first set of image data.

12. The device according to claim 9 , wherein generating the second set of image data based on the object viewed from the second viewpoint comprises:
processing the image data of the object viewed from the second viewpoint with a first neural network to generate a first portion of image data in the second set of image data; and
processing the image data of the object viewed from the second viewpoint with a second neural network to generate a second portion of image data in the second set of image data.

13. The device according to claim 9 , wherein determining the first value comprises calculating an average disagreement between discrete image data portions associated with the first set of image data, and wherein determining the second value comprises calculating an average disagreement between discrete image data portions associated with the second set of image data.

14. The device according to claim 9 , wherein the object viewed from the first viewpoint is captured by a camera associated with the device and the object viewed from the second viewpoint is captured by the camera associated with the device.

15. A computer-implemented method comprising:
generating a plurality of evaluations by at least, for each viewpoint of a plurality of viewpoints, evaluating outputs of estimators applied to image data representing an image of an object captured from the viewpoint; and
selecting, based at least in part on the plurality of evaluations, data representing the image of the object captured from one of the plurality of viewpoints.

16. The computer-implemented method according to claim 15 , wherein individual estimators of the estimators are implemented by a neural network.

17. The computer-implemented method according to claim 16 , wherein individual neural networks of the neural networks are implemented by an equivalent neural network, and each neural network of the neural networks is trained with a distinct data set.

18. The computer-implemented method according to claim 16 , wherein individual neural networks of the neural networks are different neural networks, and each neural network of the neural networks is trained with a common data set.

19. The computer-implemented method according to claim 15 , wherein the plurality of evaluations comprise evaluations generated based on the outputs of the estimators comprising at least object poses generated by the estimators based on the image data representing the object.

20. The computer-implemented method according to claim 19 , wherein the object poses comprise bounding cuboids associated with the object.

21. A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
obtain a first set of object poses generated by a plurality of estimators based on image data;
obtain a second set of object

Assignees

• Nvidia Corp

Inventors

• Tremblay Jonathan
• Tozeto Ramos Fabio
• Zhu Yuke
• Anandkumar Anima
• Shi Guanya

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V20/653 Primary

  by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06N3/045

  Combinations of networks · CPC title