Object-focused active three-dimensional reconstruction

Patent metadata

Field	Value
Publication number	US-10372968-B2
Application number	US-201615192857-A
Country	US
Kind code	B2
Filing date	Jun 24, 2016
Priority date	Jan 22, 2016
Publication date	Aug 6, 2019
Grant date	Aug 6, 2019

Abstract

A method for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning includes recognizing and localizing an object in a two-dimensional (2D) image. The method also includes computing 3D depth maps for the localized object. A 3D object map is constructed from the depth maps. A sampling based structure is grown around the 3D object map and a cost is assigned to each edge of the sampling based structure. The sampling based structure may be searched to determine a lowest cost sequence of edges that may, in turn be used to guide the robot.

First claim

What is claimed is:

1. A method for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning, comprising:
identifying an object of interest;
searching for the object of interest in an environment comprising a plurality of objects;
recognizing and localizing the object of interest in a plurality of two-dimensional (2D) images of the environment captured via the camera;
constructing a 3D object map based on the localized object in the plurality of 2D images, a depth variance associated with pixels of the 3D object map;
growing a sampling based structure around the 3D object map;
assigning a cost to each edge of the sampling based structure based on the depth variance of pixels visible along a given edge;
searching the sampling based structure to determine a lowest cost sequence of edges; and
guiding the robot through the environment based on the lowest cost sequence of edges.

2. The method of claim 1 , further comprising:
computing a plurality of 3D depth maps for the localized object based on the plurality of 2D images; and
constructing the 3D object map from the plurality of 3D depth maps.

3. The method of claim 1 , further comprising guiding the robot through the environment based on texture information of the object of interest.

4. The method of claim 1 , further comprising guiding the robot through the environment based on importance weights assigned to different portions of the object of interest.

5. The method of claim 1 , further comprising guiding the robot through the environment by incrementally creating a sampling based motion planning framework.

6. An apparatus for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning, comprising:
a memory; and
at least one processor coupled to the memory, the at least one processor configured:
to identify an object of interest;
to search for the object of interest in an environment comprising a plurality of objects;
to recognize and localize the object of interest in a plurality of two-dimensional (2D) images of the environment captured via the camera;
to construct a 3D object map based on the localized object in the plurality of 2D images, a depth variance associated with pixels of the 3D object map;
to grow a sampling based structure around the 3D object map;
to assign a cost to each edge of the sampling based structure based on the depth variance of pixels visible along a given edge;
to search the sampling based structure to determine a lowest cost sequence of edges; and
to guide the robot through the environment based on the lowest cost sequence of edges.

7. The apparatus of claim 6 , in which the at least one processor is further configured:
to compute a plurality of 3D depth maps for the localized object based on the plurality of 2D images; and
to construct the 3D object map from the plurality of 3D depth maps.

8. The apparatus of claim 6 , in which the at least one processor is further configured to guide the robot through the environment based on texture information of the object of interest.

9. The apparatus of claim 6 , in which the at least one processor is further configured to guide the robot through the environment based on importance weights assigned to different portions of the object of interest.

10. The apparatus of claim 6 , in which the at least one processor is further configured to guide the robot through the environment by incrementally creating a sampling based motion planning framework.

11. An apparatus for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning, comprising:
means for identifying an object of interest;
means for searching for the object of interest in an environment comprising a plurality of objects;
means for recognizing and localizing the object of interest in a plurality of two-dimensional (2D) images of the environment captured via the camera;
means for constructing a 3D object map based on the localized object in the plurality of 2D images, a depth variance associated with pixels of the 3D object map;
means for growing a sampling based structure around the 3D object map;
means for assigning a cost to each edge of the sampling based structure based on the depth variance of pixels visible along a given edge;
means for searching the sampling based structure to determine a lowest cost sequence of edges; and
means for guiding the robot through the environment based on the lowest cost sequence of edges.

12. The apparatus of claim 11 , further comprising:
means for computing a plurality of 3D depth maps for the localized object based on the plurality of 2D images; and
means for constructing the 3D object map from the plurality of 3D depth maps.

13. The apparatus of claim 11 , further comprising means for guiding the robot through the environment based on texture information of the object of interest.

14. The apparatus of claim 11 , further comprising means for guiding the robot through the environment based on importance weights assigned to different portions of the object of interest.

15. The apparatus of claim 11 , further comprising means for guiding the robot through the environment by incrementally creating a sampling based motion planning framework.

16. A non-transitory computer readable medium having encoded thereon program code for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning, the program code executed by a processor and comprising:
program code to identify an object of interest;
program code to search for the object of interest in the environment comprising a plurality of objects;
program code to recognize and localize the object of interest in a plurality of two-dimensional (2D) images of the environment captured via the camera;
program code to construct a 3D object map based on the localized object in the plurality of 2D images, a depth variance associated with pixels of the 3D object map;
program code to grow a sampling based structure around the 3D object map;
program code to assign a cost to each edge of the sampling based structure based on the depth variance of pixels visible along a given edge;
program code to search the sampling based structure to determine a lowest cost sequence of edges; and
program code to guide the robot through the environment based on the lowest cost sequence of edges.

17. The non-transitory computer readable medium of claim 16 , further comprising:
program code to compute a plurality of 3D depth maps for the localized object based on the plurality of 2D images; and
program code to construct the 3D object map from the plurality of 3D depth maps.

18. The non-transitory computer readable medium of claim 16 , further comprising program code to guide the robot through the environment based on texture information of the object of interest.

19. The non-transitory computer readable medium of claim 16 , further comprising program code to guide the robot through the environment based on importance weights assigned to different portions of the object of interest.

20. The non-transitory computer readable medium of claim 16 , further comprising program code to guide the robot through the environment by incrementally creating a sampling based motion planning framework.

Assignees

• Qualcomm Inc

Inventors

• Aghamohammadi Aliakbar
• Najafi Shoushtari Seyed Hesameddin
• Towal Regan Blythe

Classifications

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/42

  Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation · CPC title

• G06T7/529

  from texture · CPC title

• G05B2219/40504

  Simultaneous trajectory and camera planning · CPC title

• G06T7/13

  Edge detection · CPC title