System and method for assisted 3D scanning

Patent metadata

Field	Value
Publication number	US-9912862-B2
Application number	US-201715445735-A
Country	US
Kind code	B2
Filing date	Feb 28, 2017
Priority date	Feb 29, 2016
Publication date	Mar 6, 2018
Grant date	Mar 6, 2018

Abstract

A three-dimensional scanning system includes: a camera configured to capture images; a processor; and memory coupled to the camera and the processor, the memory being configured to store: the images captured by the camera; and instructions that, when executed by the processor, cause the processor to: control the camera to capture one or more initial images of a subject from a first pose of the camera; compute a guidance map in accordance with the one or more initial images to identify one or more next poses; control the camera to capture one or more additional images from at least one of the one or more next poses; update the guidance map in accordance with the one or more additional images; and output the images captured by the camera to generate a three-dimensional model.

First claim

What is claimed is:

1. A three-dimensional scanning system comprises:
a depth camera configured to capture images, wherein the images comprise depth images;
a processor; and
memory coupled to the depth camera and the processor, the memory being configured to store:
the depth images captured by the depth camera; and
instructions that, when executed by the processor, cause the processor to:
control the depth camera to capture one or more initial depth images of a subject from a first pose of the depth camera;
compute a guidance map in accordance with the one or more initial depth images to identify one or more next poses, the guidance map being computed by:
defining, based on the one or more initial depth images of the subject, an envelope surrounding the subject, the envelope comprising a plurality of patches, each patch corresponding to at least one potential pose of a plurality of potential poses, the plurality of potential poses comprising the one or more next poses; and
computing a plurality of incremental coverage values, each of the incremental coverage values corresponding to one of the plurality of patches and each of the incremental coverage values representing an amount of improvement in a global coverage of the subject provided by capturing an additional depth image with the depth camera at the at least one potential pose of the corresponding one of the patches;

control the depth camera to capture one or more additional depth images from at least one of the one or more next poses;
update the guidance map in accordance with the one or more additional depth images; and
output the depth images captured by the depth camera to generate a three-dimensional model.

2. The three-dimensional scanning system of claim 1 , wherein the depth camera is a stereoscopic depth camera comprising:
a first camera having a first field of view with a first optical axis; and
a second camera spaced apart from the first camera and having a second field of view overlapping the first field of view and having a second optical axis substantially parallel to the first optical axis.

3. The three-dimensional scanning system of claim 1 , further comprising a color camera registered with the depth camera.

4. The three-dimensional scanning system of claim 1 , further comprising a display coupled to the processor,
wherein the memory further stores instructions that cause the processor to control the display to provide visual feedback to a user of the three-dimensional scanning system, the visual feedback comprising a graphical representation of the incremental coverage values of the one or more next poses of the guidance map.

5. The three-dimensional scanning system of claim 4 , wherein the graphical representation is a dome around the subject, and a heat map indicates which portions of the dome correspond to the one or more next poses, the values of the heat map corresponding to the values of the incremental coverage values.

6. The three-dimensional scanning system of claim 1 , wherein the one or more next poses form a path.

7. The three-dimensional scanning system of claim 6 , wherein the path is selected in accordance with increasing coverage of the subject and reducing scanning time.

8. The three-dimensional scanning system of claim 1 , wherein the guidance map is a coverage map.

9. The three-dimensional scanning system of claim 1 , wherein the subject is a container,
wherein the memory further stores instructions that cause the processor to estimate an available volume of the container based on the three-dimensional model, and
wherein the guidance map identifies the one or more next poses to locate corners of the available volume.

10. The three-dimensional scanning system of claim 1 , wherein the subject is a stack of boxes,
wherein the memory further stores instructions that cause the processor to count the number of boxes based on the three-dimensional model, and
wherein the guidance map identifies the one or more next poses to identify boxes having low coverage.

11. A method for providing guided scanning, the method comprising:
controlling, by a processor, a depth camera to capture one or more initial depth images of a subject from a first pose of the depth camera;
computing, by the processor, a guidance map in accordance with the one or more initial depth images to identify one or more next poses, the guidance map being computed by:
defining, based on the one or more initial depth images of the subject, an envelope surrounding the subject, the envelope comprising a plurality of patches, each patch corresponding to at least one potential pose of a plurality of potential poses, the plurality of potential poses comprising the one or more next poses; and
computing a plurality of incremental coverage values, each of the incremental coverage values corresponding to one of the plurality of patches and each of the incremental coverage values representing an amount of improvement in a global coverage of the subject provided by capturing an additional depth image with the depth camera at the at least one potential pose of the corresponding one of the patches;

controlling, by the processor, the depth camera to capture one or more additional depth images from at least one of the one or more next poses;
updating, by the processor, the guidance map in accordance with the one or more additional depth images; and
outputting the depth images captured by the depth camera to generate a three-dimensional model.

12. The method of claim 11 , wherein the depth camera is a stereoscopic depth camera comprising:
a first camera having a first field of view with a first optical axis; and
a second camera spaced apart from the first camera and having a second field of view overlapping the first field of view and having a second optical axis substantially parallel to the first optical axis.

13. The method of claim 11 , further comprising providing visual feedback to a user through a display coupled to the processor, the visual feedback comprising a graphical representation of the incremental coverage values of the one or more next poses of the guidance map.

14. The method of claim 13 , wherein the graphical representation is a dome around the subject, and a heat map indicates which portions of the dome correspond to the one or more next poses, the values of the heat map corresponding to the values of the incremental coverage values.

15. The method of claim 11 , wherein the one or more next poses form a path.

16. The method of claim 15 , wherein the path is selected in accordance with increasing coverage of the subject and reducing scanning time.

17. The method of claim 11 , wherein the subject is a container,
wherein the method further comprises estimating an available volume of the container based on the three-dimensional model, and
wherein the guidance map identifies the one or more next poses to locate corners of the available volume.

18. The method of claim 11 , wherein the subject is a stack of boxes,
wherein the method further comprises counting the number of boxes based on the three-dimensional model, and
wherein the guidance map identifies the one or more next poses to identify boxes having low coverage.

19. A three-dimensional scanning system comprises:
a camera configured to capture images;
a processor; and
memory coupled to the camera and the processor, the memory being configured to store:
the images captured by the camera; and
instructions that, when executed by the processor, cause the processor to:
control the camer

Assignees

• Aquifi Inc

Inventors

• Peruch Francesco
• Murali Giridhar
• Marin Giulio
• Rafii Abbas
• Tieu Kinh

Classifications

• H04N23/63

  by using electronic viewfinders · CPC title

• H04N23/64 Primary

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• G01B11/00 Primary

  Measuring arrangements characterised by the use of optical techniques · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N9/04

  Electricity · mapped topic