Three-dimensional model generation

Patent metadata

Field	Value
Publication number	US-2016335809-A1
Application number	US-201514857289-A
Country	US
Kind code	A1
Filing date	Sep 17, 2015
Priority date	May 14, 2015
Publication date	Nov 17, 2016
Grant date	—

Abstract

A method for texture reconstruction associated with a three-dimensional scan of an object includes scanning, at a processor, a sequence of image frames captured by an image capture device at different three-dimensional viewpoints. The method also includes generating a composite confidence map based on the sequence of image frames. The composite confidence map includes pixel values for scanned pixels in the sequence of image frames. The method further includes identifying one or more holes of a three-dimensional model based on the composite confidence map.

First claim

1 . An apparatus comprising:
interface circuitry configured to receive a sequence of image frames associated with a three-dimensional scan of an object; and a processor configured to:
generate a composite confidence map based on the sequence of image frames, the composite confidence map including pixel values for scanned pixels in the sequence of image frames; and
identify one or more holes of a three-dimensional model based on the composite confidence map.

2 . The apparatus of claim 1 , wherein the processor is further configured to fill in at least one hole using color-per-vertex rendering and using the composite confidence map as an input channel.

3 . The apparatus of claim 1 , wherein the processor is further configured to identify hole triangles of the three-dimensional model based on the composite confidence map.

4 . The apparatus of claim 3 , wherein the hole triangles correspond to portions of the object that an image capture device failed to capture, portions of the object that are not included in the sequence of image frames, or a combination thereof.

5 . The apparatus of claim 3 , wherein the hole triangles correspond to triangles of the three-dimensional model that include texture pixels without pixel values.

6 . The apparatus of claim 3 , wherein the processor is further configured to fill in the hole triangles using color-per-vertex rendering and using the composite confidence map as an input channel.

7 . The apparatus of claim 1 , wherein the processor and the interface circuitry are integrated into a mobile device.

8 . The apparatus of claim 1 , wherein the processor and the interface circuitry are integrated into an aircraft, an automobile, a drone, a robot, a camera, an unmanned vehicle, or a processing system communicatively coupled to one or more mounted cameras.

9 . A method for texture reconstruction associated with a three-dimensional scan of an object, the method comprising:
scanning, at a processor, a sequence of image frames captured by an image capture device, wherein each image frame in the sequence of image frames is captured at different three-dimensional viewpoints; generating a composite confidence map based on the sequence of image frames, the composite confidence map including pixel values for scanned pixels in the sequence of image frames; and identifying one or more holes of a three-dimensional model based on the composite confidence map.

10 . The method of claim 9 , further comprising filling in at least one hole using color-per-vertex rendering and using the composite confidence map as an input channel.

11 . The method of claim 9 , further comprising identifying hole triangles of the three-dimensional model based on the composite confidence map.

12 . The method of claim 11 , wherein the hole triangles correspond to portions of the object that the image capture device failed to capture, portions of the object that are not included in the sequence of image frames, or a combination thereof.

13 . The method of claim 11 , wherein the hole triangles correspond to triangles of the three-dimensional model that include texture pixels without pixel values.

14 . The method of claim 11 , further comprising filling in the hole triangles using color-per-vertex rendering and using the composite confidence map as an input channel.

15 . The method of claim 9 , wherein the processor and the image capture device are integrated into a mobile device.

16 . The method of claim 9 , wherein the processor and the image capture device are integrated into an aircraft, an automobile, a drone, a robot, a camera, an unmanned vehicle, or a processing system communicatively coupled to one or more mounted cameras.

17 . A non-transitory computer-readable medium comprising instructions for texture reconstruction associated with a three-dimensional scan of an object, the instructions, when executed by processor, cause the processor to perform operations comprising:
scanning a sequence of image frames captured by an image capture device, wherein each image frame in the sequence of image frames is captured at different three-dimensional viewpoints; generating a composite confidence map based on the sequence of image frames, the composite confidence map including pixel values for scanned pixels in the sequence of image frames; and identifying one or more holes of a three-dimensional model based on the composite confidence map.

18 . The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise filling in at least one hole using color-per-vertex rendering and using the composite confidence map as an input channel.

19 . The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise identifying hole triangles of the three-dimensional model based on the composite confidence map.

20 . The non-transitory computer-readable medium of claim 19 , wherein the hole triangles correspond to portions of the object that the image capture device failed to capture, portions of the object that are not included in the sequence of image frames, or a combination thereof.

21 . The non-transitory computer-readable medium of claim 19 , wherein the hole triangle correspond to triangles of the three-dimensional model that include texture pixels without pixel values.

22 . The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise filling in the hole triangles using color-per-vertex rendering and using the composite confidence map as an input channel.

23 . The non-transitory computer-readable medium of claim 17 , wherein the processor and the image capture device are integrated into a mobile device.

24 . The non-transitory computer-readable medium of claim 17 , wherein the processor and the image capture device are integrated into an aircraft, an automobile, a drone, a robot, a camera, an unmanned vehicle, or a processing system communicatively coupled to one or more mounted cameras.

25 . A method for determining a color of a texture pixel associated with a three-dimensional scan of an object, the method comprising:
generating a camera pose error correction matte at a processor, wherein generating the camera pose error correction matte comprises:
rendering a depth map with depth culling using perspective key frame camera information;
generating an external silhouette matte in camera space based on the rendered depth map;
generating an internal silhouette matte in camera space based on the rendered depth map; and
performing fall-off blend processing on the external silhouette matte and on the internal silhouette matte;
generating a camera seam matte; and determining a color of a particular texture pixel based on the camera pose error correction matte and the camera seam matte.

26 . The method of claim 25 , wherein generating the external silhouette matte comprises marking edge pixels within a radius of an image frame.

27 . The method of claim 25 , wherein generating the internal silhouette matte comprises marking edge pixels within a configurable radius of an image frame.

28 . The method of claim 25 , further comprising applying a blur operation to the external silhouette, to the internal silhouette, or a combination thereof.

29 . The method of claim 25 , wherein the processor is integrated into a mobile

Assignees

• Qualcomm Inc

Inventors

• Forutanpour Bijan
• Sarkis Michel Adib
• Sundaresan Sairam

Classifications

• G06T2200/32

  involving image mosaicing · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T15/04

  Texture mapping · CPC title

• G06T7/579 Primary

  from motion · CPC title

• G06T7/50

  Depth or shape recovery · CPC title