3D model construction from 2D assets

Patent metadata

Field	Value
Publication number	US-10453252-B2
Application number	US-201715589387-A
Country	US
Kind code	B2
Filing date	May 8, 2017
Priority date	May 8, 2017
Publication date	Oct 22, 2019
Grant date	Oct 22, 2019

Abstract

Features of the surface of an object of interest captured in a two-dimensional (2D) image are identified and marked for use in point matching to align multiple 2D images and generating a point cloud representative of the surface of the object in a photogrammetry process. The features which represent actual surface features of the object may have their local contrast enhanced to facilitate their identification. Reflections on the surface of the object are suppressed by correlating such reflections with, e.g., light sources, not associated with the object of interest so that during photogrammetry, such reflections can be ignored, resulting in the creation of a 3D model that is an accurate representation of the object of interest. Prior to local contrast enhancement and the suppression of reflection information, identification and isolation of the object of interest can be improved through one or more filtering processes.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
performing, via a contrast enhancement and spectral suppression component of a three-dimensional (3D) model generation system, local contrast enhancement and spectral suppression in relation to an object captured in each of a plurality of two dimensional (2D) images, the local contrast enhancement and spectral suppression identifying information representative of the object itself in each of the plurality of 2D images to ensure that signals representing the object are actual surfaces of the object, wherein the performance of local contrast enhancement comprises identifying and isolating one or more features of a surface of the object and increasing contrast in a region substantially about the one or more isolated features;
performing, via a photogrammetry system of the 3D model generation system, photogrammetry on the plurality of 2D images based upon the information representative of the object itself while ignoring information not representative of the object itself in each of the plurality of 2D images; and
generating, via a computer processor of the 3D model generation system, a computerized three dimensional (3D) model of the object based upon output information from the performance of photogrammetry.

2. The computer-implemented method of claim 1 , wherein the one or more features of the surface of the object comprise one or more variations in surface albedo of the surface of the object.

3. The computer-implemented method of claim 1 , further comprising tagging the one or more isolated features subsequent to the performance of the local contrast enhancement.

4. The computer-implemented method of claim 2 , wherein the performance of photogrammetry comprises aligning the plurality of 2D images using the one or more isolated features as matching points.

5. The computer-implemented method of claim 1 , wherein the performance of photogrammetry further comprises generating a point cloud based upon at least the one or more isolated features.

6. The computer-implemented method of claim 1 , wherein the one or more features are associated with high frequency signals.

7. The computer-implemented method of claim 6 , further comprising amplifying the high frequency signals associated with the one or more features such that the signal-to-noise ratio of the amplified high frequency signals exceeds a 1:1 ratio.

8. The computer-implemented method of claim 1 , wherein the performance of local contrast enhancement comprises increasing contrast in a region of pixels comprising a portion of a surface of the object so that one or more features of the surface of the object become detectable.

9. The computer-implemented method of claim 1 , wherein the performance of spectral suppression comprises correlating one or more pixel values representative of a surface of the object to a reflection of a light source separate from the object.

10. The computer-implemented method of claim 9 , wherein the performance of spectral suppression further comprises the ignoring of the information not representative of the object itself, the information not representative of the object itself comprising one or more frequencies associated with the reflection when creating a data set to be utilized as input during the performance of photogrammetry.

11. A computer-implemented method, comprising:
identifying, via an object identification and selection component of a three-dimensional (3D) model generation system, an object in a media content;
retrieving, via an object image filter of the 3D model generation system, one or more two dimensional (2D) images representative of the media content in which the object is present;
filtering, via a representative image filter of the 3D model generation system, the one or more 2D images to obtain a subset of the one or more 2D images best representing the object;
performing, via a contrast enhancement and spectral suppression component of the 3D model generation system, local contrast enhancement and spectral suppression on a surface of the object, the local contrast enhancement and spectral suppression identifying information representative of the object itself in each of the one or more 2D images to ensure that signals representing the object are actual surfaces of the object, wherein the performance of local contrast enhancement comprises identifying and isolating one or more features of a surface of the object and increasing contrast in a region substantially about the one or more isolated features;
generating, via the contrast enhancement and spectral suppression component of the 3D model generation system optimized versions of the subset of the one or more 2D images based upon the information representative of the object itself while ignoring information not representative of the object itself in each of the one or more 2D images obtained through the performance of the local contrast enhancement and spectral suppression; and
generating, via a computer processor of the 3D model generation system, a three dimensional (3D) model of the object based upon the optimized versions of the subset of the one or more 2D images.

12. The computer-implemented method of claim 11 , wherein the performance of local contrast enhancement further comprises tagging the one or more isolated features and including information regarding the one or more isolated features as input to the performance of photogrammetry.

13. The computer-implemented method of claim 12 , further comprising aligning each of the subset of the one or more 2D images using the one or more isolated features as matching points.

14. The computer-implemented method of claim 11 , further comprising injecting noise into each of the subset of the one or more 2D images such that an artificial texture is generated on the surface of the object facilitating the identification and isolation of the one or more features of the surface of the object.

15. The computer-implemented method of claim 11 , wherein the one or more isolated features are associated with high frequency signals.

16. The computer-implemented method of claim 15 , further comprising amplifying the high frequency signals associated with the one or more isolated features such that the signal-to-noise ratio of the amplified high frequency signals exceeds a 1:1 ratio.

17. The computer-implemented method of claim 11 , wherein the performance of the local contrast enhancement comprises increasing the localized contrast of red, green, blue (RGB) values of pixels representative of the one or more isolated features.

18. The computer-implemented method of claim 11 , wherein the performance of spectral suppression comprises correlating one or more pixel values representative of a surface of the object to a reflection of a light source separate from the object.

19. The computer-implemented method of claim 18 , wherein the performance of spectral suppression further comprises the ignoring of the information not representative of the object itself, the information not representative of the object itself comprising one or more frequencies associated with the reflection when creating a data set to be utilized as input during the performance of photogrammetry.

Assignees

• Disney Entpr Inc

Inventors

• Chapman Steven M
• Patel Mehul

Classifications

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T7/579

  from motion · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title