Generating a three-dimensional model from a scanned object

Patent metadata

Field	Value
Publication number	US-10679408-B2
Application number	US-201715423348-A
Country	US
Kind code	B2
Filing date	Feb 2, 2017
Priority date	Feb 2, 2017
Publication date	Jun 9, 2020
Grant date	Jun 9, 2020

Abstract

The present disclosure is directed toward systems and methods that facilitate scanning an object (e.g., a three-dimensional object) having custom mesh lines thereon and generating a three-dimensional mesh of the object. For example, a three-dimensional modeling system receives a scan of the object including depth information and a two-dimensional texture map of the object. The three-dimensional modeling system further generates an edge map for the two-dimensional texture map and modifies the edge map to generate a two-dimensional mesh including edges, vertices, and faces that correspond to the custom mesh lines on the object. Based on the two-dimensional mesh and the depth information from the scan, the three-dimensional modeling system generates a three-dimensional model of the object.

First claim

What is claimed is:

1. In a digital medium environment for digitizing physical objects, a method for generating a three-dimensional model of a scanned object, the method comprising:
receiving a scan of a three-dimensional object comprising custom mesh lines placed over a surface of the three-dimensional object prior to capturing the scan, the scan comprising one or more images of the three-dimensional object and depth information for the three-dimensional object captured during the scan;
generating an edge map from the scan by detecting edges within the scan corresponding to the custom mesh lines placed over the surface of the three-dimensional object prior to capturing the scan; and
a step for constructing a three-dimensional mesh of the three-dimensional object using the edge map and the depth information, the three-dimensional mesh comprising polygons corresponding to the custom mesh lines.

2. The method of claim 1 , further comprising generating a two-dimensional texture map from the scan by aligning the one or more images of the scan.

3. The method of claim 2 , further comprising thinning edges of the edge map to have a uniform pixel thickness by iteratively deleting opposing boundary pixels of the edges of the edge map until the edges comprise thinned edges having a single-pixel thickness while preserving junctions of the edge map.

4. The method of claim 2 , further comprising combining any two or more junctions having locations within a threshold distance of each other into a vertex.

5. The method of claim 2 , wherein generating the two-dimensional texture map from the scan further comprises converting the depth information and the one or more images from a three-dimensional domain to a two-dimensional domain.

6. The method of claim 2 , wherein generating the edge map comprises generating a binary representation of the two-dimensional texture map, the binary representation comprising a first pixel value for any pixels determined to correspond to the custom mesh lines and a second pixel value for other pixels of the two-dimensional texture map.

7. The method of claim 2 , further comprising:
receiving an indication of a color of the custom mesh lines; and
wherein detecting edges within the two-dimensional texture map comprises identifying pixels of the two-dimensional texture map having pixel values corresponding to the color of the custom mesh lines.

8. The method of claim 1 , wherein the custom mesh lines comprise hand-drawn lines drawn over the surface of the three-dimensional object.

9. A non-transitory computer readable storage medium comprising instructions that, when executed by at least one processor, cause a computing device to:
receive a scan of a three-dimensional object comprising a hand-drawn custom mesh placed over a surface of the three-dimensional object prior to capturing the scan, the scan comprising one or more images of the three-dimensional object and depth information for the three-dimensional object captured during the scan, and the hand-drawn custom mesh comprising a plurality of polygons formed by intersecting hand-drawn custom mesh lines;
generate, based on the scan of the three-dimensional object, a two-dimensional texture map comprising a representation of the three-dimensional object and the hand-drawn custom mesh thereon;
generate, an edge map from the two-dimensional texture map by detecting edges within the two-dimensional texture map corresponding to the hand-drawn custom mesh lines placed over the surface of the three-dimensional object prior to capturing the scan such that the edge map comprises a binary representation of the two-dimensional texture map including a first type of pixel indicating the hand-drawn custom mesh lines and a second type of pixel in all other portions of the edge map;
construct a two-dimensional mesh of the three-dimensional object by refining the edge map by thinning the detected edges corresponding to the hand-drawn custom mesh lines and replacing curved or bending detected edges, corresponding to the hand-drawn custom mesh lines, between identified junctions of detected edges with straight lines, the two-dimensional mesh comprising two-dimensional edges, vertices, and faces that correspond to the hand-drawn custom mesh lines from the edge map; and
construct a three-dimensional mesh of the three-dimensional object based on the two-dimensional mesh and the depth information, the three-dimensional mesh comprising three-dimensional edges, vertices, and faces corresponding to the plurality of polygons formed by the intersecting hand-drawn custom mesh lines of the hand-drawn custom mesh.

10. The non-transitory computer readable storage medium of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the two-dimensional texture map, wherein generating the two-dimensional texture map comprises combining the one or more images of the three-dimensional object into a composite two-dimensional image based on the depth information for the three-dimensional object.

11. The non-transitory computer readable storage medium of claim 9 , wherein the instructions, when executed by the at least one processor, cause the computing device to construct the two-dimensional mesh by generating a thinned edge map having edges of a uniform thickness.

12. The non-transitory computer readable storage medium of claim 11 , wherein the edges of the thinned edge map are a single pixel thick.

13. The non-transitory computer readable storage medium of claim 11 , wherein the instructions, when executed by the at least one processor, cause the computing device to construct the two-dimensional mesh by performing steps comprising:
identifying junctions of the thinned edge map; and
generating the two-dimensional vertices for the two-dimensional mesh by combining junctions at locations within the thinned edge map within a threshold distance of each other.

14. The non-transitory computer readable storage medium of claim 11 , wherein the instructions, when executed by the at least one processor, cause the computing device to construct the two-dimensional texture map by performing steps comprising:
identifying junctions of the thinned edge map;
identifying two junctions connected by an edge of the thinned edge map at locations within the thinned edge map within a threshold distance of each other;
determining that combining the two junctions into a vertex would form a polygon having a predetermined number of sides; and
in response to identifying that the two junctions are at locations within the threshold distance of each other and that combining the two junctions into a vertex would form a polygon having a predetermined number of sides, removing the edge connecting the two junctions and combining the two junctions into a single vertex.

15. The non-transitory computer readable storage medium of claim 11 , wherein the instructions, when executed by the at least one processor, cause the computing device to construct the two-dimensional mesh by performing steps comprising:
identifying junctions of the thinned edge map; and
generating the two-dimensional edges, vertices, and faces that correspond to the custom mesh lines from the edge map by replacing edges connecting the identified junctions with straight edges that connect the identified junctions of the thinned edge map.

16. The non-transitory computer readable storage medium of claim 9 , wherein the instructions, when executed by the at least one processor, cause the computing device to construct the two-dimensional mesh by removing any det

Assignees

• Adobe Inc

Inventors

• Ceylan Duygu
• Ito Daichi

Classifications

• G06T2207/20212

  Image combination · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T7/13

  Edge detection · CPC title

• G06T2207/10008

  from scanner, fax or copier · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title