3D model generation from 2D images

Patent metadata

Field	Value
Publication number	US-10318102-B2
Application number	US-201615005927-A
Country	US
Kind code	B2
Filing date	Jan 25, 2016
Priority date	Jan 25, 2016
Publication date	Jun 11, 2019
Grant date	Jun 11, 2019

Abstract

Techniques and systems are described to generate a three-dimensional model from two-dimensional images. A plurality of inputs is received, formed through user interaction with a user interface. Each of the plurality of inputs define a respective user-specified point on the object in a respective one of the plurality of images. A plurality of estimated points on the object are generated automatically and without user intervention. Each of the plurality of estimated points corresponds to a respective user-specified point for other ones of the plurality of images. The plurality of estimated points is displayed for the other ones of the plurality of images in the user interface by a computing device. A mesh of the three-dimensional model of the object is generated by the computing device by mapping respective ones of the user-specified points to respective ones of the estimated points in the plurality of images.

First claim

What is claimed is:

1. In a digital medium environment to generate a three-dimensional model of an object from a plurality of images having different two-dimensional views of the object, a method implemented by a computing device, the method comprising:
receiving a plurality of inputs by the computing device, the plurality of inputs formed based on user interaction with a user interface, each of the plurality of inputs defining a respective user-specified point on the object in a respective image of the plurality of images;
generating, by the computing device, a plurality of estimated points on the object, each of the plurality of estimated points corresponding to a respective said user-specified point and defined in real time for multiple different images of the plurality of images than the respective image defining the respective user-specified point;
displaying, by the computing device, the plurality of estimated points in the user interface;
mapping, by the computing device, the user-specified points to the estimated points, respectively, in the plurality of images; and
generating, by the computing device, a mesh of the three-dimensional model of the object based on the mapping.

2. The method as described in claim 1 , wherein the generating of the plurality of points and the displaying are performed in real time as the plurality of inputs are received by the computing device.

3. The method as described in claim 1 , wherein at least one of the plurality of estimated points is moveable in relation to the object through user interaction with the user interface as correcting an error in an estimated location of the at least one of the plurality of estimated points.

4. The method as described in claim 1 , wherein a size of the object in at least one of the plurality of images is different than a size of the object in another one of the plurality of images.

5. The method as described in claim 1 , wherein the generating of the mesh includes normalizing locations of the user-specified points and locations of the estimated points.

6. The method as described in claim 1 , wherein the generating of the mesh includes mirroring at least one of the user-specified points or at least one of the estimated points to model a portion of the object that is not viewable in the plurality of images.

7. The method as described in claim 1 , wherein the generating of the plurality of estimated points includes selecting a template automatically and without user intervention.

8. The method as described in claim 1 , further comprising receiving at least one input based on user interaction with the user interface that selects a template from a plurality of templates that is to be used to generate a respective said estimated point from a respective said user-specified point.

9. The method as described in claim 1 , further comprising displaying the plurality of images simultaneously in the user interface by the computing device, each of the plurality of images including a different respective view of the object.

10. The method as described in claim 1 , further comprising receiving, by the computing device, one or more image identification inputs describing a respective view of the object that is captured by respective ones of the plurality of images and wherein the generating of the mesh is based at least in part a relationship of the respective views of the object.

11. The method as described in claim 1 , further comprising receiving one or more inputs that specify a location of the object in respective ones of the plurality of images that is used as part of the generating of the plurality of estimated points.

12. In a digital medium environment to generate a three-dimensional model of an object from a plurality of images having different two-dimensional views of the object, a system comprising:
a point manager module implemented at least partially in hardware of a processing system and a computer-readable storage medium to:
receive a plurality of inputs specifying a plurality of user-specified points based on user interaction with a user interface, each of the plurality of inputs defining a respective said user-specified point on the object in a respective image of the plurality of images;
generate a plurality of estimated points on the object, each of the plurality of estimated points corresponding to a respective said user-specified point and defined in real time for different multiple images of the plurality of images that the respective image defining the respective user-specified point;
display the plurality of points in the user interface; and

a mesh generation module implemented at least partially in hardware of the processing system and the computer-readable storage medium to map the plurality of user-specified points to the plurality of estimated points, respectively, and generate a mesh of the three-dimensional model of the object based on the mapping.

13. The system as described in claim 12 , wherein at least one of the plurality of estimated points is configured to be moveable in relation to the object based on user interaction with the user interface as correcting an error in an estimated location of the at least one of the plurality of estimated points.

14. The system as described in claim 12 , wherein the point manager module is configured to display the plurality of estimated points in real time as the plurality of inputs are received.

15. The system as described in claim 12 , wherein a size of the object in at least one of the plurality of images is different than a size of the object in another one of the plurality of images.

16. The system as described in claim 12 , wherein the mesh generation module is configured to normalize locations of the user-specified points with respect to locations of the estimated points.

17. The system as described in claim 12 , wherein the mesh generation module is configured to mirror at least one of the user-specified points or the estimated points to model a portion of the object that is not viewable in the plurality of images.

18. In a digital medium environment to generate a three-dimensional model of an object from a plurality of images having different two-dimensional views of the object, a method implemented by a computing device, the method comprising:
receiving an input, formed based on user interaction with a user interface of the computing device, defining a user-specified point on the object in a first image of the plurality of images;
displaying an estimated point in real time on the object by the computing device, the estimated point corresponding the user-specified point for a second image of the plurality of images;
receiving at least one input, by the computing device, based on a user interaction with the user interface, the at least one input defining movement of the estimated point on the object in the second image as correcting an error in an estimated location of the at least one of the plurality of estimated points; and
generating a mesh, by the computing device, of the three-dimensional model of the object, the generating including mapping the moved estimated point to the user-specified point.

19. The method as described in claim 18 , wherein the displaying is performed in real time as the input defining the user-specified point on the object is received.

20. The method as described in claim 18 , wherein a size of the object in at least one of the plurality of images is different than a size of the object in another one of the plurality of images.

Assignees

• Adobe Inc

Inventors

• Ito Daichi
• Mech Radomir
• Carr Nathan A
• Fukusato Tsukasa

Classifications

• G06F3/04845 Primary

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06T17/20 Primary

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2200/04

  involving 3D image data · CPC title