3D Model Generation from 2D Images

Patent metadata

Field	Value
Publication number	US-2017212661-A1
Application number	US-201615005927-A
Country	US
Kind code	A1
Filing date	Jan 25, 2016
Priority date	Jan 25, 2016
Publication date	Jul 27, 2017
Grant date	—

Abstract

Techniques and systems are described to generate a three-dimensional model from two-dimensional images. A plurality of inputs is received, formed through user interaction with a user interface. Each of the plurality of inputs define a respective user-specified point on the object in a respective one of the plurality of images. A plurality of estimated points on the object are generated automatically and without user intervention. Each of the plurality of estimated points corresponds to a respective user-specified point for other ones of the plurality of images. The plurality of estimated points is displayed for the other ones of the plurality of images in the user interface by a computing device. A mesh of the three-dimensional model of the object is generated by the computing device by mapping respective ones of the user-specified points to respective ones of the estimated points in the plurality of images.

First claim

What is claimed is:

1 . In a digital medium environment to generate a three-dimensional model of an object from a plurality of images having different two-dimensional views of the object, a method implemented by a computing device, the method comprising:
receiving a plurality of inputs by the computing device, the plurality of inputs formed based on user interaction with a user interface, each of the plurality of inputs defining a respective user-specified point on the object in a respective image of the plurality of images; generating, by the computing device, a plurality of estimated points on the object, each of the plurality of estimated points corresponding to a respective said user-specified point and defined for a different image of the plurality of images than the respective image defining the respective user-specified point; displaying, by the computing device, the plurality of estimated points in the user interface; and generating, by the computing device, a mesh of the three-dimensional model of the object, the generating including mapping respective ones of the user-specified points to respective ones of the estimated points in the plurality of images.

2 . The method as described in claim 1 , wherein the generating of the plurality of points and the displaying are performed in real time as the plurality of inputs are received by the computing device.

3 . The method as described in claim 1 , wherein at least one of plurality of estimated points is moveable in relation to the object through user interaction with the user interface.

4 . The method as described in claim 1 , wherein a size of the object in at least one of the plurality of images is different than a size of the object in another one of the plurality of images.

5 . The method as described in claim 1 , wherein the generating of the mesh includes normalizing locations of the user-specified points and the locations of the estimated points.

6 . The method as described in claim 1 , wherein the generating of the mesh includes mirroring at least one of the user-specified points or at least one of the estimated points to model a portion of the object that is not viewable in the plurality of images.

7 . The method as described in claim 1 , wherein the generating of the plurality of estimated points includes selecting a template automatically and without user intervention.

8 . The method as described in claim 1 , further comprising receiving at least one input based on user interaction with the user interface that selects a template from a plurality of templates that is to be used to generate a respective said estimated point from a respective said user-specified point.

9 . The method as described in claim 1 , further comprising displaying the plurality of images simultaneously in the user interface by the computing device, each of the plurality of images including a different respective view of the object.

10 . The method as described in claim 1 , further comprising receiving, by the computing device, one or more image identification inputs describing a respective view of the object that is captured by respective ones of the plurality of images and wherein the generating of the mesh is based at least in part a relationship of the respective views of the object.

11 . The method as described in claim 1 , further comprising receiving one or more inputs that specify a location of the object in respective ones of the plurality of images that is used as part of the generating of the plurality of estimated points.

12 . In a digital medium environment to generate a three-dimensional model of an object from a plurality of images having different two-dimensional views of the object, a system comprising:
a point manager module implemented at least partially in hardware to:
receive a plurality of inputs formed based on user interaction with a user interface, each of the plurality of inputs defining a respective user-specified point on the object in a respective image of the plurality of images;
generate a plurality of estimated points on the object, each of the plurality of estimated points corresponding to a respective said user-specified point and defined for a different image of the plurality of images that the respective image defining the respective user-specified point; and
display the plurality of estimated points in the user interface; and
a mesh generation module implemented at least partially in hardware to generate a mesh of the three-dimensional model of the object, the mesh generated as a mapping of respective ones of the user-specified points to respective ones of the estimated points in the plurality of images.

13 . The system as described in claim 12 , wherein at least one of plurality of estimated points is configured to be moveable in relation to the object based on user interaction with the user interface.

14 . The system as described in claim 12 , wherein the point manager module is configured to display the plurality of estimated points in real time as the plurality of inputs are received.

15 . The system as described in claim 12 , wherein a size of the object in at least one of the plurality of images is different than a size of the object in another one of the plurality of images.

16 . The system as described in claim 12 , wherein the mesh generation module is configured to normalize locations of the user-specified points with respect to the locations of the estimated points.

17 . The system as described in claim 12 , wherein the mesh generation module is configured to mirror at least one of the user-specified points or the estimated points to model a portion of the object that is not viewable in the plurality of images.

18 . In a digital medium environment to generate a three-dimensional model of an object from a plurality of images having different two-dimensional views of the object, a method implemented by a computing device, the method comprising:
receiving an input, formed based on user interaction with a user interface of the computing device, defining a user-specified point on the object in a first image of the plurality of images; displaying an estimated point on the object by the computing device, the estimated point corresponding the user-specified point for a second image of the plurality of images; receiving at least one input, by the computing device, based on a user interaction with the user interface, the at least one input defining movement of the estimated point on the object in the second image; and generating a mesh, by the computing device, of the three-dimensional model of the object, the generating including mapping respective ones of a plurality of said user-specified points to respective ones of a plurality of said estimated points.

19 . The method as described in claim 18 , wherein the displaying is performed in real time as the input defining the user-specified point on the object is received.

20 . The method as described in claim 18 , wherein a size of the object in at least one of the plurality of images is different than a size of the object in another one of the plurality of images.

Assignees

• Adobe Systems Inc

Inventors

• Ito Daichi
• Mech Radomir
• Carr Nathan A
• Fukusato Tsukasa

Classifications

• G06F3/04845 Primary

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06T15/20

  Perspective computation · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T17/20 Primary

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T2200/04

  involving 3D image data · CPC title