Selecting two-dimensional imagery data for display within a three-dimensional model

Patent metadata

Field	Value
Publication number	US-10163261-B2
Application number	US-201414219906-A
Country	US
Kind code	B2
Filing date	Mar 19, 2014
Priority date	Mar 19, 2014
Publication date	Dec 25, 2018
Grant date	Dec 25, 2018

Abstract

Systems and methods for generating three-dimensional models with correlated three-dimensional and two dimensional imagery data are provided. In particular, imagery data can be captured in two dimensions and three dimensions. Imagery data can be transformed into models. Two-dimensional data and three-dimensional data can be correlated within models. Two-dimensional data can be selected for display within a three-dimensional model. Modifications can be made to the three-dimensional model and can be displayed within a three-dimensional model or within two-dimensional data. Models can transition between two dimensional imagery data and three dimensional imagery data.

First claim

What is claimed is:

1. A system, comprising:
a memory that stores computer executable components; and
a processor that executes at least the following computer executable components stored in the memory:
an imagery component configured to receive input indicating a viewpoint from which to view a three-dimensional model of an object or environment; and
a selection component configured to determine, based on the input, a position of the viewpoint relative to the three-dimensional model, and select a two-dimensional image from a plurality of two-dimensional images captured of the object or environment for displaying based in part on the two-dimensional image having a first capture position relative to the three-dimensional model that is associated with a first navigational distance that is shorter than second navigational distances associated with second capture positions of other two-dimensional images of the plurality of two-dimensional images, wherein the first navigational distance reflects one or more first constraints associated with navigation of the three-dimensional model from the position to the first capture position, and wherein the second navigational distances reflect one or more second constraints associated with navigation of the three-dimensional model from the position to the second capture positions.

2. The system of claim 1 , wherein the selection component is further configured to select the two-dimensional image based on the first navigational distance being less than a threshold distance.

3. The system of claim 1 , wherein the viewpoint comprises a new viewpoint and wherein the computer executable components further comprise:
a construction component configured to generate and display a representation of the three-dimensional model from a previous viewpoint of the three-dimensional model, wherein the imagery component is configured to receive the input in association with the display of the representation from the previous viewpoint; and
a transition component configured to facilitate transitioning between displaying the representation of the three-dimensional model to displaying the two-dimensional image based on the input and selection of the two-dimensional image.

4. The system of claim 3 , wherein the representation comprises a three-dimensional reconstruction of the three-dimensional model, and wherein the transition component is configured to replace the representation with the two-dimensional image or a portion of the two-dimensional image data based on the input and selection of the two-dimensional image.

5. The system of claim 3 , wherein the representation comprises another two-dimensional image of the plurality of two-dimensional images, wherein the input comprises selection of a point on the other two-dimensional image, and wherein the transition component is configured to replace the representation with the two-dimensional image or a portion of the two-dimensional image data based on the input and selection of the two-dimensional image.

6. The system of claim 1 , wherein the viewpoint comprises a new viewpoint and wherein the input comprises a navigational command to move a virtual camera relative to the three-dimensional model from a previous viewpoint of the three-dimensional model to the new viewpoint, and wherein the selection component is further configured to determine the position of the new viewpoint relative to the three-dimensional model based on the command, the previous viewpoint, and a navigation mode employed to view the three-dimensional model.

7. The system of claim 1 , wherein the input comprises selection of a point on or within another two-dimensional image of the plurality of two-dimensional images, and wherein the selection component is further configured to determine the first position of the viewpoint relative to the three-dimensional model based on the point.

8. The system of claim 1 , wherein the selection component is further configured to determine a field of view for viewing the three dimensional model from the viewpoint, and select the two-dimensional image based on a difference between the field of view and a capture field of view associated with the two-dimensional image being less than differences between the field of view associated with the viewpoint and capture fields of view associated with other two-dimensional images of the plurality of two-dimensional images.

9. The system of claim 1 , wherein the position is associated with a room of the three-dimensional model and wherein the selection component is further configured to select the two-dimensional image based on the capture position of the two-dimensional image being associated with the room.

10. The system of claim 1 , wherein the one or more first or second constraints comprise a barrier constraint that restricts passage through solid objects in association with the navigation of the three-dimensional model.

11. The system of claim 1 , wherein the one or more first or second constraints comprise a room constraint that restricts the navigation of the three-dimensional model to be within the same room.

12. The system of claim 1 , wherein the selection component is further configured to:
determine an area of the three-dimensional model for viewing based on the viewpoint;
determine image data of the three-dimensional model corresponding to the area; and
select the two-dimensional image based on a degree of correspondence between the image data and the two-dimensional image.

13. The system of claim 1 , wherein the viewpoint comprises a new viewpoint and wherein the selection component is further configured to select the two-dimensional image based on a previous viewpoint from which the three-dimensional model is viewed at a time when the input is received.

14. The system of claim 13 , wherein the input comprises a navigational command that directs movement of a virtual camera away from the current viewpoint in association with navigation of the three-dimensional model in a three-dimensional walkthrough mode.

15. The system of claim 1 , wherein the viewpoint comprises a new viewpoint and wherein the selection component is configured to determine the position of the new viewpoint by casting a ray from a current viewpoint into the three-dimensional model in a direction determined based on the input, wherein the three-dimensional model is viewed from the current viewpoint at a time when the input is received.

16. A method, comprising
using a processor to execute the following computer executable instructions stored in a memory to perform the following acts:
receiving input indicating a viewpoint from which to view a three-dimensional model of an object or environment;
determining, based on the input, a position of the viewpoint relative to the three-dimensional model; and
selecting a two-dimensional image from a set of two-dimensional images captured of the object or environment for displaying based in part on the two-dimensional image having a first capture position relative to the three-dimensional model that is associated with a first navigational distance that is shorter than second navigational distances associated with second capture positions of other two-dimensional images of the set of two-dimensional images, wherein the first navigational distance reflects one or more first constraints associated with navigation of the three-dimensional model from the position to the first capture position, and wherein the second navigational distances reflect one or more second constraints associated with navigation of the three-dimensional model from the position to the second capture posi

Assignees

• Matterport Inc

Inventors

• Bell Matthew Tschudy
• Gausebeck David Alan
• Coombe Gregory William
• Ford Daniel
• Brown William John

Classifications

• G06T19/003 Primary

  Navigation within 3D models or images · CPC title

• G06T15/20

  Perspective computation · CPC title

• G06T13/80

  Two-dimensional [2D] animation, e.g. using sprites · CPC title