Building a three-dimensional composite scene

Patent metadata

Field	Value
Publication number	US-10102639-B2
Application number	US-201715689569-A
Country	US
Kind code	B2
Filing date	Aug 29, 2017
Priority date	Jun 29, 2011
Publication date	Oct 16, 2018
Grant date	Oct 16, 2018

Abstract

The capture and alignment of multiple 3D scenes is disclosed. Three dimensional capture device data from different locations is received thereby allowing for different perspectives of 3D scenes. An algorithm uses the data to determine potential alignments between different 3D scenes via coordinate transformations. Potential alignments are evaluated for quality and subsequently aligned subject to the existence of sufficiently high relative or absolute quality. A global alignment of all or most of the input 3D scenes into a single coordinate frame may be achieved. The presentation of areas around a particular hole or holes takes place thereby allowing the user to capture the requisite 3D scene containing areas within the hole or holes as well as part of the surrounding area using, for example, the 3D capture device. The new 3D captured scene is aligned with existing 3D scenes and/or 3D composite scenes.

First claim

What is claimed is:

1. A method, comprising:
obtaining, by a system comprising a processor, a plurality of sets of three-dimensional data respectively comprising three-dimensional feature information of points in a three-dimensional coordinate space, and wherein the plurality of sets are respectively associated with different capture positions relative to the three-dimensional coordinate space of a set of capture positions; and
determining, by the system prior to completion of capture of a last set of three-dimensional data of the plurality of sets of three-dimensional data, at least one first alignment between at least two sets from the plurality of sets of three-dimensional data based on selection of at least one spatial transformation to apply to at least one set of the plurality of sets of three-dimensional data according to an alignment criterion;
in response to completion of the capture of the last set of three-dimensional data, determining, by the system, a second alignment between at least one set from the plurality of sets of three-dimensional data and the last set of three-dimensional data based on selection of at least one additional spatial transformation to apply to at least one set of the plurality of sets of three-dimensional data and the last set of three-dimensional data according to the alignment criterion.

2. The method of claim 1 , further comprising determining, by the system, a spatial metric based on at least one alignment from the group consisting of the at least one first alignment and the second alignment.

3. The method of claim 2 , wherein the three-dimensional coordinate space represents at least a portion of a building, and the spatial metric is at least one of a volume of an area within the portion of the building, a square footage of an area within the portion of the building, a volume of an object within the portion of the building, or a surface area of an object within the portion of the building.

4. The method of claim 2 , wherein the three-dimensional coordinate space represents at least a portion of a building, and further comprising determining, by the system based on the spatial metric, a volume of paint required to paint at least one of an area or object within the portion of the building.

5. The method of claim 2 , wherein the three-dimensional coordinate space represents a body of a person, and the spatial metric is at least one size for at least one piece of clothing for at least one portion of the body.

6. The method of claim 1 , further comprising:
generating, by the system, a navigable three-dimensional environment representing at least the three-dimensional coordinate space based on at least one alignment from the group consisting of the at least one first alignment and the second alignment; and
inserting, by the system, a virtual object in to the navigable three-dimensional environment.

7. The method of claim 6 , wherein the three-dimensional coordinate space represents at least a portion of a building, and the virtual object is an interior decoration.

8. A system, comprising:
a memory to store executable instructions; and
a processor, coupled to the memory, that facilitates execution of the executable instructions to perform operations, comprising:
obtaining a plurality of sets of three-dimensional data respectively comprising three-dimensional feature information of points in a three-dimensional coordinate space, and wherein the plurality of sets are respectively associated with different capture positions relative to the three-dimensional coordinate space of a set of capture positions; and
determining, prior to completion of capture of a last set of three-dimensional data of the plurality of sets of three-dimensional data, at least one first alignment between at least two sets from the plurality of sets of three-dimensional data based on selection of at least one spatial transformation to apply to at least one set of the plurality of sets of three-dimensional data according to an alignment criterion;
in response to completion of the capture of the last set of three-dimensional data, determining a second alignment between at least one set from the plurality of sets of three-dimensional data and the last set of three-dimensional data based on selection of at least one additional spatial transformation to apply to at least one set of the plurality of sets of three-dimensional data and the last set of three-dimensional data according to the alignment criterion.

9. The system of claim 8 , the operations further comprising determining a spatial metric based on at least one alignment from the group consisting of the at least one first alignment and the second alignment.

10. The system of claim 9 , wherein the three-dimensional coordinate space represents at least a portion of a building, and the spatial metric is at least one of a volume of an area within the portion of the building, a square footage of an area within the portion of the building, a volume of an object within the portion of the building, or a surface area of an object within the portion of the building.

11. The system of claim 9 , wherein the three-dimensional coordinate space represents at least a portion of a building, and the operations further comprising determining, based on the spatial metric, a volume of paint required to paint at least one of an area or object within the portion of the building.

12. The system of claim 8 , the operations further comprising:
generating a navigable three-dimensional environment representing at least the three-dimensional coordinate space based on at least one alignment from the group consisting of the at least one first alignment and the second alignment; and
inserting a virtual object in to the navigable three-dimensional environment.

13. The system of claim 12 , wherein the three-dimensional coordinate space represents at least a portion of a building, and the virtual object is a remodeled interior.

14. The system of claim 8 , wherein the three-dimensional coordinate space represents a body of a person, and the operations further comprising generating, based on the determined alignment of the plurality of sets of three-dimensional data, a cast for a portion of the body.

15. A non-transitory computer-readable medium having instructions stored thereon that, in response to execution, cause a system including a processor to perform operations comprising:
obtaining a plurality of sets of three-dimensional data respectively comprising three-dimensional feature information of points in a three-dimensional coordinate space, and wherein the plurality of sets are respectively associated with different capture positions relative to the three-dimensional coordinate space of a set of capture positions; and
determining, prior to completion of capture of a last set of three-dimensional data of the plurality of sets of three-dimensional data, at least one first alignment between at least two sets from the plurality of sets of three-dimensional data based on selection of at least one spatial transformation to apply to at least one set of the plurality of sets of three-dimensional data according to an alignment criterion;
in response to completion of the capture of the last set of three-dimensional data, determining a second alignment between at least one set from the plurality of sets of three-dimensional data and the last set of three-dimensional data based on selection of at least one additional spatial transformation to apply to at least one set of the plurality of sets of three-dimensional data and the last set of three-dimensional data according to the alignment criterion.

Assignees

• Matterport Inc

Inventors

• Bell Matthew
• Gausebeck David

Classifications

• G06T7/30 Primary

  Determination of transform parameters for the alignment of images, i.e. image registration · CPC title

• G01S17/89 Primary

  for mapping or imaging · CPC title

• G06T15/10

  Geometric effects · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• H04N13/257

  Colour aspects · CPC title