Capturing and aligning three-dimensional scenes

Patent metadata

Field	Value
Publication number	US-9324190-B2
Application number	US-201313776688-A
Country	US
Kind code	B2
Filing date	Feb 25, 2013
Priority date	Feb 24, 2012
Publication date	Apr 26, 2016
Grant date	Apr 26, 2016

Abstract

Systems and methods for building a three-dimensional composite scene are disclosed. Certain embodiments of the systems and methods may include the use of a three-dimensional capture device that captures a plurality of three-dimensional images of an environment. Some embodiments may further include elements concerning aligning and/or mapping the captured images. Various embodiments may further include elements concerning reconstructing the environment from which the images were captured. The methods disclosed herein may be performed by a program embodied on a non-transitory computer-readable storage medium when executed the program is executed a processor.

First claim

What is claimed is:

1. A three-dimensional image reconstruction system, comprising:
a three-dimensional capture device comprising:
a housing;
a sensor module comprising one or more sensors respectively configured to capture three-dimensional image data captures of an object or environment at different capture positions relative to a three-dimensional coordinate space during a capture process, wherein the three-dimensional image data captures are respectively associated with distance information for features included therein; and
a communication module configured to communicatively couple the sensor module to a mobile processing device and send the three-dimensional image data captures, or sets of three-dimensional data respectively derived from the three-dimensional image data captures, to a mobile processing device as they are respectively captured or derived in association with the capture process, wherein the mobile processing device is configured to iteratively align the three-dimensional image data captures or the sets, based on the distance information, as the three-dimensional image data captures or the sets are respectively received to generate three-dimensional representations of the object or environment, and wherein the mobile processing device is configured to render the three-dimensional representations of the object or environment via a display screen of the mobile processing device.

2. The system of claim 1 , wherein in response to reception of the three-dimensional image data captures, the mobile processing device is configured to derive the distance information for the features respectively included therein, and align the features to one another in the three-dimensional coordinate space based on the distance information to generate the three-dimensional representations of the object or environment concurrently with the capture process.

3. The system of claim 1 , wherein the sets of the three-dimensional data respectively comprise the distance information derived for respective features included in the three-dimensional image data captures, and wherein in response to reception of the sets, the mobile processing device is configured to align the respective features included in the sets to one another relative to the three-dimensional coordinate space based on the distance information for the respective features to generate the three-dimensional representations of the object or environment and render three-dimensional representations of the object or environment concurrently with the capture process.

4. The system of claim 1 , wherein the communication module comprises a transmitter configured to wirelessly transmit the image data captures or the sets of the three-dimensional data to the mobile processing device.

5. The system of claim 1 , wherein the three-dimensional capture device further comprises an attachment component configured to physically couple the three-dimensional capture device to the mobile processing device.

6. The system of claim 5 , wherein the attachment component comprises one or more clips configured to attach or detach the three-dimensional capture device to or from the mobile processing device.

7. The system of claim 5 , wherein the attachment component comprises a molded docking station having a section that is an inverse shape of a section of the mobile processing device, wherein the respective inverse sections mate to securely couple the devices.

8. The system of claim 5 , wherein the attachment component comprises hardware configured to couple a data port of the mobile processing device to the three-dimensional capture device, wherein the hardware provides physical support to stabilize and fix the mobile processing device in an attached position relative to the three-dimensional capture device.

9. The system of claim 1 , further comprising a battery configured to power operation of the three-dimensional capture device.

10. The system of claim 1 , wherein the three-dimensional capture device, further comprises:
a memory to store computer executable instructions, and
a processor, coupled to the memory, that facilitates execution of the computer executable instructions to perform operations, comprising:
determining the distance information for the features respectively included in the three-dimensional image data captures; and
deriving the three-dimensional data sets from the three-dimensional image data captures.

11. The system of claim 1 , wherein the sensor module comprises two or more sensors respectively configured to capture different ones of the three-dimensional image data captures, and wherein the two or more sensors are respectively positioned at different angles.

12. The system of claim 1 , wherein the mobile processing device is at least one of: a mobile phone, a tablet computer, or a wearable computer.

13. The system of claim 1 , wherein the three-dimensional capture device and the mobile processing device are configured to be communicatively connected via a cable.

14. The three-dimensional image reconstruction system of claim 1 , wherein the one or more sensors are respectively configured to capture the three-dimensional image data captures of the object or environment at different orientations relative to the three-dimensional coordinate space during the capture process.

15. The three-dimensional image reconstruction system of claim 1 , wherein at least two of the three-dimensional image data captures or sets of three-dimensional data comprise no overlapping features.

16. The three-dimensional image reconstruction system of claim 1 , wherein the three-dimensional representations of the object or environment comprise a point cloud, a three-dimensional mesh, a volumetric rendering, a surfel cloud, or a cartoon rendering.

17. A three-dimensional capture device, comprising:
a housing comprising a handle;
one or more sensors respectively configured to capture three-dimensional image data captures of an object or environment at different capture positions relative to a three-dimensional coordinate space over a capture process, wherein the three-dimensional image data captures are respectively associated with distance information for features included therein; and
a communication module configured to send the three-dimensional image data captures, or sets of three-dimensional information respectively derived from the three-dimensional image data captures, to a mobile device as they are respectively captured or derived, wherein the mobile device is configured to iteratively align the three-dimensional image data captures or the sets, as the three-dimensional image data captures or the sets are respectively received, based on the distance information, to generate three-dimensional representations of the object or environment, and wherein the mobile device is configured to render the three-dimensional representations of the object or environment via a display screen of the mobile device.

18. The three-dimensional capture device of claim 17 , wherein in response to reception of the three-dimensional image data captures, the mobile device is configured to derive the distance information for the features respectively included therein, and align the features to one another in the three-dimensional coordinate space based on the distance information to generate the three-dimensional representations of the object or environment concurrently with the capture process.

19. The three-dimensional capture device of claim 17 , wherein the sets of the three-dimensional information respectively comprise the dis

Assignees

• Matterport Inc

Inventors

• Bell Matthew
• Gausebeck David
• Beebe Michael

Classifications

• H04N13/246

  Calibration of cameras · CPC title

• H04N13/156

  Mixing image signals · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• H04N2213/001

  Constructional or mechanical details · CPC title

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title