Capturing and aligning three-dimensional scenes

Patent metadata

Field	Value
Publication number	US-10482679-B2
Application number	US-201314070426-A
Country	US
Kind code	B2
Filing date	Nov 1, 2013
Priority date	Feb 24, 2012
Publication date	Nov 19, 2019
Grant date	Nov 19, 2019

Abstract

Systems and methods for building a three-dimensional composite scene are disclosed. Certain embodiments of the systems and methods may include the use of a three-dimensional capture device that captures a plurality of three-dimensional images of an environment. Some embodiments may further include elements concerning aligning and/or mapping the captured images. Various embodiments may further include elements concerning reconstructing the environment from which the images were captured. The methods disclosed herein may be performed by a program embodied on a non-transitory computer-readable storage medium when executed the program is executed a processor.

First claim

What is claimed is:

1. A three-dimensional capture system, comprising:
a rotatable stage configured to rotate about an axis;
one or more three-dimensional capture devices mounted on the rotatable stage and configured to capture, during a first rotation of the rotatable stage at a first location in an environment, first images of the environment; and
a communication device configured to send the first images to a computation device as they are respectively captured,
wherein the computation device is configured to align the first images to one another using a pair-wise alignment process based on correspondences between position information determined for features included in respective first images of the first images, resulting in a first set of aligned image data captured at the first location, wherein the position information is determined based on depth information respectively associated with the features, and
wherein the computation device is further configured to generate a three-dimensional reconstruction of the environment by aligning the first set of aligned image data with a second set of aligned image data using a global alignment process, wherein the second set of aligned image data comprises second images that were captured during a second rotation of the rotatable stage at a second location in the environment and aligned to one another using the pair-wise alignment process, and wherein the global alignment process comprises aligning the first set of aligned image data with the second set of aligned imaged data in a common three-dimensional coordinate space.

2. The three-dimensional capture system of claim 1 , further comprising a fixed frame provided on the rotatable stage, wherein the computation device and the one or more three-dimensional captured devices are configured to mount on the rotatable staged via the fixed frame.

3. The three-dimensional capture system of claim 1 , wherein the communication device comprises a docking station that receives the computation device and provides for at least one of, physical or electrical coupling between the computation device and the one or more three-dimensional capture devices and the rotatable stage.

4. The three-dimensional capture system of claim 3 , wherein the computation device is at least one of: a mobile phone, a tablet computer, a laptop computer, or a wearable computer.

5. The three-dimensional capture system of claim 1 , wherein the rotatable stage comprises a motor and wherein the motor is configured to control rotation of the rotatable stage in response to reception of control signals from the computation device via a wired or wireless connection between the computation device and the motor.

6. The three-dimensional capture system of claim 1 , wherein the one or more three-dimensional capture devices comprise three-dimensional sensors configured to capture the first images and the second images, and wherein the three-dimensional sensors are respectively positioned at different angles relative to a vertical plane such that respective fields of view of adjacent sensors overlap.

7. The three-dimensional capture system of claim 1 , wherein the computation device is further configured to render the three-dimensional reconstruction of the environment via a display.

8. The three-dimensional capture system of claim 1 , wherein the computation device is configured to align the first images to one another as they are received during the first rotation of the rotatable stage.

9. A three-dimensional capture system, comprising:
a rotatable stage that has mounted thereon, one or more three-dimensional capture devices configured to capture first images at various rotation points during a first rotation of the rotatable stage at a first location in an environment;
a memory that stores computer executable instructions; and a processor, coupled to the memory, that facilitates execution of the executable instructions to perform operations, comprising:
aligning the first images to one another using a pair-wise alignment process based on correspondences between position information determined for features included in respective first images of the first images, thereby generating a first set of aligned image data captured at the first location, wherein the position information is determined based on depth information respectively associated with the features; and
sending the first set of aligned image data to a computation device, wherein based on the sending, the computation device is configured to perform a global alignment process that comprises aligning the first set of aligned image data with a second set of aligned image data in a common three-dimensional coordinate space to generate a three-dimensional reconstruction of the environment, wherein the second set of aligned image data comprises second images that were captured during a second rotation of the rotatable stage at a second location in the environment and aligned to one another using the pair-wise alignment process.

10. The three-dimensional capture system of claim 9 , wherein the operations further comprise:
receiving rotation information identifying respective rotation points of the rotatable stage associated with capture of the respective first images, and wherein the performing the pair-wise alignment process comprises aligning the first images to one another during the first rotation of the rotatable stage based on the rotation information.

11. The three-dimensional capture system of claim 9 , further comprising a motor configured to drive rotation of the rotatable stage, and wherein the operations further comprise:
receiving, from a remote device, control signals requesting rotation of the rotatable stage; and
controlling the first and the second rotation of the rotatable stage via the motor based on the control signals.

12. The three-dimensional capture system of claim 9 , further comprising a robotic mount configured to move positions and a motor configured to drive movement of the robotic mount, wherein the rotatable stage is mounted on the robotic mount and wherein the operations further comprise:
receiving a control signal requesting movement of the robotic mount to reposition the one or more three-dimensional capture devices relative to an area of the environment in response to a determination, based on the first aligned image data or the three-dimensional reconstruction, that the area of the environment is associated with low-quality image data; and
controlling movement of the robotic mount via the motor based on the control signal.

13. The three-dimensional capture system of claim 9 , further comprising a robotic mount configured to move positions and a motor configured to drive movement of the robotic mount, wherein the rotatable stage is mounted on the robotic mount and wherein the operations further comprise:
receiving, from the computation device, information identifying an area of the environment associated with no or low-quality image data as determined based on analysis of the three-dimensional reconstruction by the computation device;
controlling movement of the robotic mount to reposition the one or more three-dimensional capture devices relative to the area of the environment; and
capturing one or more new images of the area via the one or more three-dimensional capture devices.

14. The three-dimensional capture system of claim 9 , wherein the one or more three-dimensional capture devices comprise at least two three-dimensional capture devices and wherein the at least two three-dimensional capture devices are calibrated such that image data captured by the at least two three-dimensional c

Assignees

• Matterport Inc

Inventors

• Bell Matthew
• Gausebeck David
• Beebe Michael

Classifications

• H04N13/10

  Processing, recording or transmission of stereoscopic or multi-view image signals · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title

• H04N2213/001

  Constructional or mechanical details · CPC title

• H04N13/246

  Calibration of cameras · CPC title