System and method of capturing and generating panoramic three-dimensional images

Patent metadata

Field	Value
Publication number	US-12140679-B2
Application number	US-202418620966-A
Country	US
Kind code	B2
Filing date	Mar 28, 2024
Priority date	Dec 30, 2019
Publication date	Nov 12, 2024
Grant date	Nov 12, 2024

Abstract

An apparatus comprising a housing, a mount configured to be coupled to a motor to horizontally move the apparatus, a wide-angle lens coupled to the housing, the wide-angle lens being positioned above the mount thereby being along an axis of rotation, the axis of rotation being the axis along which the apparatus rotates, an image capture device within the housing, the image capture device configured to receive two-dimensional images through the wide-angle lens of environment, and a LiDAR device within the housing, the LiDAR device configured to generate depth data based on the environment.

First claim

The invention claimed is:

1. An image capture device comprising:
a housing, the housing having a front side and a back side;
a first motor within the housing at a first position between the front side and the back side of the housing, the first motor configured to turn the image capture device at least 360 degrees around a first axis;
a wide-angle lens within the housing of the image capture device at a second position between the front side and the back side of the housing along the first axis, the second position being substantially at a no-parallax point, the wide-angle lens having a field of view away from the front side of the housing;
an image sensor within the housing of the image capture device and configured to generate image signals from light received by the wide-angle lens;
a LiDAR within the housing of the image capture device at a third position, the LiDAR configured to generate laser pulses and generate depth signals;
a second motor within the housing of the image capture device; and
a mirror coupled to the second motor, the second motor configured to rotate the mirror around a second axis, the mirror including an angled surface configured to receive the laser pulses from the LiDAR and direct the laser pulses around the second axis.

2. The image capture device of claim 1 , wherein the image sensor is configured to generate a first plurality of images when the image capture device is stationary and pointed in a first direction.

3. The image capture device of claim 2 , wherein the first motor is configured to turn the image capture device about the first axis after the first plurality of images are generated.

4. The image capture device of claim 3 , wherein the image sensor does not generate images while the first motor turns the image capture device and wherein the LiDAR generates depth signals based on the laser pulses while the first motor turns the image capture device.

5. The image capture device of claim 3 , wherein the image sensor is configured to generate a second plurality of images when the image capture device is stationary and pointed in a second direction and the first motor is configured to turn the image capture device 90 degrees about the first axis after the second plurality of images are generated.

6. The image capture device of claim 5 , wherein the image sensor is configured to generate a third plurality of images when the image capture device is stationary and pointed in a third direction and the first motor is configured to turn the image capture device 90 degrees about the first axis after the third plurality of images are generated.

7. The image capture device of claim 6 , wherein the image sensor is configured to generate a fourth plurality of images when the image capture device is stationary and pointed in a fourth direction and the first motor is configured to turn the image capture device 90 degrees about the first axis after the fourth plurality of images are generated.

8. The image capture device of claim 7 , further comprising a processor configured to blend frames of the first plurality of images before the image sensor generates the second plurality of images.

9. The image capture device of claim 7 , further comprising a remote digital device in communication with the image capture device and configured to generate a 3D visualization based on the first, second, third, and fourth plurality of images and the depth signals, the remote digital device being configured to generate the 3D visualization using no more images than the first, second, third, and fourth plurality of images.

10. The image capture device of claim 9 , wherein first, second, third, and fourth plurality of images are generated between turns that combined turns turning the image capture device 270 degrees around the first axis.

11. The image capture device of claim 4 , wherein a speed or rotation of the mirror around the second axis increases when the first motor turns the image capture device.

12. The image capture device of claim 1 , wherein the angled surface of the mirror is at least 45 degrees from the second axis.

13. The image capture device of claim 1 wherein the LiDAR emits the laser pulses in a direction that is opposite the front side of the housing of the image capture device.

14. A method comprising:
receiving light from a wide-angle lens of a device, the wide-angle lens being within a housing of the device, the light being received from a field of view of the wide-angle lens, the field of view extending away from a front side of the housing;
generating a first plurality of images by an image sensor of the device using the light from the wide-angle lens, the image sensor being within the housing of the device;
turning the device about a first axis by a first motor, turning the device about the first axis being at least 360 degrees about the first axis, the first motor being within the housing in a first position between the front side and a back side of the housing, the wide-angle lens being at a second position along the first axis, the second position being substantially at a no-parallax point;
rotating a mirror with an angled surface around a second axis by a second motor, the second motor being within the housing of the device;
generating laser pulses by a LiDAR, the LiDAR being within the housing of the device at a third position, the laser pulses being directed to the rotating mirror while the device turns about the first axis; and
generating depth signals by the LiDAR based on the laser pulses.

15. The method of claim 14 , wherein generating the first plurality of images by the image sensor occurs before the device about the first axis.

16. The method of claim 15 , wherein the image sensor does not generate images while the first motor turns the device and wherein the LiDAR generates the depth signals based on the laser pulses while the first motor turns the device.

17. The method of claim 16 , further comprising:
generating a second plurality of images by the image sensor when the device is stationary and pointed in a second direction; and
turning the device 90 degrees about the vertical axis by the first motor after the second plurality of images are generated.

18. The method of claim 17 , further comprising:
generating a third plurality of images by the image sensor when the device is stationary and pointed in a third direction; and
turning the device 90 degrees about the first axis by the first motor after the third plurality of images are generated.

19. The method of claim 18 , further comprising:
generating a fourth plurality of images by the image sensor when the device is stationary and pointed in a fourth direction.

20. The method of claim 19 , further comprising generating a 3D visualization using the first, second, third, and fourth plurality of images and based on the depth signals, the generating the 3D visualization not using any other images.

21. The method of claim 17 , further comprising blending frames of the first plurality of images before the image sensor generates the second plurality of images.

22. The method of claim 19 , wherein first, second, third, and fourth plurality of images are generated between turns that combined turns turning the device at least 360 degrees around the second axis.

23. The method of claim 14 , wherein a speed or rotation of the mirror around the second axis increases when the first motor turns the device.

Assignees

• Matterport Inc

Inventors

• Gausebeck David Alan
• Stromberg Kirk
• Marzano Louis D
• Proctor David
• Sakakibara Naoto
• Trieu Simeon
• Kane Kevin
• Wynn Simon

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G03B17/561

  Support related camera accessories · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G01S7/4817

  relating to scanning · CPC title

• G01S17/931 Primary

  of land vehicles · CPC title