Unmanned aerial image capture platform

Patent metadata

Field	Value
Publication number	US-12416918-B2
Application number	US-202318463149-A
Country	US
Kind code	B2
Filing date	Sep 7, 2023
Priority date	Aug 12, 2016
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

Methods and systems are disclosed for an unmanned aerial vehicle (UAV) configured to autonomously navigate a physical environment while capturing images of the physical environment. In some embodiments, the motion of the UAV and a subject in the physical environment may be estimated based in part on images of the physical environment captured by the UAV. In response to estimating the motions, image capture by the UAV may be dynamically adjusted to satisfy a specified criterion related to a quality of the image capture.

First claim

What is claimed is:

1. A method comprising:
capturing, by multiple image sensors of a vehicle, images of a physical environment surrounding the vehicle;
processing the images of the physical environment surrounding the vehicle to identify images with overlapping fields of view;
processing the images with overlapping fields of view to identify dense correspondences between the images, wherein the dense correspondences indicate matches between pixels in the images with overlapping fields of view;
estimating distances to the pixels in the images with overlapping fields of view using the identified dense correspondences; and
generating and continually updating a three-dimensional (3D) map of the physical environment based on the estimated distances.

2. The method of claim 1 , further comprising:
causing the vehicle to autonomously navigate through the physical environment based, at least in part, on the 3D map of the physical environment.

3. The method of claim 1 , wherein the images of the physical environment surrounding the vehicle are captured during sequential time steps.

4. The method of claim 1 , wherein the images of the physical environment surrounding the vehicle are stereoscopic images captured during a same timestep and the multiple image sensors are configured with pre-defined spatial offsets.

5. The method of claim 1 , further comprising:
estimating a relative position of the vehicle within the physical environment while simultaneously continually updating the 3D map.

6. The method of claim 1 , further comprising:
estimating an orientation of the vehicle within the physical environment while simultaneously continually updating the 3D map.

7. The method of claim 1 , further comprising:
estimating a relative position of a subject within the physical environment while simultaneously continually updating the 3D map.

8. The method of claim 1 , further comprising:
estimating an orientation of a subject within the physical environment while simultaneously continually updating the 3D map.

9. A vehicle comprising:
multiple image sensors configured to capture images of a physical environment surrounding the vehicle;
a control system configured to continually:
process the images of the physical environment surrounding the vehicle to identify images with overlapping fields of view;
process the images with overlapping fields of view to identify dense correspondences between the images, wherein the dense correspondences indicate matches between pixels in the images with overlapping fields of view;
estimate distances to pixels in the images with the overlapping fields of view using the identified dense correspondences; and
update a three-dimensional (3D) map of the physical environment based on the estimated distances.

10. The vehicle of claim 9 , wherein the control system is further configured to:
generate the 3D map of the physical environment based on the estimated distances.

11. The vehicle of claim 9 , wherein the control system is further configured to:
cause the vehicle to autonomously navigate through the physical environment based, at least in part, on the 3D map of the physical environment.

12. The vehicle of claim 9 , wherein the images of the physical environment surrounding the vehicle are captured during sequential time steps.

13. The vehicle of claim 9 , wherein the images of the physical environment surrounding the vehicle are stereoscopic images captured during a same timestep and the multiple image sensors are configured with pre-defined spatial offsets.

14. The vehicle of claim 9 , wherein the control system is further configured to:
estimate a relative position of the vehicle and/or a subject within the physical environment while simultaneously continually updating the 3D map.

15. The vehicle of claim 9 , wherein the control system is further configured to:
estimating an orientation of the vehicle and/or a subject within the physical environment while simultaneously continually updating the 3D map.

16. The vehicle of claim 9 , wherein the multiple image sensors are arranged to provide a 360 degree view around the vehicle, and wherein the multiple image sensors are arranged such that at least two image sensors are provided with overlapping fields of view.

17. An apparatus, comprising:
one or more memory units storing instructions that, when executed by one or more processors of a vehicle, cause the one or more processors to:
process images of a physical environment surrounding the vehicle to identify images with overlapping fields of view;
process the images with overlapping fields of view to identify dense correspondences between the images, wherein the dense correspondences indicate matches between pixels in the images with overlapping fields of view;
estimate distances to pixels in the images with the overlapping fields of view using the identified dense correspondences; and
generate and continually update a three-dimensional (3D) map of the physical environment based on the estimated distances.

18. The apparatus of claim 17 , wherein the instructions, when executed by the one or more processors of the vehicle, further cause the one or more processors to:
cause the vehicle to autonomously navigate through the physical environment based, at least in part, on the 3D map of the physical environment.

19. The apparatus of claim 17 , wherein the instructions, when executed by the one or more processors of the vehicle, further cause the one or more processors to:
estimate a relative position of the vehicle and/or a subject within the physical environment while simultaneously continually updating the 3D map.

20. The apparatus of claim 17 , wherein the instructions, when executed by the one or more processors of the vehicle, further cause the one or more processors to:
estimate an orientation of the vehicle and/or a subject within the physical environment while simultaneously continually updating the 3D map.

Assignees

• Skydio Inc

Inventors

• Martirosyan Hayk
• Bry Adam
• Donahoe Matthew
• Bachrach Abraham
• Sadowski Justin Michael

Classifications

• G05D1/689

  Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672) · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• G06T2207/10021

  Stereoscopic video; Stereoscopic image sequence · CPC title

• G06T7/292

  Multi-camera tracking · CPC title