Relative Image Capture Device Orientation Calibration

Patent metadata

Field	Value
Publication number	US-2025103046-A1
Application number	US-202418908214-A
Country	US
Kind code	A1
Filing date	Oct 7, 2024
Priority date	Jan 23, 2018
Publication date	Mar 27, 2025
Grant date	—

Abstract

Controlling an unmanned aerial vehicle may include obtaining a first image from a fixed orientation image capture device of the unmanned aerial vehicle, obtaining a second image from an adjustable orientation image capture device of the unmanned aerial vehicle, obtaining feature correlation data based on the first image and the second image, obtaining relative image capture device orientation calibration data based on the feature correlation data, the relative image capture device orientation calibration data indicating an orientation of the adjustable orientation image capture device relative to the fixed orientation image capture device, obtaining relative object orientation data based on the relative image capture device orientation calibration data, the relative object orientation data representing a three-dimensional orientation of an external object relative to the adjustable orientation image capture device, and controlling a trajectory of the unmanned aerial vehicle in response to the relative object orientation data.

First claim

What is claimed is:

1 . An unmanned aerial vehicle, comprising:
a fixed orientation image capture device; an adjustable orientation image capture device; a processor configured to execute instruction stored on a non-transitory computer readable medium to control the unmanned aerial vehicle to traverse a portion of an operational environment of the unmanned aerial vehicle using relative image capture device orientation calibration by:
obtaining a first image from the fixed orientation image capture device;
obtaining a second image from the adjustable orientation image capture device;
obtaining feature correlation data based on the first image and the second image;
obtaining relative image capture device orientation calibration data based on the feature correlation data, the relative image capture device orientation calibration data indicating an orientation of the adjustable orientation image capture device relative to the fixed orientation image capture device;
obtaining relative object orientation data based on the relative image capture device orientation calibration data, the relative object orientation data representing a three-dimensional orientation of an external object relative to the adjustable orientation image capture device; and
a trajectory controller configured to control a trajectory of the unmanned aerial vehicle in response to the relative object orientation data.

2 . The unmanned aerial vehicle of claim 1 , wherein:
a portion of a field of view of the fixed orientation image capture device corresponding to capturing the first image overlaps a portion of a field of view of the adjustable orientation image capture device corresponding to capturing the second image; obtaining feature correlation data includes obtaining spatial feature correlation data, wherein obtaining the spatial feature correlation data includes obtaining the spatial feature correlation data such that the spatial feature correlation data indicates a correlation between a feature from the first image and a corresponding feature from the second image; and obtaining the relative image capture device orientation calibration data includes using five-point relative positioning based on the spatial feature correlation data.

3 . The unmanned aerial vehicle of claim 1 , wherein:
a field of view of the fixed orientation image capture device corresponding to capturing the first image is non-overlapping with a field of view of the adjustable orientation image capture device corresponding to capturing the second image; and obtaining feature correlation data includes obtaining temporal feature correlation data.

4 . The unmanned aerial vehicle of claim 3 , wherein obtaining the temporal feature correlation data includes:
obtaining a third image from the fixed orientation image capture device, the third image sequentially subsequent to the first image; obtaining first temporal feature correlation data indicating a correlation between a feature from the first image and a corresponding feature from the third image; obtaining a fourth image from the adjustable orientation image capture device, the fourth image sequentially subsequent to the second image; and obtaining second temporal feature correlation data indicating a correlation between a feature from the second image and a corresponding feature from the fourth image.

5 . The unmanned aerial vehicle of claim 4 , wherein:
obtaining the temporal feature correlation data includes:
obtaining first velocity data based on the first temporal feature correlation data, the first velocity indicating a velocity of the fixed orientation image capture device; and
obtaining second velocity data based on the second temporal feature correlation data, the second velocity indicating a velocity of the adjustable orientation image capture device; and
obtaining the relative image capture device orientation calibration data includes obtaining the relative image capture device orientation calibration data based on the first velocity data and the second velocity data.

6 . The unmanned aerial vehicle of claim 4 , wherein obtaining the relative image capture device orientation calibration data includes obtaining the relative image capture device orientation calibration data using spatiotemporal calibration based on the temporal feature correlation data.

7 . A method comprising:
controlling, by a processor in response to instructions stored on a non-transitory computer readable medium, an unmanned aerial vehicle to traverse a portion of an operational environment of the unmanned aerial vehicle using relative image capture device orientation calibration by:
obtaining a first image from a fixed orientation image capture device of the unmanned aerial vehicle;
obtaining a second image from an adjustable orientation image capture device of the unmanned aerial vehicle;
obtaining feature correlation data based on the first image and the second image;
obtaining relative image capture device orientation calibration data based on the feature correlation data, the relative image capture device orientation calibration data indicating an orientation of the adjustable orientation image capture device relative to the fixed orientation image capture device;
obtaining relative object orientation data based on the relative image capture device orientation calibration data, the relative object orientation data representing a three-dimensional orientation of an external object relative to the adjustable orientation image capture device; and
controlling a trajectory of the unmanned aerial vehicle in response to the relative object orientation data.

8 . The method of claim 7 , wherein a portion of a field of view of the fixed orientation image capture device corresponding to capturing the first image overlaps a portion of a field of view of the adjustable orientation image capture device corresponding to capturing the second image.

9 . The method of claim 8 , wherein:
obtaining feature correlation data includes obtaining spatial feature correlation data, wherein obtaining the spatial feature correlation data includes obtaining the spatial feature correlation data such that the spatial feature correlation data indicates a correlation between a feature from the first image and a corresponding feature from the second image; and obtaining the relative image capture device orientation calibration data includes using five-point relative positioning based on the spatial feature correlation data.

10 . The method of claim 7 , wherein a field of view of the fixed orientation image capture device corresponding to capturing the first image is non-overlapping with a field of view of the adjustable orientation image capture device corresponding to capturing the second image.

11 . The method of claim 10 , wherein obtaining feature correlation data includes obtaining temporal feature correlation data, wherein obtaining the temporal feature correlation data includes:
obtaining a third image from the fixed orientation image capture device, the third image sequentially subsequent to the first image; obtaining first temporal feature correlation data indicating a correlation between a feature from the first image and a corresponding feature from the third image; obtaining a fourth image from the adjustable orientation image capture device, the fourth image sequentially subsequent to the second image; and obtaining second temporal feature correlation data indicating a correlation between a feature from the second image and a corresponding feature from the fourth image.

12 . The method of claim 11 , wherein:
temporal feature correlation includes:
obtainin

Assignees

• Skydio Inc

Inventors

• Omari Sammy
• Gohl Pascal
• Murguet Axel
• Bruneau Garance
• Diem Fabio
• Schmid Lukas

Classifications

• G05D1/226

  Communication links with the remote-control arrangements · CPC title

• G05D1/689

  Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672) · CPC title

• G05D1/2247

  providing the operator with simple or augmented images from one or more cameras · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• B64U10/14

  with four distinct rotor axes, e.g. quadcopters · CPC title