Unmanned aerial vehicle camera calibration as part of departure or arrival at a materials handling facility

Patent metadata

Field	Value
Publication number	US-9972212-B1
Application number	US-201615188904-A
Country	US
Kind code	B1
Filing date	Jun 21, 2016
Priority date	Jun 21, 2016
Publication date	May 15, 2018
Grant date	May 15, 2018

Abstract

This disclosure describes systems, methods, and apparatus for automating the verification of aerial vehicle sensors as part of a pre-flight, flight departure, in-transit flight, and/or delivery destination calibration verification process. At different stages, aerial vehicle sensors may obtain sensor measurements about objects within an environment, the obtained measurements may be processed to determine information about the object, as presented in the measurements, and the processed information may be compared with the actual information about the object to determine a variation or difference between the information. If the variation is within a tolerance range, the sensor may be auto adjusted and operation of the aerial vehicle may continue. If the variation exceeds a correction range, flight of the aerial vehicle may be aborted and the aerial vehicle routed for a full sensor calibration.

First claim

What is claimed is:

1. A camera calibration verification method, comprising:
navigating an unmanned aerial vehicle (UAV) over a marker at known position;
obtaining, with a camera coupled to the UAV, an image of the marker as the UAV is navigating over the marker;
processing the image to generate processed information corresponding to the marker as represented in the image;
comparing the processed information with actual information about the marker;
determining if a variation determined from the comparison of the processed information and the actual information is within a tolerance range; and
in response to determining that the variation is within a tolerance range, continuing a flight of the UAV toward a destination.

2. The method of claim 1 , wherein:
the processed information provides an indication of at least one of a size, a shape, or a color of the marker as represented in the image; and
the comparing includes comparing at least one of the size, the shape, or the color of the marker as indicated in the processed information with a known size, a known shape, or a known color of the marker.

3. The method of claim 1 , wherein navigating includes navigating the UAV along a defined flight path over the marker.

4. The method of claim 1 , further comprising:
determining if the variation exceeds a correction threshold; and
in response to determining that the variation exceeds a correction threshold, aborting the flight of the UAV toward the destination.

5. The method of claim 1 , further comprising:
in response to determining that the variation is within a tolerance range,
software correcting a calibration of the camera based at least in part on the variation; and
continuing the flight of the UAV toward the destination.

6. An aerial vehicle calibration system, comprising:
a marker positioned within a materials handling facility;
an aerial vehicle;
a camera coupled to the aerial vehicle;
a computing system configured to at least:
receive an image from the camera, the image including a representation of the marker and obtained as the aerial vehicle is navigating such that the marker is in a field of view of the camera;
process the image to determine a variation between at least one aspect of the marker as represented in the image with at least one known aspect of the marker;
determine that the variation does not exceed a threshold;
determine that the variation is within a tolerance range; and
adjust a calibration of the camera based at least in part on the variation.

7. The aerial vehicle calibration system of claim 6 , wherein the computing system is included in the aerial vehicle.

8. The aerial vehicle calibration system of claim 6 , wherein:
the computing system is part of an aerial vehicle management system that is remote from the aerial vehicle; and
the computing system is further configured to at least:
subsequent to adjusting the calibration of the camera,
send instructions that cause the aerial vehicle to navigate such that the marker is within the field of view of the camera;
send instructions that cause the camera to obtain a second image of the marker;
receive the second image; and
process the second image to determine that the at least one aspect of the marker as represented in the second image corresponds to the at least one known aspect of the marker.

9. The aerial vehicle calibration system of claim 6 , wherein the image is obtained as the aerial vehicle navigates a defined flight path.

10. The aerial vehicle calibration system of claim 6 , wherein the marker is representative of a marker positioned at a delivery destination.

11. The aerial vehicle calibration system of claim 6 , wherein the computing system is further configured to at least:
receive a plurality of images from the camera, each image including a respective representation of the marker;
process each of the plurality of images to determine a respective variation between at least one of a size of the marker as represented in each of the plurality of images, a shape of the marker as represented in each of the plurality of images, a color of the marker as represented in each of the plurality of images, or a pattern of the marker as represented in each of the plurality of images with at least one of a known size of the marker, a known shape of the marker, a known color of the marker, or a known pattern of the marker;
determine that the variations are each within a tolerance range; and
adjust the calibration of the camera based at least in part on at least one of the variations.

12. The aerial vehicle calibration system of claim 6 , wherein the calibration of the camera is software adjusted based at least in part on the variation.

13. The aerial vehicle calibration system of claim 6 , further comprising:
a distance determining element positioned adjacent an aerial vehicle departure location of the materials handling facility;
the computing system further configured to at least:
receive from the distance determining element a measured distance between the distance determining element and the aerial vehicle;
receive from the aerial vehicle an altitude of the aerial vehicle; and
determine that a variation between the measured distance and the altitude is within an altitude tolerance range.

14. The aerial vehicle calibration system of claim 6 , further comprising:
a distance determining element positioned adjacent an aerial vehicle departure location of the materials handling facility;
the computing system further configured to at least:
receive from the distance determining element a measured distance between the distance determining element and the aerial vehicle;
receive from the aerial vehicle an altitude of the aerial vehicle;
determine that a variation between the measured distance and the altitude exceeds an altitude tolerance range; and
cause an altitude as measured by the aerial vehicle to be altered to account for the variation between the measured distance and the altitude.

15. The aerial vehicle calibration system of claim 6 , wherein the computing system is further configured to at least:
process the image to determine a distance variation between a relative position of the marker as determined by the aerial vehicle and an actual position of the marker;
determine that the distance variation is within a distance tolerance range; and
adjust a position of the aerial vehicle as registered by the aerial vehicle based at least in part on the distance variation.

16. A camera calibration method, comprising:
navigating an unmanned aerial vehicle (UAV) such that a marker at a known position is within a field of view of a camera coupled to the UAV;
obtaining, with the camera, an image of the marker while the marker is within the field of view of the camera;
processing the image to generate processed information corresponding to the marker as represented in the image;
determining a variation between the processed information and actual information about the marker; and
adjusting a calibration of the camera based at least in part on the variation.

17. The camera calibration method of claim 16 , wherein:
the processed information includes at least one of a shape of the marker as represented in the image, a size of the marker as represented in the image, a color of the marker as represented in the image, or a pattern of the marker as represented in the image; and
determining the variation includes comparing at least one of the shape of the marker, the size of the marker, the color of the marker, or the pattern of the mar

Assignees

• Amazon Tech Inc

Inventors

• Sperindeo Samuel
• Barash Benji
• Schoenberg Yves Albers
• Buchmueller Daniel

Classifications

• B64U2201/00

  UAVs characterised by their flight controls · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• B64U50/12

  using turbine engines, e.g. turbojets or turbofans · CPC title

• B64U50/11

  using internal combustion piston engines · CPC title