Adaptive motion filtering in an unmanned autonomous vehicle

Patent metadata

Field	Value
Publication number	US-10873702-B2
Application number	US-201716324308-A
Country	US
Kind code	B2
Filing date	Feb 24, 2017
Priority date	Sep 23, 2016
Publication date	Dec 22, 2020
Grant date	Dec 22, 2020

Abstract

Embodiments include devices and methods for adaptive image processing in an unmanned autonomous vehicle (UAV). In various embodiments, an image sensor may capture an image, while a processor of the UAV obtains attitude information from one or more attitude sensors. Such information may include the relative attitude of the UAV and changes in attitude. The processor of the UAV may determine a UAV motion mode based, at least in part, on the obtained attitude information. The UAV motion mode may result in the modification of yaw correction parameters. The processor of the UAV may further execute yaw filtering on the image based, at least in part, on the determined motion mode.

First claim

What is claimed is:

1. A method of motion filtering an image in an unmanned autonomous vehicle (UAV), comprising:
capturing an image by an image sensor associated with the UAV;
obtaining attitude information of the UAV by an attitude sensor of the UAV;
determining a UAV motion mode based, at least in part, on the obtained attitude information; and
executing yaw filtering on the image based, at least in part, on the determined UAV motion mode wherein the executing yaw filtering comprises:
calculating an average yaw over a sliding window;
calculating a standard deviation of the calculated average yaw;
determining whether a yaw threshold is exceeded based, at least in part, on the calculated average yaw and the calculated standard deviation;
modifying a yaw correction parameter based, at least in part, on the determination of whether the yaw threshold is exceeded; and
executing a filter function using the yaw correction parameter.

2. The method of claim 1 , further comprising setting the yaw threshold to a frequency of 35 Hz in response to determining that the UAV motion mode is a fast yaw mode.

3. The method of claim 1 , further comprising setting the yaw threshold to a frequency of 25 Hz in response to determining that the UAV motion mode is a hover mode.

4. The method of claim 1 , further comprising in response to determining that the yaw threshold is exceeded:
incrementing the yaw correction parameter;
determining whether a maximum yaw correction is exceeded; and
setting the yaw correction parameter to the maximum yaw correction.

5. The method of claim 1 , further comprising in response to determining that the yaw threshold is not exceeded:
decrementing the yaw correction parameter;
determining whether a minimum yaw correction is exceeded; and
setting the yaw correction parameter to the minimum yaw correction.

6. The method of claim 1 , wherein the filter function includes an infinite impulse response filter.

7. An unmanned autonomous vehicle (UAV), comprising:
an image sensor;
an attitude sensor; and
a processor coupled to the image sensor and the attitude sensor, and configured with processor-executable instructions to:
capture an image by the image sensor;
obtain attitude information of the UAV by the attitude sensor;
determine a UAV motion mode based, at least in part, on the obtained attitude information; and
execute yaw filtering on the image based, at least in part, on the determined UAV motion mode by:
calculating an average yaw over a sliding window;
calculating a standard deviation of the calculated average yaw;
determining whether a yaw threshold is exceeded based, at least in part, on the calculated average yaw and the calculated standard deviation;
modifying a yaw correction parameter based, at least in part, on the determination of whether the yaw threshold is exceeded; and
executing a filter function using the yaw correction parameter.

8. The UAV of claim 7 , wherein the processor is further configured with processor-executable instructions to set the yaw threshold to a frequency of 35 Hz in response to determining that the UAV motion mode is a fast yaw mode.

9. The UAV of claim 7 , wherein the processor is further configured with processor-executable instructions to set the yaw threshold to a frequency of 25 Hz in response to determining that the UAV motion mode is a hover mode.

10. The UAV of claim 7 , wherein in response to determining that the yaw threshold is exceeded the processor is further configured with processor-executable instructions to:
increment the yaw correction parameter;
determine whether a maximum yaw correction is exceeded; and
set the yaw correction parameter to the maximum yaw correction.

11. The UAV of claim 7 , wherein in response to determining that the yaw threshold is not exceeded the processor is further configured with processor-executable instructions to:
decrement the yaw correction parameter;
determine whether a minimum yaw correction is exceeded; and
setting the yaw correction parameter to the minimum yaw correction.

12. The UAV of claim 7 , wherein the filter function includes an infinite impulse response filter.

13. An unmanned autonomous vehicle (UAV), comprising:
means for capturing an image;
means for obtaining attitude information of the UAV;
means for determining a UAV motion mode based, at least in part, on the obtained attitude information; and
means for executing yaw filtering on the image based, at least in part, on the determined UAV motion mode comprising:
means for calculating an average yaw over a sliding window;
means for calculating a standard deviation of the calculated average yaw;
means for determining whether a yaw threshold is exceeded based, at least in part, on the calculated average yaw and the calculated standard deviation;
means for modifying a yaw correction parameter based, at least in part, on the determination of whether the yaw threshold is exceeded; and
means for executing a filter function using the yaw correction parameter.

14. A non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of an unmanned autonomous vehicle (UAV) to perform operations comprising:
capturing an image by an image sensor associated with the UAV;
obtaining attitude information of the UAV by an attitude sensor of the UAV;
determining a UAV motion mode based, at least in part, on the obtained attitude information; and
executing yaw filtering on the image based, at least in part, on the determined UAV motion mode by:
calculating an average yaw over a sliding window;
calculating a standard deviation of the calculated average yaw;
determining whether a yaw threshold is exceeded based, at least in part, on the calculated average yaw and the calculated standard deviation;
modifying a yaw correction parameter based, at least in part, on the determination of whether the yaw threshold is exceeded; and
executing a filter function using the yaw correction parameter.

15. The non-transitory processor-readable storage medium of claim 14 , wherein the stored processor-executable instructions are configured to cause the processor of the UAV to perform operations further comprising setting the yaw threshold to a frequency of 35 Hz in response to determining that the UAV motion mode is a fast yaw mode.

16. The non-transitory processor-readable storage medium of claim 14 , wherein the stored processor-executable instructions are configured to cause the processor of the UAV to perform operations further comprising setting the yaw threshold to a frequency of 25 Hz in response to determining that the UAV motion mode is a hover mode.

17. The non-transitory processor-readable storage medium of claim 14 , wherein the stored processor-executable instructions are configured to cause the processor of the UAV to perform operations further comprising in response to determining that the yaw threshold is exceeded:
incrementing the yaw correction parameter;
determining whether a maximum yaw correction is exceeded; and
setting the yaw correction parameter to the maximum yaw correction.

18. The non-transitory processor-readable storage medium of claim 14 , wherein the stored processor-executable instructions are configured to cause the processor of the UAV to perform operations further comprising in response to determining that the yaw threshold is not exceeded:
decrementing the yaw correction parameter;
determining whether a minimum yaw correction is exceeded; and

Assignees

• Qualcomm Inc
• Huang Yin
• Zhang Liang
• Zhu Xiaoyi
• Wang Ruowei
• Ren Jiangtao

Inventors

• Huang Yin
• Zhang Liang
• Zhu Xiaoyi
• Wang Ruowei
• Ren Jiangtao

Classifications

• H04N25/531

  by controlling rolling shutters in CMOS SSIS · CPC title

• H04N23/6815

  by distinguishing pan or tilt from motion · CPC title

• H04N23/6811

  based on the image signal · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title