Optical flow sensor, methods, remote controller device, and rotatable electronic device

Patent metadata

Field	Value
Publication number	US-10612951-B2
Application number	US-201715609023-A
Country	US
Kind code	B2
Filing date	May 31, 2017
Priority date	May 31, 2017
Publication date	Apr 7, 2020
Grant date	Apr 7, 2020

Abstract

An optical flow sensing method includes: using an image sensor to capture images; using a directional-invariant filter device upon at least one first block of the first image to process values of pixels of the at least one first block of the first image, to generate a first filtered block image; using the first directional-invariant filter device upon at least one first block of the second image to process values of pixels of the at least one first block of the second image, to generate a second filtered block image; comparing the filtered block images to calculate a correlation result; and estimating a motion vector according to a plurality of correlation results.

First claim

What is claimed is:

1. A method of an optical flow sensor, comprising:
using a first image sensor to capture a first image and a second image;
using a first directional-invariant filter device upon at least one first block of the first image to process pixel values of the at least one first block of the first image, to generate a first filtered block image;
using the first directional-invariant filter device upon at least one first block of the second image to process pixel values of the at least one first block of the second image, to generate a second filtered block image;
comparing the first filtered block image with the second filtered block image to calculate a first correlation result; and
estimating a motion vector of a motion of the optical flow sensor according to a plurality of first correlation results;
wherein the first directional-invariant filter device comprises a plurality of directional adaptive filters with different direction enhancements, and each directional adaptive filter is applied for the at least one first block of the first image and the at least one first block of the second image.

2. The method of claim 1 , wherein a position of the at least one first block of the first image corresponds to a position of the at least one first block of the second image.

3. The method of claim 1 , wherein the comparing step comprises:
calculating the first correlation result by calculating a difference between pixel values of the first filtered block image and pixel values of the second filtered block image or by calculating a product value of the pixel values of the first filtered block image and the pixel values of the second filtered block image.

4. The method of claim 1 , wherein the estimated motion vector is further used as a reference to calculate a first correlation result of another set of filtered block images.

5. The method of claim 1 , further comprising:
detecting a status of an external control switch; and
outputting the estimated motion vector when detecting that the external control switch is at an ON state.

6. The method of claim 1 , further comprising:
obtaining proportional integral derivate information from at least one of a Gyro sensor, a barometer, and an acoustic sensor;
calibrating the estimated motion vector according to the proportional integral derivate information; and
compensating motion of an unmanned aerial vehicle device.

7. The method of claim 1 , further comprising:
using a second image sensor to capture a third image and a fourth image;
using a second directional-invariant filter device upon at least one second block image of the third image to process pixel values of the at least one second block image of the third image, to generate a third filtered block image;
using the second directional-invariant filter device upon at least one second block image of the fourth image to process pixel values of the at least one second block image of the fourth image, to generate a fourth filtered block image;
comparing the third filtered block image with the fourth filtered block image to calculate a second correlation result; and
estimating another motion vector of the motion of the optical flow sensor according to a plurality of second correlation results.

8. The method of claim 7 , further comprising:
configuring the first image sensor with a wider field of view in a vertical direction to capture images of objects at a lower altitude to obtain the first image and the second image; and
configuring the second image sensor with a narrower field of view in the vertical direction to capture images of objects at a higher altitude to obtain the third image and the fourth image.

9. The method of claim 7 , further comprising:
estimating a rotation angle according to the estimated motion vector and the another estimated motion vector.

10. The method of claim 7 , further comprising:
configuring a position of the first image sensor and a position of the second image sensor at a same plane.

11. The method of claim 7 , further comprising:
configuring a position of the first image sensor and a position of the second image sensor at two different planes respectively.

12. The method of claim 11 , further comprising:
estimating a behavior or motion of a rotatable electronic device according to the estimated motion vector and the another estimated motion vector.

13. The method of claim 1 , wherein the at least one first block of the first image excludes all pixels positioned at edges of the first image, and the at least one first block of the second image excludes all pixels positioned at edges of the second image.

14. An optical flow sensor, comprising:
a first image sensor, configured for capturing a first image and a second image;
a first directional-invariant filter device, coupled to the first image sensor, being applied for at least one first block of the first image to process pixel values of the at least one first block of the first image, to generate a first filtered block image, and applied for at least one first block of the second image to process pixel values of the at least one first block of the second image, to generate a second filtered block image; and
a processor, coupled to the first directional-invariant filter device, configured for comparing the first filtered block image with the second filtered block image to calculate a first correlation result, and for estimating a motion vector of a motion of the optical flow sensor according to a plurality of first correlation results;
wherein the first directional-invariant filter device comprises a plurality of directional adaptive filters with different direction enhancements, and each directional adaptive filter is applied for the at least one first block of the first image and the at least one first block of the second image.

15. The optical flow sensor of claim 14 , further comprising:
a global shutter;
wherein the first image sensor is used with the global shutter to respectively capture the first image and the second image at different timings.

16. The optical flow sensor of claim 14 , wherein the processor is arranged for:
calculating the first correlation result by calculating a difference between pixel values of the first filtered block image and pixel values of the second filtered block image or by calculating a product value of the pixel values of the first filtered block image and the pixel values of the second filtered block image.

17. The optical flow sensor of claim 14 , wherein the estimated motion vector is further used as a reference to calculate a first correlation result of another set of filtered block images.

18. The optical flow sensor of claim 14 , wherein the at least one first block of the first image excludes all pixels positioned at edges of the first image, and the at least one first block of the second image excludes all pixels positioned at edges of the second image.

19. A remote controller device, comprising:
the optical flow sensor of claim 13 ;
a control switch; and
a processing circuit, configured for detecting a status of the control switch and for outputting the estimated motion vector when detecting that the control switch is at an ON state.

20. The method of claim 1 , wherein each of the plurality of directional adaptive filters with the different direction enhancements is respectively arranged to process the at least one first block of the first image and the at least one first block of the second image, to generate a plurality of first filter ou

Assignees

• Pixart Imaging Inc

Inventors

• Chen Hsin-Chia
• Huang Sen-Huang
• Wang Wei-Chung
• Huang Chao-Chien
• Chang Ting-Yang
• Chen Chun-Wei

Classifications

• G06T5/10

  using non-spatial domain filtering · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title