Image-based motion sensor and related multi-purpose camera system

Patent metadata

Field	Value
Publication number	US-9609217-B2
Application number	US-201213610890-A
Country	US
Kind code	B2
Filing date	Sep 12, 2012
Priority date	Nov 2, 2011
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

An image-based motion sensor includes a camera system and a processing system. The camera system is arranged for generating an image output including a plurality of captured images. The processing system is arranged for obtaining a motion sensor output by processing the image output, wherein the motion sensor output includes information indicative of at least one of a motion status and an orientation status of the image-based motion sensor. For example, information indicative of the orientation status of the image-based motion sensor may includes roll, pitch, and yaw related motion information, and information indicative of the motion status of the image-based motion sensor may includes speed, acceleration and displacement information.

First claim

What is claimed is:

1. An image-based motion sensor, comprising:
a camera system, arranged for generating an image output including a plurality of captured images; and
a processing system, arranged for obtaining a motion sensor output by processing the image output, and identifying a user input as one of a plurality of pre-defined user actions according to the motion sensor output, wherein different functions of at least one application performed by one electronic device are controlled by the pre-defined user actions; and the motion sensor output includes information indicative of at least one of a motion status and an orientation status of the image-based motion sensor;
wherein the camera system performs an RGB-to-YUV color transform upon one color image to generate one grayscale image, and only luminance values derived from the RGB-to-YUV color transform are involved in obtaining the motion sensor result.

2. The image-based motion sensor of claim 1 , wherein the motion sensor output includes information indicative of the orientation status of the image-based motion sensor, and the information indicative of the orientation status includes roll, pitch, and yaw related motion information.

3. The image-based motion sensor of claim 1 , wherein the motion sensor output includes information indicative of the motion status of the image-based motion sensor, and the information indicative of the motion status includes speed, acceleration and displacement information.

4. The image-based motion sensor of claim 1 , wherein the processing system comprises:
a global motion estimation block, arranged for generating global motion information according to the image output; and
a processor, arranged for determining the motion sensor output according to the global motion information.

5. The image-based motion sensor of claim 4 , wherein the global motion estimation block includes an electronic image stabilization (EIS) module arranged to perform global motion estimation.

6. The image-based motion sensor of claim 4 , wherein the global motion estimation block comprises:
a motion estimation unit, arranged for performing motion estimation upon each of a plurality of pairs of co-located sub-images in a first captured image and a second captured image, and generating a plurality of motion vectors of the pairs of co-located sub-images, wherein the first captured image and the second captured image are consecutive captured images; and
a motion vector analysis unit, arranged for generating the global motion information by analyzing the motion vectors.

7. The image-based motion sensor of claim 4 , wherein the global motion estimation block comprises:
an image down-scaling unit, arranged for down-scaling a first captured image and a second captured image to generate a first down-scaled image and a second down-scaled image, respectively, wherein the first captured image and the second captured image are consecutive captured images; and
a global motion estimation unit, arranged for generating the global motion information by performing global motion estimation upon the first down-scaled image and the second down-scaled image.

8. The image-based motion sensor of claim 4 , wherein the global motion estimation block comprises:
an image down-scaling unit, arranged for generating a first down-scaled image of a first captured image and a second down-scaled image of a second captured image, wherein the first captured image and the second captured image are consecutive captured images;
a transform unit, arranged for performing a specific transform upon the first down-scaled image and the second down-scaled image to generate a first transformed image and a second transformed image, respectively; and
a global motion estimation unit, arranged for generating the global motion information by performing global motion estimation according to the first transformed image and the second transformed image.

9. The image-based motion sensor of claim 8 , wherein the specific transform is a linear polar transform or a Log-polar transform.

10. The image-based motion sensor of claim 9 , wherein the global motion estimation unit is further arranged for generating another global motion information by performing global motion estimation upon a third down-scaled image and a fourth down-scaled image; and the image down-scaling unit comprises:
an image down-scaling circuit, arranged for down-scaling the first captured image and the second captured image to generate the third down-scaled image and the fourth down-scaled image, respectively; and
an image alignment circuit, arranged for generating the first down-scaled image and the second down-scaled image by referring to the another global motion information to perform image alignment upon the third down-scaled image and the fourth down-scaled image.

11. The image-based motion sensor of claim 8 , wherein the global motion estimation unit performs the global motion estimation in one direction only.

12. The image-based motion sensor of claim 1 , wherein the processing system comprises:
an edge detection block, arranged for generating global motion information by performing edge detection according to the image output; and
a processor, arranged for determining the motion sensor output according to the global motion information.

13. The image-based motion sensor of claim 12 , wherein the edge detection block comprises:
an image down-scaling unit, arranged for down-scaling a first captured image and a second captured image to generate a first down-scaled image and a second down-scaled image, respectively, wherein the first captured image and the second captured image are consecutive captured images;
an edge detection unit, arranged for performing edge detection upon the first down-scaled image to generate first edge information, and performing edge detection upon the second down-scaled image to generate second edge information; and
an edge histogram unit, arranged for deriving a first edge histogram from the first edge information; and deriving a second edge histogram from the second edge information; and
an edge histogram matching unit, arranged for generating the global motion information by performing edge histogram matching upon the first edge histogram and the second edge histogram.

14. The image-based motion sensor of claim 13 , wherein the edge histogram unit is further arranged for performing a filtering operation upon the first edge histogram and the second edge histogram before transmitting the first edge histogram and the second edge histogram to the edge histogram matching unit.

15. The image-based motion sensor of claim 1 , wherein the image-based motion sensor is disposed in a first electronic device external to a second electronic device, the processing system is further arranged to generate a control signal to the second electronic device according to the motion sensor output.

16. A multi-purpose camera system, comprising:
an image capture block, arranged for generating an image signal; and
an image signal processing block, arranged for processing the image signal; wherein when the multi-purpose camera system is operated in a first operation mode, the multi-purpose camera system acts as a camera for generating a captured image output; and when the multi-purpose camera system is operated in a second operation mode, the multi-purpose camera system acts as part of a motion sensor for generating a motion sensor output, wherein the motion sensor output includes information indicative of at least one of a motion status and an orientation status of the multi-purpose camera system, an

Assignees

• Ju Chi-Cheng
• Ho Cheng-Tsai
• Chen Ding-Yun
• Mediatek Inc

Inventors

• Ju Chi-Cheng
• Ho Cheng-Tsai
• Chen Ding-Yun

Classifications

• H04N23/6811 Primary

  based on the image signal · CPC title

• H04N23/68

  for stable pick-up of the scene, e.g. compensating for camera body vibrations · CPC title

• H04N23/667 Primary

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• H04N23/95

  Computational photography systems, e.g. light-field imaging systems · CPC title