Image processing device, image display device, and program

Patent metadata

Field	Value
Publication number	US-11240407-B2
Application number	US-201616342393-A
Country	US
Kind code	B2
Filing date	Oct 31, 2016
Priority date	Oct 31, 2016
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

Provided is an image processing device, image display device, and program that allow an image captured by an imaging device not having a vibration suppression function or an image whose vibration has been suppressed incompletely to be displayed on a display device with the vibration suppressed. The image processing device includes a motion estimator configured to estimate the amount of motion of an object between a first image and a second image later than the first image and a motion compensator configured to perform a conversion process on the second image so that vibration of the object between the first image and the second image is suppressed, on the basis of the amount of motion of the object. The motion estimator has a first estimation mode in which the amount of motion of the object is estimated in a predetermined search area and a second estimation mode in which the amount of motion of the object is estimated in a larger area than the search area in the first estimation mode.

First claim

The invention claimed is:

1. An image processing device comprising:
a motion estimating circuit configured to estimate the amount of motion of an object between a first image and a second image later than the first image;
a motion compensating circuit configured to perform a conversion process on the second image so that vibration of the object between the first image and the second image is suppressed, on the basis of the amount of motion of the object; and
a mode switching circuit configured to switch between a first estimation mode and a second estimation mode, wherein
the motion estimating circuit has the first estimation mode in which the amount of motion of the object is estimated in a predetermined search area and the second estimation mode in which the amount of motion of the object is estimated in a larger area than the predetermined search area in the first estimation mode,
when a predetermined criterion is satisfied, the motion compensating circuit performs the conversion process on the basis of the amount of motion estimated in the second estimation mode, and
the predetermined search area of the first estimation mode is in the first image and is smaller than the first image,
the larger area of the second estimation mode is in the first image and includes the predetermined search area of the first estimation mode, and

when the predetermined criterion is not satisfied, the motion compensating circuit performs the conversion process on the basis of the amount of motion estimated in the first estimation mode,
in the first estimation mode, the motion estimating circuit defines, as an estimation error, a value calculated on the basis of a difference in luminance between the object included in the first image and a predetermined area included in the second image, and
the predetermined criterion is that the estimation error exceeds a predetermined value, or that the estimation error exceeds a predetermined multiple of an average value, or both.

2. The image processing device of claim 1 , wherein the second estimation mode is a global motion estimation mode in which the amount of motion of the object is estimated from the entire first and second images.

3. The image processing device of claim 1 , wherein in the second estimation mode, the amount of motion of the object is estimated in a new search area obtained by expanding a size of the search area in the first estimation mode.

4. The image processing device of claim 1 , wherein the motion estimator estimates the amount of motion of the object using pattern matching or a feature vector.

5. The image processing device of claim 1 , further comprising:
an area setting circuit configured to set an area corresponding to the object in the first estimation mode.

6. The image processing device of claim 5 , wherein the motion compensating circuit performs the conversion process on the second image so that the area set by the area setting circuit is located in an approximate center of a display surface on which the first and second images are displayed.

7. The image processing device of claim 1 , wherein when the predetermined criterion is not satisfied, the mode switching circuit switches the second estimation mode to the first estimation mode.

8. An image display device comprising the image processing device of claim 1 .

9. An image processing program for causing a computer to function as:
a motion estimating circuit configured to estimate the amount of motion of an object between a first image and a second image later than the first image;
a motion compensating circuit configured to perform a conversion process on the second image so that vibration of the object between the first image and the second image is suppressed, on the basis of the amount of motion of the object; and
a mode switching circuit configured to switch between a first estimation mode and a second estimation mode, wherein
the image processing program is stored in a computer-readable non-transitory storage medium,
the motion estimating circuit has the first estimation mode in which the amount of motion of the object is estimated in a predetermined search area and the second estimation mode in which the amount of motion of the object is estimated in a larger area than the predetermined search area in the first estimation mode,
when a predetermined criterion is satisfied, the motion compensating circuit performs the conversion process on the basis of the amount of motion estimated in the second estimation mode, and
the predetermined search area of the first estimation mode is in the first image and is smaller than the first image,
the larger area of the second estimation mode is in the first image and includes the predetermined search area of the first estimation mode, and

when the predetermined criterion is not satisfied, the motion compensating circuit performs the conversion process on the basis of the amount of motion estimated in the first estimation mode,
in the first estimation mode, the motion estimating circuit defines, as an estimation error, a value calculated on the basis of a difference in luminance between the object included in the first image and a predetermined area included in the second image, and
the predetermined criterion is that the estimation error exceeds a predetermined value, or that the estimation error exceeds a predetermined multiple of an average value, or both.

10. An image processing device comprising:
a motion estimating circuit configured to estimate the amount of motion of an object between a first image and a second image later than the first image; and
a motion compensating circuit configured to perform a conversion process on the second image so that vibration of the object between the first image and the second image is suppressed, on the basis of the amount of motion of the object, wherein
the motion estimating circuit has a first estimation mode in which the amount of motion of the object is estimated in a predetermined search area and a second estimation mode in which the amount of motion of the object is estimated in a larger area than the predetermined search area in the first estimation mode,
the predetermined search area of the first estimation mode is in the first image and is smaller than the first image,

the second estimation mode is a global motion estimation mode, and the larger area of the global motion estimation mode match with the entire first image,
in the global motion estimation mode, the larger area than the predetermined search area is divided into a plurality of divided areas, and movement direction is estimated for each divided area, and the amount of motion of the object is estimated from the movement directions of the divided areas,
when a predetermined criterion is satisfied, the motion compensating circuit performs the conversion process on the basis of the amount of motion estimated in the second estimation mode, and
when the predetermined criterion is not satisfied, the motion compensating circuit performs the conversion process on the basis of the amount of motion estimated in the first estimation mode.

Assignees

• Eizo Corp

Inventors

• Chen Haifeng
• Aoki Reo
• Higashi Masafumi

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• H04N23/6811 Primary

  based on the image signal · CPC title

• H04N23/81 Primary

  for suppressing or minimising disturbance in the image signal generation · CPC title