Image processing apparatus to which moving vector search technique is applicable, control method therefor, and storage medium storing control program therefor

Patent metadata

Field	Value
Publication number	US-9609355-B2
Application number	US-201514831534-A
Country	US
Kind code	B2
Filing date	Aug 20, 2015
Priority date	Aug 22, 2014
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

An image processing apparatus that is capable of improving image stabilization performance by improving motion vector search performance by restricting a search range even for an image with a large motion due to a camera shake of an image pickup apparatus at a tele-end side of a zoom lens or a shoot while walking at a wide-end side. A motion vector search unit searches for a motion vector of a reference image later input from an image pickup device with respect to a standard image input previously. A control unit sets at least one of a search range and a search position of a motion vector searched by the motion vector search unit based on position-posture change information about the image pickup apparatus detected by a detection unit and image stabilization control information obtained from an image stabilizing unit.

First claim

What is claimed is:

1. An image processing apparatus comprising:
at least one processor which executes a program stored in a memory, wherein the at least one processor functions as a motion vector search unit and a control unit, and wherein
the motion vector search unit is configured to search for a motion vector of a reference image later input with respect to a standard image input previously; and
the control unit is configured to change, based on position-posture change information about an image pickup apparatus and image stabilization control information, at least one of a search range and a search position of a motion vector searched by said motion vector search unit.

2. The image processing apparatus according to claim 1 , wherein the at least one processor further functions as:
a camera-work-determination unit configured to determine whether a camera work exists based on the position-posture change information,
an edge-process-determination unit configured to determine whether an edge process is necessary based on the position-posture change information, and
an image-stabilization-control-information generation unit configured to generate image stabilization control information including determination results and intensity levels of said camera-work-determination unit and edge-process-determination unit, and
wherein said control unit expands the search range according to at least one of the intensity levels, when it is determined that at least one of the camera work and edge process is necessary based on the image stabilization control information.

3. An image processing apparatus comprising:
a motion vector search unit configured to search for a motion vector of a reference image later input with respect to a standard image input previously; and
a control unit configured to receive position-posture change information about an image pickup apparatus detected by a rotation sensor that is attached to a rotation axis coaxial to an optical axis of the image pickup apparatus, to expand a search range for a motion vector by said motion vector search unit when a rotation amount around the rotation axis is larger than a predetermined value, and to reduce the search range when the rotation amount around the rotation axis is smaller than the predetermined value.

4. The image processing apparatus according to claim 1 , wherein said control unit sets a wide search range by easing or releasing the control of the search range when the number of the motion vectors becomes equal to or smaller than a preset threshold, and narrows the search range when the number of the motion vectors exceeds the threshold.

5. The image processing apparatus according to claim 1 , wherein said control unit reduces the search range when position-posture displacement information that is found by integrating the position-posture change information and a difference of change in an image stabilization position detected with an image stabilization position detection unit are small, and expands the search range when the position-posture displacement information and the difference are large.

6. The image processing apparatus according to claim 5 , wherein said control unit sets a wide search range by easing or releasing the control of the search range when a deviation between an image motion prediction value obtained from the position-posture displacement information and the difference of change in the image stabilization position, and a value of the motion vector detected with said motion vector search unit is large, and narrows the search range when the deviation is small.

7. A control method for an image processing apparatus, the control method comprising:
a motion vector search step of searching for a motion vector of a reference image later input with respect to a standard image input previously; and
a control step of changing, based on position-posture change information about an image pickup apparatus and image stabilization control information, at least one of a search range and a search position of a motion vector searched in said motion vector search step.

8. A non-transitory computer-readable storage medium storing a control program causing a computer to execute a control method for an image processing apparatus, the control method comprising:
a motion vector search step of searching for a motion vector of a reference image later input with respect to a standard image input previously; and
a control step of changing, based on position-posture change information about an image pickup apparatus and image stabilization control information, at least one of a search range and a search position of a motion vector searched in said motion vector search step.

9. An image pickup apparatus comprising:
an image pickup device;
at least one processor which executes a program stored in a memory, wherein the at least one processor functions as an image stabilization unit, a motion vector search unit, a detection unit and a control unit, and wherein:
the motion vector search unit is configured to search for a motion vector of a reference image later input from said image pickup device with respect to a standard image input previously;
the detection unit is configured to detect position-posture change information about an image pickup apparatus;
the control unit is configured to change, based on the position-posture change information detected by said detection unit and image stabilization control information obtained from said image stabilizing unit, set at least one of a search range and a search position of a motion vector searched by said motion vector search unit.

10. An image pickup apparatus comprising:
an image stabilization unit;
an image pickup device;
a motion vector search unit configured to search for a motion vector of a reference image later input from said image pickup device with respect to a standard image input previously;
a rotation sensor that is attached to a rotation axis coaxial to an optical axis of the image pickup apparatus; and
a control unit configured to receive position-posture change information about the image pickup apparatus detected by said rotation sensor, to expand a search range for a motion vector by said motion vector search unit when a rotation amount around the rotation axis is larger than a predetermined value, and to reduce the search range when the rotation amount around the rotation axis is smaller than the predetermined value.

Assignees

• Canon Kk

Inventors

• Tsubaki Hidetoshi

Classifications

• H04N23/687

  by shifting the lens or sensor position · CPC title

• H04N19/521 Primary

  for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors · CPC title

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/6811

  based on the image signal · CPC title

• G06T7/248

  involving reference images or patches · CPC title