Image processing device, image processing method, and computer program product for correcting a motion vector of a low-reliability block

Patent metadata

Field	Value
Publication number	US-10516825-B2
Application number	US-201715813529-A
Country	US
Kind code	B2
Filing date	Nov 15, 2017
Priority date	May 21, 2015
Publication date	Dec 24, 2019
Grant date	Dec 24, 2019

Abstract

Motion vector calculator sets motion vector of block corresponding to partial area of first image based on the block and similarity between the block and partial area of second image, motion vector reliability calculator calculates reliability of the motion vector based on distribution of the similarities, motion vector corrector corrects motion vector of low-reliability block having reliability lower than predetermined reliability based on motion vector of high-reliability block different from the low-reliability block and having reliability equal to or higher than the predetermined reliability, and image processor sets second pixel of the second image and corresponds to first pixel of the first image based on the motion vector corrected by the motion vector corrector, and performs image processing using signal value of the first pixel and signal value of the second pixel.

First claim

What is claimed is:

1. An image processing device comprising:
a motion vector calculator which sets a motion vector of a block corresponding to a partial area of a first image based on the block and a similarity between the block and a partial area of a second image;
a motion vector reliability calculator which calculates a reliability of the motion vector based on a distribution of the similarities;
a frame motion vector calculator which calculates a frame motion vector which is a motion vector of the entire first image based on a motion vector of a high-reliability block having a reliability equal to or higher than a predetermined reliability, and a reliability of the frame motion vector based on a distribution of the motion vectors of the high-reliability blocks;
a motion vector corrector which determines whether to correct a motion vector of a low-reliability block having a reliability lower than the predetermined reliability using the frame motion vector based on the reliability of the frame motion vector and corrects the motion vector of the low-reliability block based on the frame motion vector when determining that the motion vector of the low-reliability block is to be corrected;
an image processor which sets a second pixel that is a pixel of the second image and corresponds to a first pixel which is a pixel of the first image based on the motion vector corrected by the motion vector corrector, and performs image processing using a signal value of the first pixel and a signal value of the second pixel; and
a distortion-corrected coordinate calculator which corrects a set of coordinates of each pixel representing the first image based on distortion correcting data representing a relationship between a set of coordinates of each pixel forming a captured image and a set of coordinates after distortion correction.

2. An image processing device comprising:
a motion vector calculator which sets a motion vector of a block corresponding to a partial area of a first image based on the block and a similarity between the block and a partial area of a second image;
a motion vector reliability calculator which calculates a reliability of the motion vector based on a distribution of the similarities;
a frame motion vector calculator which calculates a frame motion vector which is a motion vector of the entire first image based on a motion vector of a high-reliability block different from a low-reliability block having a reliability lower than a predetermined reliability and having a reliability equal to or higher than the predetermined reliability, and a reliability of the frame motion vector based on a distribution of the motion vectors of the high-reliability blocks;
a motion vector corrector which determines whether to correct motion vectors of all blocks of the first image using the frame motion vector based on the reliability of the frame motion vector; and
an image processor which sets a second pixel that is a pixel of the second image and corresponds to a first pixel which is a pixel of the first image based on the motion vector corrected by the motion vector corrector, and performs image processing using a signal value of the first pixel and a signal value of the second pixel; and
a distortion-corrected coordinate calculator which corrects a set of coordinates of each pixel representing the first image based on distortion correcting data representing a relationship between a set of coordinates of each pixel forming a captured image and a set of coordinates after distortion correction.

3. An image processing method of an image processing device, comprising:
a motion vector calculation step of setting a motion vector of a block corresponding to a partial area of a first image based on the block and a similarity between the block and a partial area of a second image;
a motion vector reliability calculation step of calculating a reliability of the motion vector based on a distribution of the similarities;
a frame motion vector calculation step of calculating a frame motion vector which is a motion vector of the entire first image based on a motion vector of a high-reliability block having a reliability equal to or higher than a predetermined reliability, and a reliability of the frame motion vector based on a distribution of the motion vectors of the high-reliability blocks;
a motion vector correction step of determining whether to correct a motion vector of a low-reliability block having a reliability lower than the predetermined reliability using the frame motion vector based on the reliability of the frame motion vector and correcting the motion vector of the low-reliability block based on the frame motion vector when determining the motion vector of the low-reliability block to be corrected;
an image processing step of setting a second pixel that is a pixel of the second image and corresponds to a first pixel which is a pixel of the first image based on the motion vector corrected by the motion vector corrector, and performing image processing using a signal value of the first pixel and a signal value of the second pixel; and
a distortion-corrected coordinate calculation step of correcting a set of coordinates of each pixel representing the first image based on distortion correcting data representing a relationship between a set of coordinates of each pixel forming a captured image and a set of coordinates after distortion correction.

4. An image processing method of an image processing device, comprising:
a motion vector calculation step of setting a motion vector of a block corresponding to a partial area of a first image based on the block and a similarity between the block and a partial area of a second image;
a motion vector reliability calculation step of calculating a reliability of the motion vector based on a distribution of the similarities;
a frame motion vector calculation step of calculating a frame motion vector which is a motion vector of the entire first image based on a motion vector of a high-reliability block different from a low-reliability block having a reliability lower than a predetermined reliability and having a reliability equal to or higher than the predetermined reliability, and a reliability of the frame motion vector based on a distribution of the motion vectors of the high-reliability blocks;
a motion vector correction step of determining whether to correct motion vectors of all blocks of the first image using the frame motion vector based on the reliability of the frame motion vector;
an image processing step of setting a second pixel that is a pixel of the second image and corresponds to a first pixel which is a pixel of the first image based on the motion vector corrected by the motion vector corrector, and performing image processing using a signal value of the first pixel and a signal value of the second pixel; and
a distortion-corrected coordinate calculation step of correcting a set of coordinates of each pixel representing the first image based on distortion correcting data representing a relationship between a set of coordinates of each pixel forming a captured image and a set of coordinates after distortion correction.

5. A non-transitory computer program product comprising computer readable instructions, which when loaded and run in a computer processor, causes the computer processor of an image processing device to execute:
a motion vector calculation procedure for setting a motion vector of a block corresponding to a partial area of a first image based on the block and a similarity between the block and a partial area of a second image;
a motion vector reliability calculation procedure for calculating a reliability of the motion vector based on a distribution of the similarities;
a frame motion vector calculation procedure for calculating a frame motion

Assignees

• Olympus Corp

Inventors

• Ikeda Hideru
• Ueno Akira

Classifications

• H04N5/144 Primary

  Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20) · CPC title

• H04N23/6811

  based on the image signal · CPC title

• H04N23/81

  for suppressing or minimising disturbance in the image signal generation · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N5/217

  Electricity · mapped topic