Image processing apparatus and image processing method

Patent metadata

Field	Value
Publication number	US-2016247259-A1
Application number	US-201415029302-A
Country	US
Kind code	A1
Filing date	Jul 10, 2014
Priority date	Dec 3, 2013
Publication date	Aug 25, 2016
Grant date	—

Abstract

An image processing apparatus and image processing method enlarge an input image (Din) to generate a low-resolution enlarged image (D 101 ). Depending on a result of identification of the pattern of the input image (Din), coefficient data (D 108 ) for conversion to a high-resolution are selected, and a feature component (D 102 H) of a low resolution is converted to a feature component (D 103 H) of a high resolution. Decision as to whether or not the pattern of a local region of the input image (Din) is flat is made. If it is flat, the coefficient data are so selected that no substantial alteration is made to pixel values in a high-resolution conversion unit ( 103 ). It is possible to reduce the circuit size and the memory capacity, and to improve the noise immunity, and achieve conversion to a high resolution suitable for implementation by hardware.

First claim

1 . An image processing apparatus comprising:
an image enlarging unit for enlarging an input image to generate a low-resolution enlarged image; a feature component separating unit for separating said low-resolution enlarged image into a feature component of a low resolution and a non-feature component for each patch; a high-resolution conversion unit for converting said feature component of the low resolution into a feature component of a high resolution; a feature component combining unit for combining said feature component of the high resolution and said non-feature component to generate a high-resolution patch; a patch averaging unit for averaging, for each pixel, values of one or more high-resolution patches to generate an output image; a coefficient data storage unit for storing a plurality of sets of coefficient data respectively corresponding to a plurality of mutually different patterns; a pattern identification unit for identifying a pattern of a local region in said input image, corresponding to a patch in said low-resolution enlarged image, and outputting a pattern identification signal indicating a result of the identification; and a coefficient data selection unit for selecting, from among the plurality of sets of coefficient data stored in said coefficient data storage unit, a set of coefficient data corresponding to said pattern identification signal, and outputting the selected set of coefficient data; wherein said high-resolution conversion unit converts said feature component of the low resolution into said feature component of the high resolution, using said coefficient data selected by said coefficient data selection unit.

2 . The image processing apparatus as set forth in claim 1 , wherein
said pattern identification unit includes a flatness decision unit for deciding whether or not the pattern of said local region in said input image is flat, and said coefficient data selection unit performs the selection of said coefficient data based on a result of the decision by said flatness decision unit.

3 . The image processing apparatus as set forth in claim 2 , wherein said coefficient data selected when the result of the decision by said flatness decision unit indicates flatness are such coefficient data by which no substantial alteration is made to the pixel values in the conversion at said high-resolution conversion unit.

4 . The image processing apparatus as set forth in claim 1 , wherein
said pattern identification unit includes a flatness decision unit for deciding whether or not the pattern of the local region in said input image is flat; when the result of the decision by said flatness decision unit indicates non-flatness, said high-resolution conversion unit outputs, as said feature component of the high resolution, a result of the conversion of said feature component of the low-resolution using said coefficient data, and when the result of the decision by said flatness decision unit indicates flatness, said high-resolution conversion unit outputs, as said feature component of the high resolution, entirety or part of said feature component of the low resolution, as it is, without using said coefficient data.

5 . The image processing apparatus as set forth in claim 1 , wherein said pattern identification unit includes:
an average unit for taking, as a pixel of interest, a pixel at a position in said input image, corresponding to a center of the patch in said low-resolution enlarged image, and determining an average value of pixel values of pixels in the local region centered around said pixel of interest; an LBP calculation unit for comparing pixel values of pixels neighboring said pixel of interest, with said average value, and generating a pattern number representing a combination of results of the comparison pertaining to the respective pixels; a flatness decision unit for deciding whether or not said local region is flat, based on whether or not an average of absolute values of differences between the pixel values of the pixels neighboring said pixel of interest and said average value is smaller than a predetermined threshold value, and outputting a flatness decision signal indicating a result of the decision; and a pattern identification signal generating unit for generating the pattern identification signal from said pattern number and said flatness decision signal.

6 . The image processing apparatus as set forth in claim 1 , wherein said pattern identification unit includes:
an average unit for taking, as a pixel of interest, a pixel at a position in said input image, corresponding to a center of the patch in said low-resolution enlarged image, and determining an average value of pixel values of pixels in the local region centered around said pixel of interest; an LBP calculation unit for comparing pixel values of pixels neighboring said pixel of interest, with said average value, and generating a pattern number representing a combination of results of the comparison pertaining to the respective pixels, as said pattern identification signal; and a flatness decision unit for deciding whether or not said local region is flat, based on whether or not an average of absolute values of differences between the pixel values of the pixels neighboring said pixel of interest and said average value is smaller than a predetermined threshold value, and outputting a flatness decision signal indicating a result of the decision; wherein said high-resolution conversion unit receives said flatness decision signal, when said flatness decision signal indicates non-flatness, said high-resolution conversion unit outputs, as said feature component of the high-resolution, a result of the conversion from said feature component of the low resolution using said coefficient data, and when said flatness decision signal indicates flatness, said high-resolution conversion unit outputs, entirety or part of said feature component of the low resolution, as it is, without using said coefficient data.

7 . The image processing apparatus as set forth claim 1 , wherein said non-feature component separated by said feature component separating unit is a mean value of pixel values of pixels constituting said patch, and said feature component of the low-resolution separated by said feature component separating unit is formed of differences of the pixel values of the pixels constituting said patch, from said mean value.

8 . The image processing apparatus as set forth in claim 7 , wherein said feature component combining unit performs the combination by adding said non-feature component to said feature component.

9 . An image processing method comprising:
an image enlarging step for enlarging an input image to generate a low-resolution enlarged image; a feature component separating step for separating said low-resolution enlarged image into a feature component of a low resolution and a non-feature component for each patch; a high-resolution conversion step for converting said feature component of the low resolution into a feature component of a high resolution; a feature component combining step for combining said feature component of the high resolution and said non-feature component to generate a high-resolution patch; a patch averaging step for averaging, for each pixel, values of one or more high-resolution patches to generate an output image; a pattern identification step for identifying a pattern of a local region in said input image, corresponding to a patch in said low-resolution enlarged image, and outputting a pattern identification signal indicating a result of the identification; and a coefficient data selection step for selecting, from among a plurality of sets of coefficient data stored respectively corre

Assignees

• Mitsubishi Electric Corp

Inventors

• Toyoda Yoshitaka
• Kurihara Kohei
• Moriya Shotaro
• Fujita Takeo
• Matoba Narihiro

Classifications

• G06T3/40 Primary

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06T3/4053 Primary

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• H04N1/387

  Composing, repositioning or otherwise {geometrically} modifying originals · CPC title

• H04N1/3871

  the composed originals being of different kinds, e.g. low- and high-resolution originals · CPC title