Image processing apparatus and method for performing preprocessing to obtain image with improved sharpness

Patent metadata

Field	Value
Publication number	US-10007972-B2
Application number	US-201615331980-A
Country	US
Kind code	B2
Filing date	Oct 24, 2016
Priority date	Jun 10, 2016
Publication date	Jun 26, 2018
Grant date	Jun 26, 2018

Abstract

An image processing apparatus for transforming a wide-angle image into a narrow-angle image includes a deblurrer performing deblurring on the wide-angle image to generate a deblurred image; a super-resolution (SR) unit performing SR processing on the deblurred image to generate an upscaled transformed image; and a view transformer performing view transformation to obtain the narrow-angle image based on the wide-angle image and the upscaled transformed image.

First claim

What is claimed is:

1. An image processing apparatus for transforming a wide-angle image into a narrow-angle image, the image processing apparatus comprising:
a memory storing an application program; and
a processor configured to execute the application program stored in the memory, wherein execution of the application program causes the processor to:
perform deblurring on the wide-angle image to generate a deblurred image, and estimating whether an outer portion of the input wide-angle image has a Gaussian blur kernel based on blur image modeling;
perform super-resolution (SR) processing on the deblurred image to generate an upscaled transformed image; and
perform view transformation to obtain the narrow-angle image based on the wide-angle image and the upscaled transformed image,

wherein the processor performs the deblurring by:
performing a multi-scaling operation on the input wide-angle image to perform deblurring on a plurality of low-resolution images having different resolutions,
generating a downscaled low-resolution image by interpolating the plurality of deblurred images,
obtaining high frequency information from the downscaled low resolution image, and
synthesizing the obtained high frequency information with the input wide-angle image.

2. The image processing apparatus according to claim 1 , wherein the upscaled transformed image has sharpness enhanced through preprocessing, and the preprocessing includes the deblurring of the wide-angle image and the SR processing of the deblurred image, and the upscaled transformed image is used as an input to the view transformation to enhance performance of the view transformation to obtain the narrow-angle image.

3. The image processing apparatus according to claim 1 , wherein the processor performs deblurring by using self-similarity to downscale the wide-angle image, by searching for high frequency information, and by synthesizing the high frequency information with the wide-angle image to thereby output the deblurred image with improved sharpness.

4. The image processing apparatus according to claim 1 , wherein the processor performs SR processing by applying a filter appropriate for a corresponding image according to an image pattern based on an SR scheme using a local binary pattern (LBP)-based classifier.

5. The image processing apparatus according to claim 1 , wherein the processor performs view transformation by estimating pixel values present at non-integer coordinate positions using adjacent pixel values present in integer positions of a wide-angle image referred to from a look-up table (LUT) on the basis of an LUT-based view transformation interpolation scheme.

6. The image processing apparatus according to claim 1 , wherein the processor performs deblurring by:
generating the downscaled low resolution image using linear interpolation with respect to the input wide-angle image;
applying a first Gaussian low pass filter (LPF) to the downscaled low resolution image;
performing a second Gaussian LPF upscaling and low-pass filtering to the downscaled low resolution image;
subtracting an output from the first Gaussian LPF from the downscaled low resolution image to generate a high frequency (HF) component image;
searching and outputting a high frequency component of the output of the subtracting that corresponds to an output image from the second Gaussian LPF using matching of an output image from the first Gaussian LPF and the output image from the second Gaussian LPF; and
synthesizing the output image from the second Gaussian LPF and the high frequency component from the block matching unit to output a deblurred image.

7. The image processing apparatus according to claim 1 , wherein the processor deblurs the plurality of low resolution images obtained by the multi-scaling of the wide-angle image and obtains a first sharp downscaled image through interpolation on the deblurred images, and generates the deblurred image through interpolation by using the wide-angle image and the first downscaled image.

8. The image processing apparatus according to claim 1 , wherein the processor deblurs an edge portion excluding a flat region and a texture region of the wide-angle image.

9. The image processing apparatus according to claim 1 , wherein the processor performing SR processing obtains the upscaled transformed image using a two-dimensional finite impulse response (2D-FIR) filter.

10. The image processing apparatus according to claim 1 , wherein the processor performs view transformation using a Lanczos filter.

11. An image processing method for transforming a wide-angle image into a narrow-angle image, the method comprising:
deblurring the wide-angle image to generate a deblurred image and estimating whether an outer portion of the input wide-angle image has a Gaussian blur kernel based on blur image modeling;
performing super-resolution (SR) processing on the deblurred image to generate an upscaled transformed image; and
performing view transformation to obtain the narrow-angle image based on the wide-angle image and the upscaled transformed image,
wherein the performing the deblurring includes:
performing a multi-scaling operation on the input wide-angle image to perform deblurrering on a plurality of low-resolution images having different resolutions,
generating a downscaled low-resolution image by interpolating the plurality of deblurred images,
obtaining high frequency information from the downscaled low resolution image, and
synthesizing the obtained high frequency information with the input wide-angle image.

12. The method according to claim 11 , wherein the upscaled transformed image has sharpness enhanced through preprocessing, and the preprocessing includes the deblurring of the wide-angle image and the SR processing of the deblurred image, and the upscaled transformed image is used as an input to the view transformation to enhance performance of the view transformation to obtain the narrow-angle image.

13. The method according to claim 11 , wherein, in the deblurring, a deblur scheme using self-similarity is performed to downscale the wide-angle image and high frequency information is searched and synthesized with the input wide-angle image to thereby output the unblurred image.

14. The method according to claim 11 , wherein, in the generating the upscaled transformed image, the upscaled transformed image is obtained by applying a filter appropriate for a corresponding image according to an image pattern based on an SR scheme using an active local binary pattern (LBP)-based classifier.

15. The method according to claim 11 , wherein, in the performing the view transformation, pixel values present at non-integer coordinate positions are estimated using adjacent pixel values present in integer positions of a wide-angle image referred to from a look-up table (LUT) on the basis of an LUT-based view transformation interpolation scheme.

16. The method according to claim 11 , wherein the deblurring includes:
generating the downscaled low resolution image using linear interpolation with respect to the wide-angle image;
low-pass-filtering, by a first Gaussian low-pass filter (LPF), the downscaled low resolution image;
performing upscaling and low-pass filtering, by a second Gaussian LPF, on the downscaled low resolution image;
subtracting, by a first calculator, an output from the first Gaussian LPF from the downscaled low resolution image;
searching and outputting, by a block matching unit, a high frequency component of the first calculator corresponding to an output image from the second Gaussian LPF using matc

Assignees

• Hyundai Motor Co Ltd
• Kia Motors Corp
• Univ Inha Res & Business Found

Inventors

• Choi Jin Wook
• Song Byung Cheol
• Choi Dong Yoon
• Choi Ji Hoon

Classifications

• G06T3/4053

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• G06T5/003 Primary

  Physics · mapped topic

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T3/0062

  Physics · mapped topic

• G06T5/20

  using local operators · CPC title