Edge guided interpolation and sharpening

Patent metadata

Field	Value
Publication number	US-9875556-B2
Application number	US-201615235040-A
Country	US
Kind code	B2
Filing date	Aug 11, 2016
Priority date	Aug 17, 2015
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

Techniques, methods, and systems for image processing may be provided. The image processing may be provided for upsampling and interpolating images. The upsampling and interpolating may include interpolating the image through at least an edge weight and a spatial weight. In various embodiments, the edge weight and/or the spatial weight may be calculated with a kernel. The kernel may be a kernel with a two dimensional (2D) distribution such as a Gaussian kernel, a Laplacian kernel, or another such statistically based kernel. The image processing may also include refining the upsampled and interpolated image through a refinement weight calculation and/or through back projection.

First claim

What is claimed is:

1. A method comprising:
receiving an image, the image comprising a plurality of pixels with each pixel including a pixel value;
selecting a pixel to be processed;
determining an edge weight and a spatial weight associated with the selected pixel, wherein at least the spatial weight is determined with a spatial kernel; and
processing the image with at least the edge weight and the spatial weight.

2. The method of claim 1 , wherein the spatial kernel is a Gaussian based kernel and/or is based on a two dimensional (2D) distribution.

3. The method of claim 1 , wherein the selected pixel is an upsampled pixel, the upsampled pixel is the location of a pixel in the upsampled image, the processing the image comprises interpolating the image with at least the edge weight and the spatial weight and the method further comprises upsampling the image.

4. The method of claim 3 , wherein the determining the edge weight comprises:
applying an edge weight kernel to a neighborhood set of the pixels to determine weighted pixel edge values for the pixels of the neighborhood set;
determining an edge indicator value using at least the weighted pixel edge values; and
determining the edge weight using at least the edge indicator value.

5. The method of claim 3 , wherein the spatial kernel comprises an N×N distribution, where N is an integer equal to or greater than 3 and the determining the spatial weight comprises:
determining a distance from the upsampled pixel to at least one pixel within a neighborhood set of the pixels, wherein the at least one pixel is a central pixel of the neighborhood set, and wherein the central pixel is a pixel of the neighborhood set closest to the upsampled pixel;
shifting the spatial kernel according to the distance; and
applying the spatial kernel to a plurality of pixels within the neighborhood set to determine the spatial weight.

6. The method of claim 3 , further comprising:
applying a point spread function (PSF) to the interpolated and upsampled image to create a PSF processed image;
downsampling the PSF processed image;
determining a differenced image by differencing the PSF processed image with the received original image;
upsampling the differenced image;
applying an inverse of the PSF to the upsampled differenced image to create a convolved image; and
combining the convolved image with the interpolated and upsampled image.

7. The method of claim 1 , further comprising:
determining pixel values of at least two pixels located on either side of the selected pixel within a portion of the image, wherein the portion includes the selected pixel;
determining a distance between the selected pixel and at least one of the two pixels;
determining a refinement weight using, at least, the pixel values of the at least two pixels and the distance between the selected pixel and the at least one of the two pixels; and
further processing the processed image with at least the refinement weight.

8. The method of claim 1 , further comprising:
applying a mask to at least a portion of the image; and
determining, after applying the mask, an edge within the image.

9. The method of claim 1 , wherein:
the processing the image comprises sharpening the image with at least the edge weight and the spatial weight;
the determining an edge weight comprises:
applying an edge weight kernel to a neighborhood set of the pixels of the image to determine weighted pixel edge values for the pixels of the neighborhood set,
determining an edge indicator value using at least the weighted pixel edge values, and
determining the edge weight using at least the edge indicator value; and

the image includes a neighborhood set of the pixels, the neighborhood set includes a plurality of pixels including the selected pixel, and determining the spatial weight comprises:
determining a distance from the selected pixel to at least one pixel within the neighborhood set,
shifting the spatial kernel according to the distance, and
applying the spatial kernel to a plurality of pixels within the neighborhood set to determine the spatial weight.

10. The method of claim 1 , wherein the selected pixel is a first pixel, the method further comprising:
selecting a second pixel to be processed;
determining an edge weight and a spatial weight associated with the second selected pixel, wherein at least the spatial weight is determined with a spatial kernel; and
processing the image with at least the edge weight and the spatial weight associated with the second selected pixel.

11. A system comprising:
a memory component adapted to receive an image associated with a scene, wherein the image comprises a plurality of pixels with each pixel including a pixel value; and
a processing component configured to execute the instructions to:
receive the image;
select a pixel to be processed;
determine an edge weight and a spatial weight associated with the selected pixel, wherein at least the spatial weight is determined with a spatial kernel; and
process the image with at least the edge weight and the spatial weight.

12. The system of claim 11 , wherein the spatial kernel is a Gaussian based kernel and/or is based on a two dimensional (2D) distribution.

13. The system of claim 11 , wherein the selected pixel is an upsampled pixel, the upsampled pixel is the location of a pixel in the upsampled image, the processing the image comprises interpolating the image with, at least, the edge weight and the spatial weight, and the processing component is further configured to execute instructions to upsample the image.

14. The system of claim 13 , wherein the determining the edge weight comprises:
applying an edge weight kernel to a neighborhood set of the pixels to determine weighted pixel edge values for the pixels of the neighborhood set;
determining an edge indicator value using at least the weighted pixel edge values; and
determining the edge weight using at least the edge indicator value.

15. The system of claim 13 , wherein the spatial kernel comprises an N×N distribution, where N is an integer equal to or greater than 3 and the determining the spatial weight comprises:
determining a distance from the upsampled pixel to at least one pixel within a neighborhood set of the pixels, wherein the at least one pixel is a central pixel of the neighborhood set, and wherein the central pixel is a pixel of the neighborhood set closest to the upsampled pixel;
shifting the spatial kernel according to the distance; and
applying the spatial kernel to a plurality of pixels within the neighborhood set to determine the spatial weight.

16. The system of claim 13 , wherein the processing component is further configured to execute instructions to:
apply a point spread function (PSF) to the interpolated and upsampled image to create a PSF processed image;
downsample the PSF processed image;
determine a differenced image by differencing the PSF processed image with the received original image;
upsample the differenced image;
apply an inverse of the PSF to the upsampled differenced image to create a convolved image; and
combine the convolved image with the interpolated and upsampled image.

17. The system of claim 11 , wherein the processing component is further configured to execute instructions to:
determine pixel values of at least two pixels located on either side of the selected pixel within a portion of the image, wherein the portion includes the selected pixel;
determine a distance between the selected pixel and at least one of the two pixels;
determine a

Assignees

• Flir Systems

Inventors

• Mcnally Scott

Classifications

• G06T3/403 Primary

  Edge-driven scaling; Edge-based scaling · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• G06T7/60 Primary

  Analysis of geometric attributes · CPC title

• H04N5/2175

  Electricity · mapped topic

• H04N5/2254

  Electricity · mapped topic