Image sensor down-up sampling using a compressed guide

Patent metadata

Field	Value
Publication number	US-11399160-B2
Application number	US-202016888291-A
Country	US
Kind code	B2
Filing date	May 29, 2020
Priority date	May 29, 2020
Publication date	Jul 26, 2022
Grant date	Jul 26, 2022

Abstract

A method for down-up sampling of image sensor data includes the steps of down sampling green pixels in a super Bayer pattern of image sensor data, where a decimation factor of the downsampling corresponds to a size of a color cluster in the super Bayer pattern, where two green pixels that are those closest to a non-green pixel cluster remain in each green cluster after downsampling, and wherein each non-green pixel cluster is bordered by four downsampled green pixels, and up sampling non-green color pixels in a non-green cluster from the remaining green pixels that are closest to the non-green cluster by a bilinear interpolation of each non-green pixel with respect to the closest remaining green pixels, where the up sampling of the non-green color pixels is guided by a compressed array that corresponds to the image sensor data.

First claim

What is claimed is:

1. A method for down-up sampling of image sensor data, comprising the steps of:
down sampling green pixels in a super Bayer pattern of image sensor data, wherein a decimation factor of the downsampling corresponds to a size of a color cluster in the super Bayer pattern, wherein two green pixels that are those closest to a non-green pixel cluster remain in each green cluster after downsampling, and wherein each non-green pixel cluster is bordered by four downsampled green pixels; and
up sampling non-green color pixels in a non-green cluster from the remaining green pixels that are closest to the non-green cluster by a bilinear interpolation of each non-green pixel with respect to the closest remaining green pixels, wherein the up sampling of the non-green color pixels is guided by a compressed array that corresponds to the image sensor data.

2. The method of claim 1 ,
wherein the decimation factor for an n×n color pixel array is n 2 , and
wherein down sampling green pixels comprises keeping pixels at two diagonally opposites corners of the n×n color pixel array while flipping the diagonally opposite corners for alternate pixel array rows.

3. The method of claim 1 , wherein the compressed array is one of a pixel direction map or an edge map, wherein the pixel direction map specifies one of six directions for each pixel, wherein the six directions are horizontal, vertical, slash, backslash, flat, and unknown.

4. The method of claim 3 , wherein the pixel direction map is determined by splitting the image sensor data into red, green and blue clusters, wherein the method further comprises
within each cluster, determining if each pixel is flat by using a max-min measurement for each pixel and comparing the max-min measurement to a threshold, wherein if the max-min measurement for a pixel is below this threshold, the pixel is determined to be flat,
and if the pixel is determined to be non-flat, determining a direction for the pixel.

5. The method of claim 4 , wherein determining if each pixel is flat further comprises associating a patch of pixels about each pixel, wherein the pixel patch is larger than a size of the color pixel array, calculating the max-min for each color group of pixels within the pixel patch, and determining a pixel at a center of the pixel patch to be flat when all three max-min values are below the threshold.

6. The method of claim 5 , wherein determining a direction for the pixel comprises
calculating the difference between every pair of adjacent pixels with the same color in the patch for the horizontal, vertical, slash and backslash directions, summing the differences in the same direction, and selecting a summation with a lowest sum of differences as a direction for the pixel at the center of the patch,
wherein if a pixel direction's difference is not smaller than the other directions' differences within a predetermined interval, the pixel's direction is denoted as unknown.

7. The method of claim 3 , wherein up sampling non-green color pixels in a non-green cluster using the direction map comprises:
interpolating a flat or unknown direction pixel {circumflex over (p)} in a cluster being up sampled from the down sampled green pixels g 1 , g 2 , g 3 and g 4 according to

p
^

=

(

A
-

d
x

)

⁢

(

A
-

d
y

)

A
2

⁢

g
1

+

d
x

⁡

(

A
-

d
y

)

A
2

⁢

g
2

+

d
x

⁢

d
y

A
2

⁢

g
3

+

(

A
-

d
x

)

⁢

d
y

A
2

⁢

g
4

,

wherein A is an edge length of the cluster being up sampled and (d x , d y ) are the distances of pixel {circumflex over (p)} from green pixel g 1 ;
interpolating a horizontal, vertical, slash or backslash direction pixel {circumflex over (p)} in the cluster being up sampled according to

p
^

=

d
2

d
1

+

Assignees

• Samsung Electronics Co Ltd

Inventors

• Bernstein Tal
• Bitan Gal

Classifications

• G06T3/4007 Primary

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title

• H04N25/13 Primary

  characterised by the spectral characteristics of the filter elements · CPC title

• H04N25/447

  by preserving the colour pattern with or without loss of information · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• G06T3/4015

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title