Circuit for combined down sampling and correction of image data

Patent metadata

Field	Value
Publication number	US-12367545-B2
Application number	US-202318230560-A
Country	US
Kind code	B2
Filing date	Aug 4, 2023
Priority date	Jan 7, 2021
Publication date	Jul 22, 2025
Grant date	Jul 22, 2025

Abstract

A foveated down sampling and correction (FDS-C) circuit for combined down sampling and correction of chromatic aberrations in images. The FDS-C circuit performs down sampling and interpolation of pixel values of a first subset of pixels of a color in a raw image using down sampling scale factors and first interpolation coefficients to generate first corrected pixel values for pixels of the color in a first corrected version of the raw image. The FDS-C circuit further performs interpolation of pixel values of a second subset of the pixels in the first corrected version using second interpolation coefficients to generate second corrected pixel values for pixels of the color in a second corrected version of the raw image. Pixels in the first subset are arranged in a first direction, pixels in the second subset are arranged in a second direction, and the down sampling scale factors vary along the first direction.

First claim

What is claimed is:

1. An image processor, comprising:
a first circuit configured to:
down sample and interpolate pixel values of a first subset of pixels of a first color but not of a second color of a plurality of colors in a raw image, based on first down sampling scale factors for the first color but not for the second color and first interpolation coefficients for the first color but not for the second color; and
generate first corrected pixel values for the pixels of the first color but not of the second color of a first corrected version of the raw image, based on the down-sampled and interpolated pixel values of the first subset of pixels of the first color but not of the second color; and

a second circuit configured to:
interpolate pixel values of a second subset of the pixels of the first color but not of the second color in the first corrected version, based on second interpolation coefficients for the first color but not for the second color; and
generate second corrected pixel values for the pixels of the first color but not of the second color of a second corrected version of the raw image, based on the interpolated pixel values of the second subset of the pixels of the first color but not of the second color.

2. The image processor of claim 1 , wherein the first subset of the pixels of the first color are arranged in a first direction, and wherein the second subset of the pixels of the first color are arranged in a second direction.

3. The image processor of claim 2 , wherein the first direction is perpendicular to the second direction.

4. The image processor of claim 1 , wherein, to generate the first corrected pixel values, the first circuit is configured to:
down sample and interpolate a number of the pixels of the first color in a first column of the raw image, based on a corresponding one of the first down sampling scale factors for the first color and a corresponding subset of the first interpolation coefficients for the first color.

5. The image processor of claim 1 , wherein the second circuit is further configured to:
interpolate a number of the pixels of the first color in a first row of the first corrected version, based on a corresponding subset of the second interpolation coefficients for the first color; and
generate at least one of the second corrected pixel values for a pixel of the first color in the second corrected version.

6. The image processor of claim 1 , wherein the first subset of the pixels of the first color are in a first column of the raw image and are arranged in accordance with a Bayer pattern, and wherein the second subset of the pixels of the first color are in a first row of the first corrected version of the raw image and are arranged in accordance with the Bayer pattern.

7. The image processor of claim 1 , wherein a value of each of the first down sampling scale factors for the first color is based on locations of a number of the pixels of the first color in a first column of the raw image.

8. A method, comprising:
down sampling and interpolating pixel values of a first subset of pixels of a first color but not of a second color of a plurality of colors in a raw image, based on first down sampling scale factors for the first color but not for the second color and first interpolation coefficients for the first color but not for the second color;
generating first corrected pixel values for the pixels of the first color but not of the second color of a first corrected version of the raw image, based on the down-sampled and interpolated pixel values of the first subset of pixels of the first color but not of the second color;
interpolating pixel values of a second subset of the pixels of the first color but not of the second color in the first corrected version, based on second interpolation coefficients for the first color but not for the second color; and
generating second corrected pixel values for the pixels of the first color but not of the second color of a second corrected version of the raw image, based on the interpolated pixel values of the second subset of the pixels of the first color but not of the second color.

9. The method of claim 8 , further comprising:
arranging the first subset of the pixels of the first color in a first direction; and
arranging the second subset of the pixels of the first color in a second direction.

10. The method of claim 9 , wherein the first direction is perpendicular to the second direction.

11. The method of claim 8 , wherein generating the first corrected pixel values comprises down sampling and interpolating a number of the pixels of the first color in a first column of the raw image, based on a corresponding one of the first down sampling scale factors for the first color and a corresponding subset of the first interpolation coefficients for the first color.

12. The method of claim 8 , further comprising:
interpolating a number of the pixels of the first color in a first row of the first corrected version, based on a corresponding subset of the second interpolation coefficients for the first color; and
generating at least one of the second corrected pixel values for a pixel of the first color in the second corrected version.

13. The method of claim 8 , further comprising:
arranging the first subset of the pixels of the first color but not of the second color in a first column of the raw image, in accordance with a Bayer pattern; and
arranging the second subset of the pixels of the first color but not of the second color in a first row of the first corrected version of the raw image, in accordance with the Bayer pattern.

14. The method of claim 8 , wherein a value of each of the first down sampling scale factors for the first color is based on locations of a number of the pixels of the first color in a first column of the raw image.

15. A system, comprising:
at least one image sensor configured to capture a raw image; and
an image processor coupled to the at least one image sensor, the image sensor comprising:
a first circuit configured to:
down sample and interpolate pixel values of a first subset of pixels of a first color but not of a second color of a plurality of colors in the raw image, based on first down sampling scale factors for the first color but not for the second color and first interpolation coefficients for the first color but not for the second color; and
generate first corrected pixel values for the pixels of the first color but not of the second color of a first corrected version of the raw image, based on the down-sampled and interpolated pixel values of the first subset of pixels of the first color but not of the second color; and

a second circuit configured to:
interpolate pixel values of a second subset of the pixels of the first color but not of the second color in the first corrected version, based on second interpolation coefficients for the first color but not for the second color; and
generate second corrected pixel values for the pixels of the first color but not of the second color of a second corrected version of the raw image, based on the interpolated pixel values of the second subset of the pixels of the first color but not of the second color.

16. The system of claim 15 , wherein the first subset of the pixels of the first color are arranged in a first direction, and wherein the second subset of the pixels of the first color are arranged in a second direction.

17. The system of claim 16 , wherein the first direction is perpendicular to the second direction.

18. The system of claim 15 , wherein,

Assignees

• Apple Inc

Inventors

• Wu Chihsin
• Pope David R
• Lin Sheng
• Silverstein Amnon D

Classifications

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• G06T3/4015 Primary

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title

• G06T3/4007 Primary

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title