Image processing method and system

Patent metadata

Field	Value
Publication number	US-12045948-B2
Application number	US-202117387508-A
Country	US
Kind code	B2
Filing date	Jul 28, 2021
Priority date	Jul 28, 2021
Publication date	Jul 23, 2024
Grant date	Jul 23, 2024

Abstract

An image processing method including obtaining image data. The image data includes a plurality of image data values. The image processing method also includes processing the image data, thereby generating output data. Processing the image data includes applying a convolution operation to the plurality of image data values using a kernel including a plurality of coefficients. Applying the convolution operation includes obtaining a sum of image data values of the plurality of image data values that correspond respectively to coefficients of the plurality of coefficients that each have a common coefficient value. Applying the convolution operation also includes multiplying the sum by the common coefficient value.

First claim

What is claimed is:

1. An image processing method, comprising:
obtaining image data, the image data comprising a plurality of image data values; and
processing the image data, thereby generating output data, wherein processing the image data comprises applying a convolution operation to the plurality of image data values using a kernel comprising a plurality of coefficients, wherein applying the convolution operation comprises:
obtaining a sum of image data values of the plurality of image data values that correspond respectively to coefficients of the plurality of coefficients that each have a common coefficient value; and
multiplying the sum by the common coefficient value,

wherein applying the convolution operation to the plurality of image data values forms at least part of a demosaicing algorithm within an image processing pipeline and wherein the image processing pipeline further comprises:
detecting a region of the image comprising an image data value with a measured error exceeding a threshold, wherein the measured error is associated with the demosaicing algorithm; and
processing a portion of the image data representing the region of the image to desaturate the region of the image.

2. The method of claim 1 , wherein the common coefficient value is a first common coefficient value, the sum is a first sum, further coefficients of the plurality of coefficients each have a second common coefficient value, different from the first common coefficient value, and applying the convolution operation comprises:
obtaining a second sum of image data values of the plurality of image data values that correspond respectively to the further coefficients; and
multiplying the second sum by the second common coefficient value.

3. The method of claim 1 , wherein the kernel is symmetric.

4. The method of claim 1 , wherein the convolution operation is a first convolution operation, and the method comprises:
applying a further convolution operation to at least some of the plurality of image data values using a further kernel, wherein the further kernel represents a bandpass filter for edge detection; and
combining an output of the first convolution operation with a further output of the further convolution operation, thereby generating the output data.

5. The method of claim 4 , wherein the plurality of image data values are associated with a plurality of pixel locations, respectively, each pixel location associated with one of a first color channel and a further color channel, different from the first color channel, the further convolution operation is applied to the at least some of the plurality of image data values, and the further output comprises equal contributions from pixel locations associated with the first color channel and pixel locations associated with the further color channel, independent of a pixel location associated with a respective image data value to which the further convolution operation is applied.

6. The method of claim 4 , wherein the at least some of the plurality of image data values comprises a first set of the plurality of image data values, and the method comprises, for at least a second set of the plurality of the image data values, different from the first set of the plurality of the image data values, determining to bypass the applying of the further convolution operation to at least the second set of the plurality of image data values.

7. The method of claim 6 , wherein the determining is based on an amount of information related to at least one color channel in the second set of the plurality of image data values.

8. The method of claim 7 , wherein the amount of information related to the at least one color channel in the second set of the plurality of image data values is less than an amount of information related to the at least one color channel in the first set of the plurality of image data values.

9. The method of claim 1 , wherein the image data comprises green color channel data, and the method comprises inputting the green color channel data to both the demosaicing algorithm and a computer vision system.

10. The method of claim 1 , wherein the convolution operation is applied to implement at least part of a neural network architecture.

11. The method of claim 1 , wherein the plurality of image data values is a first plurality of image data values associated with a corresponding first plurality of pixel locations associated with a first color channel, the image data further comprises a second plurality of image data values associated with a corresponding second plurality of pixel locations associated with a second color channel, different from the first color channel, the second plurality of pixel locations different from the first plurality of pixel locations, the convolution operation is a first convolution operation and the kernel is a first kernel, and the method comprises applying a second convolution operation to the second plurality of image data values using a second kernel.

12. The method of claim 11 , wherein the first convolution operation is applied to the first plurality of image data values, thereby generating a first output, the second convolution operation is applied to the second plurality of image data values, thereby generating a second output, and the method comprises obtaining, for a pixel location of the second plurality of pixel locations, an image data value for the first color channel, wherein obtaining the image data value comprises calculating, based on the pixel location, a combination of the first output and the second output, wherein the output data comprises the image data value.

13. The method of claim 12 , wherein the image data value is a first image data value and the combination is a first combination, wherein the image data further comprises:
a third plurality of image data values associated with a corresponding third plurality of pixel locations associated with a third color channel, different from the first color channel and the second color channel, the third plurality of pixel locations different from the first and second plurality of pixel locations; and
a fourth plurality of image data values associated with a corresponding fourth plurality of pixel locations associated with the second color channel, the fourth plurality of pixel locations, different from the first, second and third plurality of pixel locations,
wherein the method comprises:
applying a third convolution operation to the third plurality of image data values using a third kernel, thereby generating a third output;
applying a fourth convolution operation to the fourth plurality of image data values using a fourth kernel, thereby generating a fourth output;
calculating, based on the pixel location, a second combination of the first output, the second output, the third output and the fourth output, thereby generating, for the pixel location, a second image data value for the second color channel; and
calculating, based on the pixel location, a third combination of the first output, the second output, the third output and the fourth output, thereby generating, for the pixel location, a third image data value for the third color channel, wherein the output data comprises the second image data value and the third image data value.

14. An image processing system, comprising:
storage for storing a plurality of coefficients; and
at least one processor to:
obtain image data, the image data comprising a plurality of image data values; and
process the image data, thereby generating output data, wherein processing the image data comprises applying a convolution operation to the plurality of imag

Assignees

• Apical Ltd
• Advanced Risc Mach Ltd

Inventors

• Novikov Maxim
• Hanwell David
• Gomez Alexis Leonardo Lluis

Classifications

• G06T5/94

  based on local image properties, e.g. for local contrast enhancement · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/13

  Edge detection · CPC title

• G06T5/60

  using machine learning, e.g. neural networks · CPC title