Learning based bad pixel correction

Patent metadata

Field	Value
Publication number	US-12482065-B2
Application number	US-202217657171-A
Country	US
Kind code	B2
Filing date	Mar 30, 2022
Priority date	Mar 30, 2022
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

Methods and apparatuses for correcting bad image pixels are described. The described sensor-independent image processing techniques leverage one or more dynamic dictionaries of learned filters for bad pixel correction (e.g., where a camera leverages such dictionaries to efficiently identify filters to accurately adjust and correct bad pixel values). For example, a dictionary may store filters that are learned offline (via a self-supervised learning algorithm implemented at a server using known images and ground truth bad pixel correction values). To select a filter for a bad pixel correction operation, a camera may encode an image patch surrounding a bad pixel (into an encoded patch descriptor) and search the dictionary for a matching patch descriptor key. The camera may then apply the filter (value) corresponding to the searched patch descriptor (key) of the dictionary to the image patch to correct the bad pixel and generate a corrected output image.

First claim

What is claimed is:

1 . A method for image processing, comprising:
receiving an image including a bad pixel; identifying a patch of pixels surrounding the bad pixel; generating a patch descriptor corresponding to the patch; selecting a patch descriptor key corresponding to the patch descriptor from a dictionary including pairs of patch descriptor keys and filters; comparing the patch descriptor to each of a plurality of patch descriptor keys in the dictionary; obtaining a similarity value for each of the plurality of patch descriptor keys in the dictionary based on a result of the comparison, wherein the similarity value comprises a distance between the patch descriptor and a corresponding patch descriptor key; identifying the selected patch descriptor key based on the similarity value; identifying a filter paired with the identified patch descriptor key among the pairs of patch descriptor keys and filters in the dictionary; and correcting the bad pixel by applying the identified filter to the patch.

2 . The method of claim 1 , wherein the patch descriptor is obtained by encoding pixel values of the patch.

3 . The method of claim 1 , further comprising:
generating a training patch descriptor for each of a plurality of patches in the set of training images; clustering the plurality of patches based on the training patch descriptor to obtain a plurality of patch clusters; and identifying a plurality of patch descriptor keys in the dictionary based on the plurality of patch clusters.

4 . The method of claim 3 , wherein:
the selected patch descriptor key comprises a representative image patch or a descriptor of the representative image patch for a corresponding cluster of the plurality of patch clusters.

5 . The method of claim 3 , further comprising:
performing a self-supervised learning algorithm on a cluster corresponding to the selected patch descriptor key to obtain the filter.

6 . The method of claim 3 , further comprising:
obtaining a predicted pixel each of the one or more patches by applying a filter to one or more patches in a corresponding cluster from the of patch clusters; comparing the predicted pixel to an actual pixel in a corresponding patch of the one or more patches; and updating the filter based on the comparison.

7 . The method of claim 1 , further comprising:
identifying a plurality of bad pixels in the image; and generating a corrected image by correcting each of the plurality of bad pixels.

8 . The method of claim 7 , further comprising:
capturing the image with a camera; and presenting the corrected image to a user.

9 . An apparatus for image processing, comprising:
a patch selection component configured to select a patch surrounding a bad pixel in one image; a dictionary component configured to: compare a patch descriptor to each of a plurality of patch descriptor keys in a dictionary learned through a set of images each different from the one image, obtain a similarity value for each of the plurality of patch descriptor keys in the dictionary based on a result of the comparison, identify a patch descriptor key based on the similarity value, and identify a filter corresponding to the patch descriptor key based on the patch from the dictionary, wherein the similarity value comprises a distance between the patch descriptor and a corresponding patch descriptor key; and a patch correction component configured to correct the bad pixel by applying the filter to the patch.

10 . The apparatus of claim 9 , further comprising:
a camera configured to capture the one image.

11 . The apparatus of claim 9 , further comprising:
a descriptor component configured to generate the patch descriptor for the patch, wherein the patch descriptor key is selected based on the descriptor.

12 . The apparatus of claim 9 , further comprising:
a clustering component configured to cluster a plurality of patches to obtain a plurality of patch clusters, wherein the patch descriptor key corresponds to one of the plurality of patch clusters.

13 . The apparatus of claim 9 , further comprising:
a filter training component configured to perform a self-supervised learning algorithm on a cluster of patches corresponding to the patch descriptor key to obtain the filter.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Avishai Igal
• Bitan Gal
• Yam Roy

Classifications

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• G06V10/7747

  Organisation of the process, e.g. bagging or boosting · CPC title

• G06V10/22

  by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition · CPC title

• G06T9/00 Primary

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title

• G06V10/762

  using clustering, e.g. of similar faces in social networks · CPC title