Method and apparatus for sensor characterization

Patent metadata

Field	Value
Publication number	US-9544516-B2
Application number	US-64338609-A
Country	US
Kind code	B2
Filing date	Dec 21, 2009
Priority date	Dec 26, 2008
Publication date	Jan 10, 2017
Grant date	Jan 10, 2017

Abstract

The detailed technology relates to optical sensors, and more particularly relates to characterizing local behaviors of a sensor, and using such characterization information in a sensor-based system. In a particular arrangement, output signals produced from each of plural sensing elements in a 2D optical sensor are checked to determine whether they tend to differ from output signals produced by one or more neighboring sensing elements—when combined across plural captured image frames. The results are stored in an associated memory, and can be consulted in determining how image data captured from the sensor should be used. The technology is particularly illustrated in the context of a watermark decoder for a cell phone camera. A variety of other features and arrangements are also detailed.

First claim

I claim:

1. A method employing an image sensor that is manufactured by a producer, and subsequently incorporated into an end-product employed by a user, the image sensor comprising plural pixels that each produces a pixel value corresponding to an intensity of incident illumination, in which the pixels should produce identical pixel values given identical incident illumination, the method comprising the acts:
processing plural image frames captured by the sensor, when used in said end-product by the user, to dynamically characterize the sensor performance as a background operation, said processing including:
identifying a first neighborhood grouping of sensor pixels, the first neighborhood grouping consisting of a first target pixel and a first set of one or more comparison pixels, the first neighborhood grouping being a subset of the sensor pixels, wherein certain sensor pixels are not included within the first neighborhood grouping;
comparing an average pixel value of the first target pixel, computed across said plural frames, with an average pixel value of the first set of comparison pixels, computed across said plural frames to determine whether the first target pixel produces output signals that are larger, on average, than output signals from the first set of comparison pixels;
identifying a second neighborhood grouping of sensor pixels, the second neighborhood grouping consisting of a second target pixel and a second set of one or more comparison pixels, the second neighborhood grouping being a subset of the sensor pixels, wherein certain sensor pixels are not included within the second neighborhood grouping;
comparing an average pixel value of the second target pixel, computed across said plural frames, with an average pixel value of the second set of comparison pixels, computed across said plural frames to determine whether the second target pixel produces output signals that are larger, on average, than output signals from the second set of comparison pixels;
the second set of comparison pixels being different than the first set of comparison pixels;
storing error data for at least one of the first and second target pixels, based on results of said comparing acts;
decoding a steganographically-encoded digital watermark signal from a further frame of captured imagery, said decoding employing the stored error data;
wherein the method adapts through use of the end-product, discerning that at least one of said target pixels that outputs pixel values actually outputs biased pixel values, and employs the stored error data to achieve improved watermark decoding results.

2. The method of claim 1 that further includes compensating at least one value in the further frame of imagery prior to decoding of the digital watermark signal, based on the stored error data.

3. The method of claim 1 in which said decoding act comprises decoding said digital watermark signal from the further frame of imagery, wherein the decoding includes disregarding pixel data from at least one of the first and second target pixels, based on the stored error data.

4. The method of claim 1 in which the decoding act includes weighting votes cast by one or more of said sensor pixels in accordance with the stored error data.

5. The method of claim 1 in which:
the stored error data comprises first data indicating that the first target pixel outputs pixel values larger, on average, than the average pixel values of the first neighborhood; and
the stored error data comprises second data indicating that the second target pixel outputs pixel values smaller, on average, than the average pixel values of the second neighborhood.

6. The method of claim 1 in which the end-product is a smartphone, and the processing is performed by a processor in the smartphone.

7. The method of claim 1 in which the sensor includes N rows by M columns of pixels, and the method includes determining whether the pixel value for each of the N*M pixels of said sensor is (a) larger or (b) smaller, on average across the plural captured frames of image data, than an average pixel value from a corresponding set of one or more comparison pixels, wherein there is a different set of comparison pixels respectively corresponding to each of the N*M pixels.

8. The method of claim 1 in which said processing plural image frames captured by the sensor comprises processing at least 100 image frames captured by the sensor.

9. A smartphone comprising an image sensor, a processor and a memory, the image sensor comprising plural pixels that each produces a pixel value corresponding to an intensity of incident illumination, in which the pixels should produce identical pixel values given identical incident illumination, the memory containing instructions configuring the processor to perform the following acts:
processing plural image frames captured by the sensor to dynamically characterize the sensor performance, as a background operation during the smartphone's normal operation, said processing including:
identifying a first neighborhood grouping of sensor pixels, the first neighborhood grouping consisting of a first target pixel and a first set of one or more comparison pixels, the first neighborhood grouping being a subset of the sensor pixels, wherein certain sensor pixels are not included within the first neighborhood grouping;
comparing an average pixel value of the first target pixel, computed across said plural frames, with an average pixel value of the first set of comparison pixels, computed across said plural frames to determine whether the first target pixel produces output signals that are larger, on average, than output signals from the first set of comparison pixels;
identifying a second neighborhood grouping of sensor pixels, the second neighborhood grouping consisting of a second target pixel and a second set of one or more comparison pixels, the second neighborhood grouping being a subset of the sensor pixels, wherein certain sensor pixels are not included within the second neighborhood grouping;
comparing an average pixel value of the second target pixel, computed across said plural frames, with an average pixel value of the second set of comparison pixels, computed across said plural frames to determine whether the second target pixel produces output signals that are larger, on average, than output signals from the second set of comparison pixels;
the second set of comparison pixels being different than the first set of comparison pixels;
storing error data for at least one of the first and second target pixels, based on results of said comparing acts;
decoding a steganographically-encoded digital watermark signal from a further frame of captured imagery, employing the stored error data to achieve improved watermark decoding results;
wherein the smartphone adapts through use, discerning that at least one of said target pixels that outputs pixel values actually outputs biased pixel values, and employs the stored error data to achieve improved watermark decoding results.

10. A non-transitory computer readable medium containing software instructions operative to configure a device programmed thereby to perform the following operations:
processing plural image frames captured by a sensor in the device to dynamically characterize the sensor performance, as a background operation during the device's normal operation, the sensor comprising plural pixels that each produces a pixel value corresponding to an intensity of illumination, wherein the pixels should all produce identical pixel values given identical incident illumination, said processing including:
identifying a first neighborhood grouping of sensor pixels, the first neighborhood grouping consisting of a first ta

Assignees

• Rhoads Geoffrey B
• Digimarc Corp

Inventors

• Rhoads Geoffrey B

Classifications

• H04N1/00002 Primary

  Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for (error detection, error correction or monitoring in digital computers or digital computer components G06F11/00) · CPC title

• H04N1/00076

  locally · CPC title

• H04N1/00068

  Calculating or estimating · CPC title

• H04N1/0005

  in service, i.e. during normal operation · CPC title

• H04N1/00013

  Reading apparatus · CPC title