Focal compensation for thermal expansion of camera optics using chromatic aberration

Patent metadata

Field	Value
Publication number	US-9876996-B2
Application number	US-201615348972-A
Country	US
Kind code	B2
Filing date	Nov 10, 2016
Priority date	Dec 30, 2015
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

An image processor adjusts luminance values of pixels in a captured image to compensate for effects on focal quality due to thermal expansion or contraction. Responsivity values for different colors are characterized and a table is generated mapping temperature values to weights for each pixel color based on the relationship between temperature and focal length and the relationship between focal length and focal quality in each different color. Luminance values for one color may be compensated based on the measured luminance values for other colors, the relative responsivity values, and the weights.

First claim

The invention claimed is:

1. A method for temperature-based focal compensation in a camera having an image sensor comprising a plurality of pixels to sense light of a first color, a second color, and a third color, the method comprising:
receiving for a pixel, a first responsivity value for the first color, a second responsivity value for the second color, and a third responsivity value for the third color, the responsivity values representing respective luminance values generated by the image sensor in response to white light when operating at a baseline temperature;
receiving a sensed temperature;
capturing for the pixel of the image sensor, a first luminance value for the first color, a second luminance value for a second color, and a third luminance value for the third color;
determining, based on the sensed temperature, a first weight for the first luminance value, a second weight for the second luminance value, and a third weight for the third luminance value, the first, second, and third weights varying with focal qualities of each respective color when operating at the sensed temperature;
normalizing the second luminance value based on a ratio of the first responsivity value to the second responsivity value to generate a normalized second color value;
normalizing the third luminance value based on a ratio of the first responsivity value to the third responsivity value to generate a normalized third luminance value;
generating, by a processor, a compensated first luminance value as a combination of the first luminance value weighted by the first weight, the normalized second luminance value weighted by the second weight, and the normalized third luminance value weighted by the third weight.

2. The method of claim 1 wherein a highest weight of the first weight, the second weight, and the third weight is applied to one of the first luminance value, the normalized second luminance value, and the normalized third luminance value that is associated with a best focused color of the first color, the second color, and the third color at the sensed temperature.

3. The method of claim 1 , wherein the sum of the first weight, the second weight and the third weight is equal to 1.

4. The method of claim 1 , wherein determining the first weight, the second weight, and the third weight comprises:
performing a lookup in a look up table using the sensed temperature to determine the first, second, and third weights.

5. The method of claim 4 , wherein the look up table includes predetermined weights for a plurality of temperatures, the predetermined weights derived from a relationship between spot size and focal distance to an image sensor.

6. The method of claim 5 , wherein the relationship between the spot size and the focal distance is different for different temperatures of the plurality of temperatures.

7. A computer-implemented method for characterizing a fixed-focal length camera for thermal-based compensation, the camera having an image sensor to sense light of a first color, a second color, and a third color, the method comprising:
calibrating a lens assembly of the camera to have a baseline focal plane distance with respect to an image sensor when operating at a baseline temperature;
characterizing for each pixel of the image sensor, a first responsivity value for the first color, a second responsivity for the second color, and a third responsivity, the responsivity values representing respective color values generated by the image sensor in response to white light when operating at a baseline temperature;
characterizing a focal plane distance transfer function of the camera representing a change in focal plane distance of the camera from the baseline focal plane distance with respect to a change in temperature from the baseline temperature;
characterizing a set of focal quality functions representing respective changes in focal quality of the first color, the second color, and the third color with respect to the change in focal plane distance;
generating based on the focal plane distance transfer function and the set of focal quality functions, a lookup table indicating for a plurality of different temperatures, a first weight for the first color value, a second weight for the second color value, and a third weight for the third color value, the first, second, and third weights varying with the focal quality of a respective color when operating at respective temperatures; and
storing the lookup table and the sensor responsivities to a storage medium.

8. The method of claim 7 , wherein determining the first weight, the second weight, and the third weight comprises:
determining a best focused color of the first, second, and third color for a change in focal plane distance when operating the image sensor at a given temperature;
determining a baseline weight associated with the best focused color; and
determining secondary weights associated with remaining colors that do not correspond to the best focused color, the secondary weights lower than the baseline weight.

9. The method of claim 7 , wherein the sum of the first weight, the second weight and the third weight is equal to 1.

10. The method of claim 7 , wherein the focal quality is based on a spot size measure.

11. A non-transitory computer-readable storage medium storing instructions for temperature-based focal compensation in a camera having an image sensor comprising a plurality of pixels to sense light of a first color, a second color, and a third color, the instructions when executed by a processor causing the processor to perform steps comprising:
receiving for a pixel, a first responsivity value for the first color, a second responsivity value for the second color, and a third responsivity value for the third color, the responsivity values representing respective luminance values generated by the image sensor in response to white light when operating at a baseline temperature;
receiving a sensed temperature;
capturing for the pixel of the image sensor, a first luminance value for the first color, a second luminance value for a second color, and a third luminance value for the third color;
determining, based on the sensed temperature, a first weight for the first luminance value, a second weight for the second luminance value, and a third weight for the third luminance value, the first, second, and third weights varying with focal qualities of each respective color when operating at the sensed temperature;
normalizing the second luminance value based on a ratio of the first responsivity value to the second responsivity value to generate a normalized second color value;
normalizing the third luminance value based on a ratio of the first responsivity value to the third responsivity value to generate a normalized third luminance value;
generating a compensated first luminance value as a combination of the first luminance value weighted by the first weight, the normalized second luminance value weighted by the second weight, and the normalized third luminance value weighted by the third weight.

12. The non-transitory computer-readable storage medium of claim 11 wherein a highest weight of the first weight, the second weight, and the third weight is applied to one of the first luminance value, the normalized second luminance value, and the normalized third luminance value that is associated with a best focused color of the first color, the second color, and the third color at the sensed temperature.

13. The non-transitory computer-readable storage medium of claim 11 , wherein the sum of the first weight, the second weight and the third weight is equal to 1.

14. The

Assignees

• Gopro Inc

Inventors

• Campbell Scott Patrick
• Fong Gary

Classifications

• H04N23/88 Primary

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N25/68

  applied to defects · CPC title

• H04N9/045 Primary

  Electricity · mapped topic

• H04N9/73

  Colour balance circuits, e.g. white balance circuits or colour temperature control (camera processing pipelines for colour balance H04N23/88) · CPC title

• H04N5/367

  Electricity · mapped topic