Methods and apparatus for retargeting and prioritized interpolation of lens profiles

Patent metadata

Field	Value
Publication number	US-11403739-B2
Application number	US-75864710-A
Country	US
Kind code	B2
Filing date	Apr 12, 2010
Priority date	Apr 12, 2010
Publication date	Aug 2, 2022
Grant date	Aug 2, 2022

Abstract

Methods and apparatus for retargeting and prioritized interpolation of lens profiles. A lens profile file may include a set of lens sub-profiles. The camera body and/or settings described in the file may not exactly match that of camera body and/or settings used to capture a target image. A sub-profile processing module may perform a prioritized sub-profile sorting and interpolation method to generate an interpolated sub-profile that may be applied to the target image to correct aberrations including, but not limited to, geometric distortion, lateral chromatic aberration, and vignette. Thus, models generated for a reference camera at a variety of settings may be applied to a target image captured with the same type of lens but with a different camera and/or with different settings that are not exactly modeled in the lens profile file.

First claim

What is claimed is:

1. A computer-implemented method comprising:
obtaining a target image, the target image including image data captured by an image sensor of a camera and image metadata including parameters that describe image shooting conditions associated with the capture of the image data;
responsive to obtaining the target image, obtaining a lens profile file for a camera/lens combination, the lens profile file including a set of sub-profiles associated with different aberrations of the camera/lens combination, each of the sub-profiles including:
a mathematical model for correcting a respective aberration of the different aberrations; and
sub-profile metadata specifying one or more camera settings;

prioritizing the image shooting conditions based on a particular aberration of the different aberrations to correct in the target image;
comparing the parameters to the one or more camera settings specified in the sub-profile metadata of each sub-profile;
determining, based on the comparing, that the sub-profiles do not include any matching sub-profiles, a matching sub-profile having sub-profile metadata that specifies one or more camera settings which match the shooting conditions described by the parameters;
locating two of the sub-profiles having the camera settings that best match the image shooting conditions described by at least one of the parameters included in the image metadata, including describing a highest-priority image shooting condition according to the prioritizing;
generating an interpolated mathematical model for correcting the particular aberration by interpolating a first mathematical model of a first of the two sub-profiles with a second mathematical model of a second of the two sub-profiles; and
responsive to the generating, correcting the particular aberration in the target image by applying the interpolated mathematical model to the target image.

2. The computer-implemented method as recited in claim 1 , further comprising updating the lens profile file to include a new sub-profile that includes the interpolated mathematical model.

3. The computer-implemented method as recited in claim 1 , further comprising retargeting the lens profile file based on at least one of the parameters of the target image, the retargeting including:
comparing orientation of the target image to orientation of mathematical models in the lens profile file;
determining that the orientation of the target image is different than the orientation of the mathematical models in the lens profile file based on comparing the orientations; and
responsive to the determining, changing the orientation of the mathematical models to match the orientation of the target image.

4. The computer-implemented method as recited in claim 1 , further comprising retargeting the lens profile file based on at least one of the parameters of the target image, the retargeting including:
comparing image resolution of the target image to image resolution of mathematical models in the lens profile file;
determining that the image resolution of the target image is different than the image resolution of the mathematical models in the lens profile file based on comparing the image resolutions; and
responsive to the determining, scaling the mathematical models in the lens profile file to normalize the image resolution.

5. The computer-implemented method as recited in claim 1 , further comprising retargeting the lens profile file based on at least one of the parameters of the target image, the retargeting including:
comparing sensor size of the image sensor to sensor size of mathematical models in the lens profile file;
determining that the sensor size of the image sensor is different than the sensor size of the mathematical models in the lens profile file based on comparing the sensor sizes; and
responsive to the determining, scaling the mathematical models in the lens profile file to the sensor size of the image sensor.

6. The computer-implemented method as recited in claim 1 , wherein the two sub-profiles have values for the highest-priority shooting condition that bracket a corresponding parameter value included in the image metadata.

7. The computer-implemented method as recited in claim 1 , wherein:
the different aberrations include geometric distortion, lateral chromatic aberration, and vignette; and
each sub-profile includes a geometric distortion mathematical model, a lateral chromatic aberration mathematical model, or a vignette mathematical model.

8. The computer-implemented method as recited in claim 1 , wherein the different aberrations include geometric distortion and lateral chromatic aberration and the sub-profile metadata of each sub-profile specifies camera settings for sub-profile focal length and sub-profile focus distance.

9. The computer-implemented method as recited in claim 1 , wherein:
the different aberrations include vignette;
the image shooting conditions include focal length, aperture, and focus distance; and
a determination regarding the best match is based on a priority associated with the focal length, the aperture, and the focus distance, from highest to lowest priority.

10. The computer-implemented method as recited in claim 1 , further comprising locating the two sub-profiles having the camera settings that best match the shooting conditions described by at least two of the parameters included in the image metadata, the at least two parameters describing the highest-priority image shooting condition and a second highest-priority image shooting condition according to the prioritizing.

11. The computer-implemented method as recited in claim 1 , wherein the image shooting conditions include a target image focal length, target image focus distance, target image aperture, and target image sensor format factor.

12. A computer-readable storage medium, excluding signals per se, storing program instructions, wherein the program instructions are computer-executable to perform operations comprising:
receiving, by an image processing application, input specifying to correct aberrations in a target image;
selecting, by the image processing application, a lens profile file according to a camera/lens combination, the lens profile file including a set of sub-profiles, each of the sub-profiles including a mathematical model for correcting a respective aberration of different aberrations associated with particular camera settings of the camera/lens combination and metadata specifying the particular camera settings;
obtaining, by the image processing application, parameters from image metadata included with the target image, the parameters describing camera settings associated with capture of image data of the target image;
comparing, by the image processing application, the parameters to the particular camera settings of each of the sub-profiles; and
according to the comparing:
if the obtained parameters match the particular camera settings of at least one of the sub-profiles, correcting, by the image processing application, the different aberrations in the target image according to the mathematical model included with the matching sub-profiles; or
if the obtained parameters do not match the particular camera settings of at least one of the sub-profiles, correcting, by the image processing application, the different aberrations in the target image, in part, by:
selecting a particular aberration of the different aberrations to correct in the target image;
determining priorities of the camera settings described by the parameters based on the particular aberration;
locating two of the sub-profiles having the particular camera settings that bracket a parameter fro

Assignees

• Chen Simon
• Chan Eric
• Jin Hailin
• Chien Jen Chan
• Adobe Inc

Inventors

• Chen Simon
• Chan Eric
• Jin Hailin
• Chien Jen-Chan

Classifications

• H04N25/61

  the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N23/81

  for suppressing or minimising disturbance in the image signal generation · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title

• H04N9/04515

  Electricity · mapped topic