Method and system for color calibration of an imaging device

Patent metadata

Field	Value
Publication number	US-12019239-B2
Application number	US-202117154895-A
Country	US
Kind code	B2
Filing date	Jan 21, 2021
Priority date	Jul 24, 2018
Publication date	Jun 25, 2024
Grant date	Jun 25, 2024

Abstract

A method for characterizing a digital color camera includes, for each of three primary colors used in a field sequential color virtual image, determining a conversion model for each color using RGB values and the color-measurement values. For each primary color, the method includes illuminating a display device using an input light beam of a primary color having spectral properties representative of a light beam in a virtual image in a wearable device. The method includes capturing, with the digital color camera, an image of the display device, and determining, from the image, RGB values for each primary color. The method includes capturing, with a color-measurement device, a color-measurement value associated with each corresponding primary color at the display device, thereby acquiring a color-measurement value in an absolute color space. A conversion model for each color is determined using RGB values and the color-measurement values.

First claim

What is claimed is:

1. A method for utilizing a color camera for measuring chromaticity and luminance of a virtual image generated by a device, wherein the the method comprises:
illuminating an integrating sphere or a projection screen with narrowband light sources that correspond to light sources in the device that generate the virtual image, or having spectral bands that are characteristic of spectral bands emerging from the virtual image, wherein the device includes a field sequential color display illuminated by narrowband sources and a waveguide type optical element for displaying the virtual image over a field of view, the waveguide type optical element having properties that further narrow and spectrally shift dominant wavelength, saturation, and hue of light over the field of view of the virtual image, wherein the waveguide type optical element includes diffractive structures;
adding dichroic filters, and rotating the dichroic filters, prior to the integrating sphere or the projection screen to mimic bandwidth narrowing and spectrally shifted dominant wavelength of light in the virtual image;
using the color camera to capture images of the integrating sphere or screen at a conjugate distance close to a virtual image distance, the color camera characterized by an RGB (red green blue) color space;
focusing the color camera close to a virtual image plane;
in parallel, using a spectroradiometer to capture a spectrum and/or values in CIE XYZ color space of light at the integrating sphere or screen;
focusing the spectroradiometer at the integrating sphere or projection screen;
capturing images from the color camera and spectral data from the spectroradiometer at separate primary colors (red, green, or blue), which are close to primary colors expected for field sequential display of the virtual image;
creating a conversion model from the camera's RGB color space to XYZ absolute color space for each virtual image primary color by:
assembling training samples from captured images and corresponding spectroradiometer measurements in the camera's RGB color space and spectroradiometer's XYZ space;
applying conversion methods, including one or more of linear regression, polynomial regression, or neural networks, to generate a transformation from RGB training data to XYZ training data;
applying methods, including one or more of a k-fold cross validation or leave-one-out cross validation, to further optimize robustness and accuracy of the conversion model; and

applying the conversion model to the color camera for measuring absolute chromaticity and luminance of the virtual images.

2. The method of claim 1 , wherein the narrowband light sources comprise LEDs, lasers, or a monochromator.

3. The method of claim 1 , further comprising converting spectrum or XYZ values to values in an absolute color space, the values in the absolute color space including one or more of values in CIE XYZ (CIE XYZ color space), CIE xyY (CIE xyY color space), CIELUV (CIE 1976 L*, u*, v* color space), CIELab (CIE L*a*b* color space), or LCh (CIE L*C*h* color space).

4. The method of claim 1 , further comprising providing variations in primary color spectral content by one or more of altering power to the narrowband light sources, applying pulse-width-modulation of the narrowband light sources, filtering the narrowband light sources, or combining two or more narrowband light sources to provide data spanning a range of expected virtual image primaries.

5. A method for characterizing a digital color camera, comprising: for each of three primary colors used in a field sequential color virtual image: illuminating a display device using an input light beam of a primary color having spectral properties representative of a light beam in a virtual image in a wearable device; capturing, with the digital color camera, an image of the display device, the digital color camera characterized in an RGB (red green blue) color space; determining, from the image, RGB values for each primary color; capturing, with a color-measurement device, a color-measurement value associated with each corresponding primary color at the display device, thereby acquiring the color-measurement value in an absolute color space; determining a conversion model for each color using the RGB values and the color-measurement values by assembling training data from captured images in the RGB color space of the digital color camera and corresponding spectroradiometer measurements in absolute XYZ space; and applying a conversion method, including one or more of linear regression, polynomial regression, or neural networks, to generate the conversion model from converting RGB training data to XYZ training data; and outputting the conversion model to the digital color camera for use in correcting measurements of absolute chromaticity and luminance of the virtual image in the wearable device.

6. The method of claim 5 , wherein the method comprises determining the conversion model for each of red, green, and blue primary colors.

7. The method of claim 5 , further comprising providing the input light beam using a light source and a spectral modification device that provide narrowed and shifted spectral bands representative of the virtual image.

8. The method of claim 5 , wherein the display device is an integrating sphere or a projection screen.

9. The method of claim 5 , wherein the color-measurement device is a spectroradiometer or a colorimeter.

10. The method of claim 5 , wherein the conversion model for the display device comprises a 3-by-3 matrix.

11. The method of claim 5 , wherein the color-measurement value in the absolute color space comprises spectrum or XYZ values.

12. The method of claim 5 , wherein the color-measurement value in the absolute color space comprises values in CIE XYZ (CIE XYZ color space), CIE xyY (CIE xyY color space), CIELUV (CIE 1976 L*, u*, v* color space), CIELab (CIE L*a*b* color space), or LCh (CIE L*C*h* color space).

13. A system for characterizing a digital color camera, the system comprising:
a light source for projecting an image;
a spectral modification device;
an integrating sphere;
a digital color camera configured to capture RGB (red green blue) values of the image at each pixel for each color;
a spectroradiometer configured to determine color-measurement values of the image in an absolute color space;
a processor; and
a non-transitory computer-readable memory comprising instructions that, when executed by the processor, cause the processor to perform operations including:
determining a conversion model between RGB values of the image captured by the digital color camera and the color-measurement values of the image in the absolute color space determined by the spectroradiometer for each of three primary colors used in a field sequential color virtual image by:
assembling training data from captured images from the digital color camera and corresponding spectroradiometer measurements in the absolute color space; and
applying a conversion method, including one or more of linear regression, polynomial regression, or neural networks, to generate the conversion model from converting RGB training data to XYZ training data; and

outputting the conversion model to the digital color camera for use in correcting measurements of absolute chromaticity and luminance of the field sequential color virtual image output by a wearable device.

14. The system of claim 13 , wherein the system is configured to provide an input light beam using the light source and the spectral modification device that provide narrowed and shifted spectral bands representative of the field sequential

Assignees

• Magic Leap Inc

Inventors

• Schuck Iii Miller Harry
• Zhang Lei
• Grossmann Etienne Gregoire
• Shah Nukul Sanjay
• Zohar Ohad
• Zito Robert
• Morley Nicholas Ihle
• Schaefer Jason
• Jia Zhiheng
• Browy Eric C
• Capps Marshall Charles
• Tanaka Kazunori
• Vesom Grace
• Monos John

Classifications

• G06T2207/10024

  Color image · CPC title

• G01J3/462

  Computing operations in or between colour spaces; Colour management systems · CPC title

• G01J3/0251

  Colorimeters making use of an integrating sphere · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title