Method and apparatus to estimate scene illuminant based on skin reflectance database

Patent metadata

Field	Value
Publication number	US-12444229-B2
Application number	US-202418410713-A
Country	US
Kind code	B2
Filing date	Jan 11, 2024
Priority date	Nov 30, 2018
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

Some embodiments of a method may include obtaining an image a real-world environment, determining an estimated illuminant spectral power distribution of an illuminant of the real-world environment, and detecting a region of the image representing human skin. The method may further include determining a representative skin color value of the region and based on the estimated illuminant spectral power distribution and the representative skin color value most closely matching the representative color value, selecting a candidate skin reflectance spectrum. The method may further include updating the estimated illuminant spectral power distribution base on the representative skin color value and the selected candidate skin reflectance spectrum.

First claim

What is claimed:

1. A method comprising:
obtaining an image of a real-world environment;
determining at least one area of the image representing human skin;
determining a representative color value for at least one of the areas of the image representing human skin;
comparing the representative color value with a plurality of expected skin color values, each expected skin color value being associated with a respective one of a plurality of prospective illuminants;
selecting an estimated illuminant from the plurality of prospective illuminants based at least in part on the comparison of the representative color value; and
rendering a virtual object using color values computed based on the selected estimated illuminant.

2. The method of claim 1 , wherein the representative color value is an average of color values for the area of the image representing human skin.

3. The method of claim 1 , wherein the representative color value is an sRGB, CIE RGB, CIE XYZ, CIELUV, or CIELAB value.

4. The method of claim 1 , wherein determining the representative color value and comparing the representative color value are performed in a device-dependent color space.

5. The method of claim 1 , wherein determining the representative color value and comparing the representative color value are performed in a device-independent color space.

6. The method of claim 1 , wherein the expected skin color value is based at least in part on a spectral power distribution of the prospective illuminant.

7. The method of claim 1 , wherein the plurality of expected skin color values are stored in a database associating each expected skin color value with a respective prospective illuminant.

8. The method of claim 1 , wherein selecting the estimated illuminant comprises minimizing a plurality of difference values, each difference value corresponding to a difference for one of the at least one determined areas of the image representing human skin between the representative color value and the expected skin color value.

9. The method of claim 1 , further comprising updating iteratively the prospective illuminant based on the comparison of the representative color value with the expected skin color value corresponding to the prospective illuminant prior to updating until a difference between the representative color value and the expected skin color value is less than a threshold.

10. The method of claim 1 , wherein the prospective illuminant is a mixture of two or more illuminants chosen from the plurality of illuminants.

11. An apparatus comprising one or more processors configured to perform at least:
obtaining an image of a real-world environment;
determining at least one area of the image representing human skin;
determining a representative color value for at least one of the areas of the image representing human skin;
comparing the representative color value with a plurality of expected skin color values, each expected skin color value being associated with a respective one of a plurality of prospective illuminants;
selecting an estimated illuminant from the plurality of prospective illuminants based at least in part on the comparison of the representative color value; and
rendering a virtual object using color values computed based on the selected estimated illuminant.

12. The apparatus of claim 11 , wherein the representative color value is an average of color values for the area of the image representing human skin.

13. The apparatus of claim 11 , wherein the representative color value is an sRGB, CIE RGB, CIE XYZ, CIELUV, or CIELAB value.

14. The apparatus of claim 11 , wherein determining the representative color value and comparing the representative color value are performed in a device-dependent color space.

15. The apparatus of claim 11 , wherein determining the representative color value and comparing the representative color value are performed in a device-independent color space.

16. The apparatus of claim 11 , wherein the expected skin color value is based at least in part on a spectral power distribution of the prospective illuminant.

17. The apparatus of claim 11 , wherein the plurality of expected skin color values are stored in a database associating each expected skin color value with a respective prospective illuminant.

18. The apparatus of claim 11 , wherein selecting the estimated illuminant comprises minimizing a plurality of difference values, each difference value corresponding to a difference for one of the at least one determined areas of the image representing human skin between the representative color value and the expected skin color value.

19. The apparatus of claim 11 , further configured to update iteratively the prospective illuminant based on the comparison of the representative color value with the expected skin color value corresponding to the prospective illuminant prior to updating until a difference between the representative color value and the expected skin color value is less than a threshold.

20. The apparatus of claim 11 , wherein the prospective illuminant is a mixture of two or more illuminants chosen from the plurality of illuminants.

Assignees

• Interdigital Vc Holdings Inc

Inventors

• Wyble David

Classifications

• G06T15/506

  Illumination models · CPC title

• G06V10/60 Primary

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/56

  relating to colour · CPC title

• G06V40/162 Primary

  using pixel segmentation or colour matching · CPC title