Method and apparatus for estimating reflectance parameters and a position of a light source in a scene

Patent metadata

Field	Value
Publication number	US-10132912-B2
Application number	US-201615268584-A
Country	US
Kind code	B2
Filing date	Sep 17, 2016
Priority date	Sep 17, 2015
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

A method, apparatus and system for estimating reflectance parameters and a position of the light source(s) of specular reflections of a scene include RGB sequence analysis with measured geometry in order to estimate specular reflectance parameters of an observed 3D scene. Embodiments include pixel-based image registration from which profiles of 3D scene points image intensities over the sequence are estimated. A profile is attached to a 3D point and to the set of pixels that display its intensity in the registered sequence. Subsequently, distinction is made between variable profiles that reveal specular effects and constant profiles that show diffuse reflections only. Then, for each variable profile diffuse reflectance is estimated and subtracted from the intensity profile to deduce the specular profile and the specular parameters are estimated for each observed 3D point. Then, the location of at least one light source responsible for the specular effects is estimated. Optionally, the parameters can be iteratively refined to determine color information and specular reflectance parameters.

First claim

The invention claimed is:

1. A method for estimating reflectance parameters and a position of at least one light source in a scene, from a sequence of RGB-D maps showing the scene, the method comprising:
determining an intensity profile for a plurality of first locations in the scene, an intensity profile being the evolution of the intensity of a first location from map to map of said sequence and a first location being a three-dimension point determined according to pixels of said RGB-D maps;
estimating, for second locations of said plurality of first locations having a variable intensity profile, a diffuse reflectance component according to said variable intensity profile;
determining, for said second locations, a light source direction from specular parameters derived from the diffuse reflectance component; and
estimating a position of the at least one light source using at least one determined light source direction.

2. The method of claim 1 , further comprising:
separating constant intensity profiles from variable intensity profiles,

wherein said separating comprises:
determining, for at least one of the intensity profiles, at least a minimum intensity value, a maximum intensity value, a mean intensity value, a median intensity value, a variance of the intensity profile and a standard deviation of the intensity profile;
applying a threshold value to the determined intensity values of at least one intensity profile to determine if an intensity profile is a variable profile or a constant profile.

3. The method of claim 2 , wherein applying a threshold value comprises:
determining if the absolute value of a difference between a median intensity value and a mean intensity value of an intensity profile is greater than a threshold; and
if the value of the difference is greater than the threshold, considering the intensity profile a variable intensity profile.

4. The method of claim 2 , wherein applying a threshold value comprises:
determining if a standard deviation of an intensity profile is greater than a threshold; and
if the value of the standard deviation is greater than the threshold, considering the intensity profile a variable intensity profile.

5. The method of claim 1 , wherein the diffuse reflectance component for a second location is equal to the minimum intensity value of a respective variable intensity profile.

6. The method of claim 5 , wherein a diffuse reflectance component is estimated for each color component of the variable intensity profile.

7. The method of claim 1 , wherein said specular parameters comprise at least one of a specular light intensity, a mirror reflection vector and a shininess component.

8. The method of claim 7 , wherein for deriving the specular light intensity a maximum intensity value of a respective intensity profile is used.

9. The method of claim 8 , wherein the mirror reflection vector is derived using a maximum specular light intensity value and is considered equal to a viewpoint vector.

10. The method of claim 7 , wherein the shininess component is derived by applying a minimization function to known specular light intensity and mirror reflection values and using a result of the function that has a least value as a best estimate of the shininess component.

11. The method of claim 1 , wherein the light source direction is determined, for said second locations, as a function of a mirror reflection vector and a normal vector at said second locations.

12. The method of claim 1 , comprising:
updating the specular parameters and light source direction to determine a color vector and a specular reflectance parameter.

13. An apparatus for estimating reflectance parameters and a position of at least one light source in a scene, from a sequence of RGB-D maps showing the scene, the apparatus comprising:
a memory adapted to store control programs, instructions, content and data; and
a processor adapted to execute the control programs and instructions, said processor when executing said control programs causing said apparatus to:
determine an intensity profile for a plurality of first locations in the scene, an intensity profile being the evolution of the intensity of a first location from map to map of said sequence and a first location being a three-dimension point determined according to pixels of said RGB-D maps;
estimate, for second locations of said plurality of first locations having a variable intensity profile, a diffuse reflectance component according to said variable intensity profile;
determine, for said second locations, a light source direction from specular parameters derived from the diffuse reflectance component; and
estimate a position of the at least one light source using at least one determined light source direction.

14. The apparatus of claim 13 , wherein the light source direction is determined, for said second locations, as a function of a mirror reflection vector and a normal vector at said second locations.

15. The apparatus of claim 13 , wherein said processor further causing the apparatus to:
separate constant intensity profiles from variable intensity profiles, wherein said separating comprises:
determine, for at least one of the intensity profiles, at least a minimum intensity value, a maximum intensity value, a mean intensity value, a median intensity value, a variance of the intensity profile and a standard deviation of the intensity profile;
apply a threshold value to the determined intensity values of at least one intensity profile to determine if an intensity profile is a variable profile or a constant profile.

16. The apparatus of claim 15 , wherein to apply a threshold value comprises:
determining if the absolute value of a difference between a median intensity value and a mean intensity value of an intensity profile is greater than a threshold; and
if the value of the difference is greater than the threshold, considering the intensity profile a variable intensity profile.

17. The apparatus of claim 15 , wherein to apply a threshold value comprises:
determining if a standard deviation of an intensity profile is greater than a threshold; and
if the value of the standard deviation is greater than the threshold, considering the intensity profile a variable intensity profile.

18. The apparatus of claim 13 , wherein the diffuse reflectance component for a second location is equal to the minimum intensity value of a respective variable intensity profile.

19. The apparatus of claim 18 , wherein the diffuse reflectance component is estimated for each color component of the variable intensity profile.

20. The apparatus of claim 13 , wherein said specular parameters comprise at least one of a specular light intensity, a mirror reflection vector and a shininess component.

Assignees

• Thomson Licensing

Inventors

• Robert Philippe
• Jiddi Salma
• Hudon Matis

Classifications

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/10024

  Color image · CPC title

• G01N21/4738

  Diffuse reflection (precedence is given to G01N21/55 - G01N21/57 if specular component is taken into consideration), e.g. also for testing fluids, fibrous materials · CPC title

• G01S3/783 Primary

  using amplitude comparison of signals derived from static detectors or detector systems · CPC title