Method for processing images having specularities and corresponding computer program product

Patent metadata

Field	Value
Publication number	US-10497136-B2
Application number	US-201615574475-A
Country	US
Kind code	B2
Filing date	Jun 7, 2016
Priority date	Jun 8, 2015
Publication date	Dec 3, 2019
Grant date	Dec 3, 2019

Abstract

A method for processing images of a scene including a surface made of a material of unknown reflectance comprises the following steps: from at least 3 different positions of an image sensor, the positions and corresponding orientations of the sensor known, acquiring images of the scene illuminated by a light source, each image containing specularity generated by reflection of the light source from the surface and depending on the position, the shape and the intensity of the light source and the reflectance of the material; in each image, detecting each specularity and for each specularity, estimating a conic approximating the specularity. It comprises constructing a quadric representing the position, intensity and shape of the light source and the reflectance of the material, on the basis of the conics, and of the position and orientation of the image sensor during the acquisition of the images containing the specularities respectively approximated by these conics.

First claim

The invention claimed is:

1. A method for processing images of a scene including a surface made of a material of unknown reflectance, the method comprising the following steps:
from at least 3 different positions of an image sensor, the positions and corresponding orientations of the sensor being known, acquiring images of the scene illuminated by a light source, each image containing at least one specularity generated by reflection of the light source from said surface and depending on the position of the light source, on the shape of the light source and on the intensity of the light source and on the reflectance of the material;
in each image, detecting each specularity; and
for each specularity, estimating a conic approximating said specularity;
constructing a quadric representing the position of the light source, the intensity of the light source, the shape of the light source, the reflectance of the material, on the basis of the conics, the position, and the orientation of the image sensor during the acquisition of the images containing the specularities respectively approximated by these conics.

2. The image-processing method as claimed in claim 1 , the surface having an unknown roughness, the quadric also represents the roughness of the surface.

3. The image-processing method as claimed in claim 1 , wherein the scene is illuminated by at least one other light source, each image containing at least one specularity generated by reflection of the at least one other light source from said surface, and further comprising for each specularity:
temporal tracking of the specularity and
matching said specularity, and therefore the conic approximating it, with a light source,
and wherein the step of constructing a quadric is carried out for each light source, on the basis of the conics matched with said light source, and of the position and orientation of the image sensors during the acquisition of the images containing the specularities respectively approximated by these conics.

4. The image-processing method as claimed in claim 1 , further comprising the following steps:
for each specularity, minimizing the distance between the specularity and a conic resulting from projecting the quadric onto the image containing said specularity along an axis determined by the position and orientation of the sensor, in order to determine a new quadric; and
iterating the preceding step until a preset stop criterion is reached, the new quadric resulting from the last iteration being a refined quadric.

5. The image-processing method as claimed in claim 1 , comprising a step of choosing key images from the acquired images, depending on a predefined criterion of distribution of viewpoints of the sensor in the scene with respect to the light source.

6. The image-processing method as claimed in claim 1 , wherein the light source is a point source and wherein the corresponding quadric is a sphere.

7. The image-processing method as claimed in claim 1 , wherein the light source is an extended source and wherein the corresponding quadric is an ellipsoid.

8. The image-processing method as claimed in claim 1 , comprising a step of calculating a prediction of the position and shape of a conic called the predicted conic approximating a specularity formed on a surface made of a material of known normal, depending on a new position and a new orientation of said image sensor during the acquisition of a new image and on the projection of the quadric onto said new image.

9. The image-processing method as claimed in claim 8 , wherein quadric scales being preset, the conic prediction calculation is carried out for each scale so as to obtain as many conics as scales, each conic corresponding to an intensity level of the predicted specularity.

10. The image-processing method as claimed in claim 8 , wherein the material of the predicted specularity is the same as the material associated with the quadric, and wherein the size of the predicted specularity is also predicted.

11. The image-processing method as claimed in claim 1 , comprising a step of correcting error in the position and orientation of the image sensor on the basis of a calculation of a discrepancy between a specularity present in the image associated with said position and orientation and a conic resulting from the projection of the quadric onto said image.

12. The image-processing method as claimed in claim 1 , wherein the different positions and orientations of the image sensor are obtained using the same moving image sensor or using a plurality of stationary image sensors.

13. A computer-program, comprising instructions stored on a tangible non-transitory storage medium for executing, on the basis of:
at least 3 different positions of an image sensor, the positions and corresponding orientations of the sensor being known, and
images of a scene including a surface, made of a material of unknown reflectance, illuminated by a light source, which images are acquired by said image sensor, each image containing at least one specularity generated by reflection of the light source from said surface and dependent on the position of the light source, on the shape of the light source, on the intensity of the light source and on the reflectance of the material,
the following steps to be carried out when said program is executed on a computer:
in each image, detecting each specularity;
for each specularity, estimating a conic approximating said specularity; and
constructing a quadric representing the position of the light source, the intensity of the light source, the shape of the light source, the reflectance of the material, on the basis of the conics, the position, and the orientation of the image sensor during the acquisition of the images containing the specularities respectively approximated by these conics.

14. A computer-program as claimed in claim 13 , wherein the surface having an unknown roughness, the quadric also represents the roughness of the surface.

Assignees

• Commissariat Energie Atomique

Inventors

• Tamaazousti Mohamed
• Morgand Alexandre
• Bartoli Adrien

Classifications

• G06T7/73

  using feature-based methods · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T15/83

  Phong shading · CPC title

• G06T15/506

  Illumination models · CPC title

• G06T7/41 Primary

  based on statistical description of texture · CPC title