Shadow-based estimation of 3D lighting parameters from reference object and reference virtual viewpoint

Patent metadata

Field	Value
Publication number	US-12333646-B2
Application number	US-202118012500-A
Country	US
Kind code	B2
Filing date	Jun 23, 2021
Priority date	Jun 30, 2020
Publication date	Jun 17, 2025
Grant date	Jun 17, 2025

Abstract

A method and device are provided for processing images. In one embodiment, the method comprises receiving an image is received of a scene having a plurality of real objects and determining if at least one of these real objects is a reference object stored in a database. Subsequently, candidate shadow map of the reference object identified is retrieved when available and when a reference object cannot be identified, an object in the scene with adequate parameters is selected and the candidate shadow maps of this new reference object using said lighting parameters is computed.

First claim

The invention claimed is:

1. A method implemented by a device, the method comprising:
determining a real object in an augmented reality scene observed through the device, for which a set of candidate shadow maps and associated lighting models are stored in a memory associated with the device;
selecting, from the set of candidate shadow maps, a candidate shadow map that matches with a shadow map determined from a shadow mask of the real object in the augmented reality scene as observed through the device; and
adding virtual shadows to virtual objects in the augmented reality scene according to a lighting model associated with the selected candidate shadow map; and
wherein the shadow mask of the real object in the augmented reality scene is based on shadows cast by the real object on a non-planar surface on which the real object is placed, a geometry of the non-planar surface being included in a geometric model used to obtain the set of candidate shadow maps.

2. The method of claim 1 , wherein the determined real object is placed in an area of the augmented reality scene such that shadows cast in the area are only due to the real object.

3. The method of claim 2 , further comprising:
delineating an area of the augmented reality scene containing the shadows cast by the real object.

4. The method of claim 3 , wherein the delineating is performed by placement of vertices of a polygon.

5. A device comprising at least one processor, the at least one processor configured to:
determine a real object in an augmented reality scene observed through the device, for which a set of candidate shadow maps and associated lighting models are stored in a memory associated with the device;
select, from the set of candidate shadow maps, a candidate shadow map that matches with a shadow map determined from a shadow mask of the real object in the augmented reality scene as observed through the device; and
add virtual shadows to virtual objects in the augmented reality scene according to a lighting model associated with the selected candidate shadow map;
wherein the shadow mask of the real object in the augmented reality scene is based on shadows cast by the real object on a non-planar surface on which the real object is placed, a geometry of the non-planar surface being included in a geometric model used to obtain the set of candidate shadow maps.

6. The device of claim 5 , wherein the determined real object is placed in an area of the augmented reality scene such that shadows cast in the area are only due to the real object.

7. The device of claim 6 , wherein the at least one processor is further configured to delineate an area of the augmented reality scene containing the shadows cast by the real object.

8. The device of claim 7 , wherein the at least one processor is further configured to perform the delineating by placement of vertices of a polygon.

9. A non-transitory computer readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising:
determining a real object in an augmented reality scene observed through the device, for which a set of candidate shadow maps and associated lighting models are stored in a memory associated with the device;
selecting, from the set of candidate shadow maps, a candidate shadow map that matches with a shadow map determined from a shadow mask of the real object in the augmented reality scene as observed through the device; and
adding virtual shadows to virtual objects in the augmented reality scene according to a lighting model associated with the selected candidate shadow map;
wherein the shadow mask of the real object in the augmented reality scene is based on shadows cast by the real object on a non-planar surface on which the real object is placed, geometry of the non-planar surface being included in a geometric model used to obtain the set of candidate shadow maps.

10. The non-transitory computer readable medium of claim 9 , wherein the determined real object is placed in an area of the augmented reality scene such that shadows cast in the area are only due to the real object.

11. The non-transitory computer readable medium of claim 10 , wherein the instructions further cause the one or more processors to delineate an area of the augmented reality scene containing the shadows cast by the real object.

12. The non-transitory computer readable medium of claim 11 , wherein the instructions further cause the one or more processors to delineate the area by placement of vertices of a polygon.

Assignees

• Interdigital Ce Patent Holdings Sas

Inventors

• Robert Philippe
• Laurent Anthony
• Jouet Pierrick
• Luo Tao

Classifications

• G06T2215/16

  Using real world measurements to influence rendering · CPC title

• G06T15/506

  Illumination models · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06T15/04

  Texture mapping · CPC title