Handling real-world light sources in virtual, augmented, and mixed reality (xR) applications

Patent metadata

Field	Value
Publication number	US-10540812-B1
Application number	US-201916243924-A
Country	US
Kind code	B1
Filing date	Jan 9, 2019
Priority date	Jan 9, 2019
Publication date	Jan 21, 2020
Grant date	Jan 21, 2020

Abstract

Embodiments of systems and methods for handling real-world light sources in virtual, augmented, and mixed reality (xR) applications are described. In an illustrative, non-limiting implementation, an Information Handling System (IHS) may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution by the processor, cause the IHS to: detect a real-world light source using a sensor mounted on a Head-Mounted Device (HMD); identify a property of the real-world light source; and indicate the property to a rendering engine during execution of an xR application, where the rendering engine is configured to render an image for display by the HMD based on the property.

First claim

The invention claimed is:

1. An Information Handling System (IHS), comprising:
a processor; and
a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution by the processor, cause the IHS to:
detect a real-world light source using a sensor mounted on a Head-Mounted Device (HMD);
identify a property of the real-world light source; and
indicate the property to a rendering engine during execution of a virtual, augmented, or mixed reality (xR) application, at least in part, by providing at least a portion of a look-up table (LUT) of virtual light sources comprising type, position, color, intensity, and size using an Application Programming Interface (API), wherein the rendering engine is configured to render an image for display by the HMD based on the property.

2. The IHS of claim 1 , wherein the real-world light source comprises a natural or artificial light source.

3. The IHS of claim 1 , wherein the sensor comprises an RGB camera.

4. The IHS of claim 1 , wherein the property comprises at least one of: location, size, intensity, or color.

5. The IHS of claim 4 , wherein determining the location further comprises determining HMD pose derived from Simultaneous Localization and Mapping (SLAM) data.

6. The IHS of claim 4 , wherein determining the location further comprises determining a distance between the real-world light source and the HMD using a depth camera.

7. The IHS of claim 1 , wherein the program instructions, upon execution by the processor, further cause the IHS to transform a real-world location of the real-world light source to a virtual location in a coordinate system used by the xR application.

8. The IHS of claim 1 , wherein the program instructions, upon execution by the processor, further cause the IHS to classify the real-world light source as: point, spot, directional, or area.

9. The IHS of claim 1 , wherein the program instructions, upon execution by the processor, further cause the IHS to:
detect movement of the real-world light source during execution of a virtual, augmented, or mixed reality (xR) application; and
update the property to reflect the movement.

10. The IHS of claim 1 , wherein the program instructions, upon execution by the processor, further cause the IHS to:
detect a change in an attribute of the real-world light source during execution of a virtual, augmented, or mixed reality (xR) application; and
update the property to reflect the change in the attribute.

11. A method, comprising:
identifying a real-world light source using a camera mounted on a Head-Mounted Device (HMD);
modeling the real-world light source as a virtual light source having a type, position, color, intensity, and size;
providing at least a portion of a look-up table (LUT) of virtual light sources comprising type, position, color, intensity, and size to a rendering engine using an Application Programming Interface (API); and
rendering a three-dimensional overlay image for display by the HMD, at least in part, using the portion of the LUT.

12. The method of claim 11 , further comprising classifying the real-world light source as: point, spot, directional, or area.

13. The method of claim 11 , further comprising:
detecting movement of the real-world light source;
simulating a corresponding movement of the virtual light source; and
updating an illumination of the overlay image according to the movement.

14. The method of claim 11 , further comprising:
detecting a change in intensity of the real-world light source;
simulating a corresponding change of intensity of the virtual light source; and
updating an illumination of the overlay image according to the change in intensity.

15. The method of claim 11 , further comprising:
detecting a change in color of the real-world light source;
simulating a corresponding change of color of the virtual light source; and
updating an illumination of the overlay image according to the change in color.

16. A hardware memory of a Head-Mounted Device (HMD), the hardware memory having program instructions stored thereon that, upon execution by a processor, cause the HMD to:
capture an image of a real-world light source during execution of an augmented reality (AR) application using a sensor mounted on a Head-Mounted Device (HMD);
provide the image to an Information Handling System (IHS) coupled to the HMD, wherein the IHS is configured to: (a) identify a property of the real-world light source, at least in part, using at least a portion of a look-up table (LUT) of virtual light sources comprising type, position, color, intensity, and size using an Application Programming Interface (API), and (b) illuminate or shade an image rendered using the property;
receive the rendered image from the IHS; and
display the rendered image.

17. The hardware memory of claim 16 , wherein to illuminate or shade the image rendered using the property, the IHS is further configured to transform a real-world location of the real-world light source to another location in a coordinate system used by the AR application.

Assignees

• Dell Products Lp

Inventors

• Yildiz Yagiz Can
• Gatson Michael S
• Comer Ryan Nicholas

Classifications

• G06T15/50 Primary

  Lighting effects · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2219/2016

  Rotation, translation, scaling · CPC title

• G06T2219/2012

  Colour editing, changing, or manipulating; Use of colour codes · CPC title

• G06T2215/16

  Using real world measurements to influence rendering · CPC title