3D lighting

Patent metadata

Field	Value
Publication number	US-10262452-B2
Application number	US-201615274284-A
Country	US
Kind code	B2
Filing date	Sep 23, 2016
Priority date	Sep 30, 2015
Publication date	Apr 16, 2019
Grant date	Apr 16, 2019

Abstract

Techniques are disclosed for displaying a graphical element in a manner that simulates three-dimensional (3D) visibility (including parallax and shadowing). More particularly, a number of images, each captured with a known spatial relationship to a target 3D object, may be used to construct a lighting model of the target object. In one embodiment, for example, polynomial texture maps (PTM) using spherical or hemispherical harmonics may be used to do this. Using PTM techniques a relatively small number of basis images may be identified. When the target object is to be displayed, orientation information may be used to generate a combination of the basis images so as to simulate the 3D presentation of the target object.

First claim

The invention claimed is:

1. An electronic device, comprising:
memory;
a display unit coupled to the memory;
one or more sensor elements;
one or more processors coupled to the memory, the display unit, the one or more sensor elements, the one or more processors configured to execute program instructions stored in the memory to cause the electronic device to—
display a first image of an object on the display unit, the first image indicative of a first three-dimensional presentation of the object at a first orientation,
determine, based on output from the one or more sensors, orientation information of the electronic device corresponding to a second orientation,
determine a second image of the object based on a light model of the object and the orientation information, wherein the light model of the object comprises a plurality of images of the object at different viewing angles,
add synthetic shadows, based on the orientation information, to the second image of the object to generate a third image of the object, and
display the third image of the object on the display unit, wherein the third image is indicative of a second three-dimensional presentation of the object.

2. The electronic device of claim 1 , wherein the orientation information comprises an orientation of the electronic device relative to a gravity field.

3. The electronic device of claim 1 , wherein the orientation information comprises an orientation of the electronic device relative to a light source.

4. The electronic device of claim 1 , further comprising an image capture unit configured to capture a scene from a same direction as light emitted by the display unit.

5. The electronic device of claim 4 , further comprising program instructions stored in the memory to cause the electronic device to:
capture an image with the image capture unit coincident with display of the first image of the object; and
determine the orientation information of the electronic device based on analysis of the captured image.

6. The electronic device of claim 5 , wherein analysis of the captured image includes face detection.

7. The electronic device of claim 1 , wherein the light model of the object comprises a polynomial texture map (PTM) model.

8. The electronic device of claim 1 , wherein the light model of the object includes parallax information.

9. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:
obtain a first image of an object from a memory;
display the first image of the object on a display unit, the first image indicative of a first three-dimensional presentation of the object at a first orientation;
determine, based on output from one or more sensors, orientation information of an electronic device corresponding to a second orientation, wherein the electronic device includes the memory and the display unit;
determine a second image of the object based on a light model of the object and the orientation information, wherein the light model of the object comprises a plurality of images of the object at different viewing angles;
add synthetic shadows, based on the orientation information, to the second image of the object to generate a third image of the object; and
display the third image of the object on the display unit, wherein the third image is indicative of a second three-dimensional presentation of the object.

10. The non-transitory program storage device of claim 9 , wherein the instructions to cause the one or more processors to determine orientation information comprise instructions to cause the one or more processors to determine the orientation information based on a gravity field.

11. The non-transitory program storage device of claim 9 , wherein the instructions to cause the one or more processors to determine orientation information comprise instructions to cause the one or more processors to determine the orientation information based on detection of a light source.

12. The non-transitory program storage device of claim 9 , further comprising instructions to cause the one or more processors to:
capture, with an image capture unit, a scene from a same direction as light emitted by the display unit, wherein the capture occurs at a time coincident with display of the first image; and
determine the orientation information based on analysis of the captured image.

13. The non-transitory program storage device of claim 12 , wherein the instructions to cause the one or more processors to determine the orientation information based on analysis of the captured image comprise instructions to cause the one or more processors to detect a face in the captured image.

14. The non-transitory program storage device of claim 12 , wherein the light model of the object comprises a polynomial texture map (PTM) model.

15. The non-transitory program storage device of claim 9 , wherein the light model of the object includes parallax information.

16. A method to display a three-dimensional representation of an object, comprising:
displaying a first image of an object on a display unit of an electronic device, the first image indicative of a first three-dimensional presentation of the object at a first orientation;
determining, based on output from one or more sensors of the electronic device, orientation information of the electronic device corresponding to a second orientation;
determining a second image of the object based on a light model of the object and the orientation information, wherein the light model of the object comprises a plurality of images of the object at different viewing angles;
adding synthetic shadows, based on the orientation information, to the second image of the object to generate a third image of the object; and
displaying the third image of the object on the display unit, wherein the third image is indicative of a second three-dimensional presentation of the object.

17. The method of claim 16 , wherein the orientation information comprises an orientation of the electronic device relative to a gravity field.

18. The method of claim 16 , wherein the orientation information comprises an orientation of the electronic device relative to a light source.

19. The method of claim 16 , further comprising:
capturing an image with an image capture unit of the electronic device coincident in time with displaying the first image, wherein the image capture unit is configured to capture image in a direction outward from the display unit; and
determining the orientation information based on analysis of the captured image.

20. The method of claim 19 , wherein analysis of the captured image includes detecting a face in the captured image.

21. The method of claim 16 , wherein the light model of the object comprises a polynomial texture map (PTM) model.

22. The method of claim 16 , wherein the light model of the object includes parallax information.

Assignees

• Apple Inc

Inventors

• Motta Ricardo
• Youngs Lynn R
• Kim Minwoong

Classifications

• G06T15/60

  Shadow generation · CPC title

• G06T2215/16

  Using real world measurements to influence rendering · CPC title

• G06T15/205 Primary

  Image-based rendering · CPC title

• G06K9/00228

  Physics · mapped topic

• G06T15/50

  Lighting effects · CPC title