Interactive three dimensional displays on handheld devices

Patent metadata

Field	Value
Publication number	US-9208618-B2
Application number	US-201314076839-A
Country	US
Kind code	B2
Filing date	Nov 11, 2013
Priority date	Apr 8, 2010
Publication date	Dec 8, 2015
Grant date	Dec 8, 2015

Abstract

Techniques are disclosed for rendering an anamorphic projection of 3D scene geometry on a handled device using a correct asymmetric perspective geometry projection. Once pose of the handheld device is determined, a relative eye position may be inferred when the device is tilted away from an initial or default pose, based on data supplied by accelerometers. Thus, embodiments of the invention result in a holographic style display without the need for glasses or external sensing attachments.

First claim

What is claimed is:

1. A computer-implemented method for rendering a set of three-dimensional (3D) geometry on a display of a handheld device, the method comprising:
determining, based on accelerometer data, a current pose of a handheld device;
determining, based on a current ambient lighting state, a lighting response to apply in rendering the 3D geometry on the display; and in response to the current pose of the handheld device and the current ambient lighting state, rendering an anamorphic 3D image of the 3D geometry on the display of the handheld device, wherein the anamorphic 3D image presents a holographic style image on the display of the handheld device when viewed from a relative eye position corresponding to the current pose, lighted based on the current ambient lighting state;
wherein the holographic style image creates an illusion of at least a portion of the 3D geometry appearing in 3D space above or below the plane of the display of the handheld device by distorting the 3D geometry using an anamorphic projection.

2. The method of claim 1 , wherein the lighting response applies the current ambient lighting state in rendering the 3D geometry.

3. The method of claim 1 , further comprising:
capturing an image using a camera on the handheld device; and
recovering an approximate light probe from RGB intensity values recorded by the image, wherein the lighting state is determined from the approximate light probe.

4. The method of claim 1 , further comprising:
capturing an image using a camera on the handheld device, wherein one or more elements of the 3D geometry are generated from the captured image.

5. The method of claim 4 , further comprising:
overlying one or more virtual 3D objects over at least one elements of the 3D geometry generated from the captured image.

6. The method of claim 1 , wherein the handheld device includes a camera, and wherein the method further comprises:
tracking an eye position of a viewer using the camera; and
wherein the current pose the handheld device is determined based on the tracked eye position.

7. The method of claim 1 , further comprising, filtering the accelerometer data using a low-pass filter.

8. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, perform an operation for rendering a set of three-dimensional (3D) geometry on a display of a handheld device, the operation comprising:
determining, based on accelerometer data, a current pose of a handheld device;
determining, based on a current ambient lighting state, a lighting response to apply in rendering the 3D geometry on the display; and
in response to the current pose of the handheld device and the current ambient lighting state, rendering an anamorphic 3D image of the 3D geometry on the display of the handheld device, wherein the anamorphic 3D image presents a holographic style image on the display of the handheld device when viewed from a relative eye position corresponding to the current pose, lighted based on the current ambient lighting state;
wherein the holographic style image creates an illusion of at least a portion of the 3D geometry appearing in 3D space above or below the plane of the display of the handheld device by distorting the 3D geometry using an anamorphic projection.

9. The computer-readable storage medium of claim 8 , wherein the lighting response applies the current ambient lighting state in rendering the 3D geometry.

10. The computer-readable storage medium of claim 8 , wherein the operation further comprises:
capturing an image using a camera on the handheld device; and
recovering an approximate light probe from RGB intensity values recorded by the image, wherein the lighting state is determined from the approximate light probe.

11. The computer-readable storage medium of claim 8 , wherein the operation further comprises:
capturing an image using a camera on the handheld device, wherein one or more elements of the 3D geometry are generated from the captured image.

12. The computer-readable storage medium of claim 11 , wherein the operation further comprises:
overlying one or more virtual 3D objects over at least one elements of the 3D geometry generated from the captured image.

13. The computer-readable storage medium of claim 8 , wherein the handheld device includes a camera, and wherein the method further comprises:
tracking an eye position of a viewer using the camera; and
wherein the current pose the handheld device is determined based on the tracked eye position.

14. The computer-readable storage medium of claim 8 , wherein the operation further comprises, filtering the accelerometer data using a low-pass filter.

15. A system, comprising:
a processor; and
a memory configured to perform an operation for rendering a set of three-dimensional (3D) geometry on a display of a handheld device, the operation comprising:
determining, based on accelerometer data, a current pose of a handheld device, determining, based on a current ambient lighting state, a lighting response to apply in rendering the 3D geometry on the display, and
in response to the current pose of the handheld device and the current ambient lighting state, rendering an anamorphic 3D image of the 3D geometry on the display of the handheld device, wherein the anamorphic 3D image presents a holographic style image on the display of the handheld device when viewed from a relative eye position corresponding to the current pose, lighted based on the current ambient lighting state;
wherein the holographic style image creates an illusion of at least a portion of the 3D geometry appearing in 3D space above or below the plane of the display of the handheld device by distorting the 3D geometry using an anamorphic projection.

16. The system of claim 15 , wherein the lighting response applies the current ambient lighting state in rendering the 3D geometry.

17. The system of claim 15 , wherein the operation further comprises:
capturing an image using a camera on the handheld device; and
recovering an approximate light probe from RGB intensity values recorded by the image, wherein the lighting state is determined from the approximate light probe.

18. The system of claim 15 , wherein the operation further comprises:
capturing an image using a camera on the handheld device, wherein one or more elements of the 3D geometry are generated from the captured image.

19. The system of claim 18 , wherein the operation further comprises:
overlying one or more virtual 3D objects over at least one elements of the 3D geometry generated from the captured image.

20. The system of claim 15 , wherein the handheld device includes a camera, and wherein the method further comprises:
tracking an eye position of a viewer using the camera; and
wherein the current pose the handheld device is determined based on the tracked eye position.

21. The system of claim 15 , wherein the operation further comprises, filtering the accelerometer data using a low-pass filter.

Assignees

• Disney Entpr Inc

Inventors

• Mitchell Kenny

Classifications

• G06F1/1613

  for portable computers (cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202) · CPC title

• G06F2200/1614

  Image rotation following screen orientation, e.g. switching from landscape to portrait mode · CPC title

• G06F2200/1637

  Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer · CPC title

• G06T15/20 Primary

  Perspective computation · CPC title

• H04N7/183

  for receiving images from a single remote source · CPC title