Augmented reality light guide display

Patent metadata

Field	Value
Publication number	US-9558590-B2
Application number	US-201213432372-A
Country	US
Kind code	B2
Filing date	Mar 28, 2012
Priority date	Mar 28, 2012
Publication date	Jan 31, 2017
Grant date	Jan 31, 2017

Abstract

Augmented reality light guide display techniques are described. In one or more implementations, an apparatus includes a housing configured in a hand-held form factor, one or more sensors configured to detect a position and orientation of the housing in three dimensions in a physical environment of the housing, a light guide that is at least partially transparent and supported by the housing, a light engine that is optically coupled to the light guide, and one or more modules disposed within the housing and implemented at least partially in hardware. The one or more modules are configured to calculate a position and orientation of an augmentation and cause the light engine to output the augmentation for display using the light guide such that the augmentation is viewable concurrently with at least a portion of the physical environment through the light guide.

First claim

What is claimed is:

1. An apparatus comprising:
a housing configured in a hand-held form factor such that the housing is movable relative to a user;
one or more sensors configured to detect a position and orientation of the housing in three dimensions in a physical environment of the housing;
a light guide that is at least partially transparent and supported by the housing;
a light engine able to produce images, the light engine optically coupled to the light guide; and
one or more circuits configured to calculate a position and orientation of an augmentation based on the detected position and orientation of the housing in the physical environment and cause the light engine to output the augmentation for display using the light guide such that the augmentation changes based on a distance between the user and the housing as the housing is moved relative to the user and is viewable by the user concurrently with at least a portion of the physical environment through the light guide.

2. An apparatus as described in claim 1 , wherein the augmentation is viewable in three dimensions by the user.

3. An apparatus as described in claim 1 , wherein the housing is formed as part of a mobile phone or tablet and the one or more modules include wireless communication functionality.

4. An apparatus as described in claim 1 , wherein the light guide is configured to display the augmentation along an image plane focused at infinity.

5. An apparatus as described in claim 1 , wherein the light guide includes a layer that is selectable by the one or more modules to alter transparency of portions of the light guide such that a first said portion is transparent while a second said portion is not transparent.

6. An apparatus as described in claim 5 , wherein the first said portion is used to view the portion of the physical environment and the second said portion is used to view the augmentation.

7. An apparatus as described in claim 1 , wherein the one or more sensors are configured as one or more cameras, one or more depth sensing cameras, a gyroscope, one or more accelerometers, or a magnetometer.

8. A method comprising:
capturing one or more images of a user using one or more cameras of a handheld device that is held by a user, the handheld device movable relative to the user;
calculating a location of the user's pupils in three dimensional space from the captured one or more images by the handheld device, the location of the user's pupils comprising at least a distance from the handheld device to the user's pupils;
calculating a location of the handheld device in three dimensional space relative to a physical surroundings of the handheld device from one or more sensors of the handheld device; and
displaying an augmentation on a transparent display of the handheld device based on the calculated location of the user's pupils and the calculated location of the handheld device relative to the physical surrounding, the augmentation changing as the handheld device is moved relative to the user and viewable concurrently with at least a portion of the physical surroundings of the handheld device that is viewable through the transparent display.

9. A method as described in claim 8 , wherein the transparent display is a light guide that is optically coupled to a light engine, the light engine configured to output the augmentation as specified by the one or more modules.

10. A method as described in claim 9 , wherein the light engine does not output a display of the portion of the physical surroundings of the handheld device.

11. A method as described in claim 9 , wherein the light guide is configured to display the augmentation along an image plane focused at infinity.

12. A method as described in claim 9 , wherein the light guide includes a layer that is selectable to alter transparency of portions of the light guide such that a first said portion is transparent while a second said portion is not transparent.

13. A method as described in claim 12 , wherein the first said portion is used to view the portion of the physical environment and the second said portion is used to view the augmentation.

14. A method as described in claim 8 , wherein the handheld device is configured as a mobile phone or tablet.

15. A method as described in claim 8 , wherein the transparent display includes a first side that is viewable by the user to see the augmentation and a second side opposing the first side, the second side including one or more touch sensors configured to detect one or more gestures that are recognizable by the handheld device to initiate an operation of the handheld device.

16. A method as described in claim 8 , wherein the transparent display is configured to display the augmentation to be viewable in three dimensions by the user.

17. An apparatus comprising:
a housing configured in a hand-held form factor such that the housing is movable relative to a user;
one or more cameras positioned in the housing to track one or more eyes of the user;
one or more sensors configured to detect a position and orientation of the housing in three dimensions in a physical environment of the housing;
a light guide that is at least partially transparent and supported by the housing;
a light engine that is optically coupled to the light guide; and
one or more modules disposed within the housing and implemented at least partially in hardware, the one or more modules configured to calculate a position of one or more pupils of the user in three-dimensional space and cause the light engine to output an augmentation for display based on the calculated position of the housing relative to the pupils of the user as well as the position and orientation of the housing in the physical environment using the light guide such that the augmentation changes as the housing is moved relative to the user and is viewable concurrently with at least a portion of the physical environment through the light guide, the position of the one or more pupils of the user comprising at least a distance between the housing and the one or more pupils of the user.

18. An apparatus as described in claim 17 , wherein the light guide includes a layer that is selectable to alter transparency of portions of the light guide such that a first said portion that corresponds to the portion of the physical environment is made transparent while a second said portion that corresponds to the augmentation is not made transparent.

19. An apparatus as described in claim 17 , wherein the augmentation is viewable in three dimensions by the user.

20. An apparatus as described in claim 17 , wherein the one or more sensors are configured as one or more cameras, one or more depth sensing cameras, a gyroscope, one or more accelerometers, or a magnetometer.

Assignees

• Westerinen William J
• Robbins Steven John
• Badyal Rajeev
• Fleck Rod G
• Microsoft Technology Licensing Llc

Inventors

• Westerinen William J
• Robbins Steven John
• Badyal Rajeev
• Fleck Rod G

Classifications

• G06F1/1643

  the display being associated to a digitizer, e.g. laptops that can be used as penpads (details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615) · CPC title

• G06F1/1626

  with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs] · CPC title

• G09G2320/0261

  in the context of movement of objects on the screen or movement of the observer relative to the screen · CPC title

• G06F2203/04804

  Transparency, e.g. transparent or translucent windows · CPC title

• G06F1/1637

  Details related to the display arrangement, including those related to the mounting of the display in the housing · CPC title