Automatic variable virtual focus for augmented reality displays

Patent metadata

Field	Value
Publication number	US-9588341-B2
Application number	US-201615043144-A
Country	US
Kind code	B2
Filing date	Feb 12, 2016
Priority date	Nov 8, 2010
Publication date	Mar 7, 2017
Grant date	Mar 7, 2017

Abstract

The technology provides an augmented reality display system for displaying a virtual object to be in focus when viewed by a user. In one embodiment, the focal region of the user is tracked, and a virtual object within the user focal region is displayed to appear in the focal region. As the user changes focus between virtual objects, they appear to naturally move in and out of focus as real objects in a physical environment would. The change of focus for the virtual object images is caused by changing a focal region of light processing elements in an optical path of a microdisplay assembly of the augmented reality display system. In some embodiments, a range of focal regions are swept through at a sweep rate by adjusting the elements in the optical path of the microdisplay assembly.

First claim

What is claimed is:

1. A method for displaying virtual objects with variable focus by an augmented reality system comprising:
identifying by one or more processors a three dimensional location of one or more virtual objects within a user field of view of a near-eye display device of the augmented reality system, the near-eye display device comprising a microdisplay assembly and a display unit optically coupled to receive one or more images from the microdisplay assembly;
determining by the one or more processors a three dimensional focal region of a user wearing the near-eye display device within the user field of view;
identifying which of the one or more virtual objects are in the focal region of the user based on the three dimensional locations of the one or more virtual objects within the user field of view;
generating a series of layered image data at different user focal regions, by the microdisplay assembly, for each of the one or more virtual objects, such that the one or more virtual objects are identified as being in the user focal region; and
receiving from the microdisplay assembly and displaying by the display unit the generated image data of the one or more virtual objects.

2. The method of claim 1 , wherein determining by the one or more processors a three dimensional focal region of a user wearing the near-eye display device within the user field of view further comprises:
determining the three dimensional focal region of the user based on a Panum's fusional area.

3. The method of claim 1 , further comprising:
applying at least one artificial depth of field technique to a virtual object which is within the user field of view but outside the focal region of the user as a function of distance from the focal region.

4. The method of claim 1 , wherein the microdisplay assembly including one or more light processing elements including at least one optical element and a microdisplay unit aligned in an optical path with the at least one optical element.

5. The method of claim 1 , further comprising:
adjusting a focal region of a the microdisplay assembly for generating image data in the different focal regions by actuating of the one or more light processing elements in the optical path of the microdisplay assembly under control of the one or more processors.

6. The method of claim 4 , wherein moving the one or more light processing elements in the optical path of the microdisplay assembly under control of the one or more processors further comprises:
determining by the one or more processors a displacement value, S1, between a front nodal point of the at least one optical element and a target location of at least one virtual object, the at least one optical element being located in an optical path between a microdisplay and a reflecting element,
determining by the one or more processors a displacement value, S2, between the reflecting element and a rear nodal point of the at least one optical element for a focal length of the at least one optical element, f, based on the equation 1/S1+1/S2=1/f, and
moving the at least one optical element under control of the one or more processors to generate the determined displacement values S1 and S2.

7. An augmented reality system providing variable focus of virtual objects comprising:
a near-eye support structure;
one or more processors;
a see-through display unit positioned by the near-eye support structure to be seen through by a user eye;
a microdisplay assembly for generating one or more images, supported by the near-eye support structure and optically coupled to the see-through display unit for outputting image data for a user focal region to the see-through display unit, the microdisplay assembly includes one or more light processing elements including a microdisplay and at least one optical element movable to place one or more virtual objects in the user focal region by actuating the one or more light processing elements in an optical path, in which the microdisplay is aligned, under control of the one or more processors;
an armature for supporting the at least one light processing element of the microdisplay assembly and stabilizing movement of the at least one light processing element in the optical path;
the one or more processors controlling the microdisplay assembly for generating the image data for the one or more virtual objects in the user focal region;
the display unit receiving the generated image data from the microdisplay assembly and displaying the generated image data and including a driver to control the at least one optical element, the processor directing the microdisplay to include at least one of the one or more virtual objects in the user focal region; and
a display optically coupled to the microdisplay assembly via the optical path for displaying three dimensional virtual images.

8. The augmented reality system of claim 7 , wherein the armature is under the control of a variable virtual adjuster.

9. The augmented reality system of claim 7 , wherein the actuation is controlled by a piezoelectric actuator.

10. The augmented reality system of claim 7 , further comprising a timing generator, under control of the one or more processors, for providing a rate to the driver controlling the actuation of the one or more light processing elements;
the at least one light processing element of the microdisplay assembly moving at the rate through a range of displacements in the optical path of the microdisplay assembly, the range of displacements corresponding to a range of focal regions;
the range of focal regions including a focal region at which image generated by the microdisplay results in image data being displayed by the display unit at the user focal region; and
the one or more processors directing the microdisplay to generate an image including any virtual object identified to be located in a respective focal region of the range when the respective displacement in the range of displacements is reached.

11. The augmented reality system of claim 8 , wherein:
the at least one optical element of the microdisplay assembly comprises a set of insertable lenses and the variable virtual adjuster comprises a respective arm for moving each insertable lens in the set into and out of the optical path; and
the variable virtual adjuster changing the focal region of the microdisplay assembly to place one or more virtual objects in a current focal region of a user by moving the one or more light processing elements in the optical path under control of one or more processors further comprises moving the respective arm for moving a position of a respective insertable lens in the set with respect to the optical path for changing the focal region.

12. The augmented reality system of claim 8 , wherein:
the at least one optical element of the microdisplay assembly comprises a set of lenses in a rotatable support under control of the variable virtual adjuster; and
the variable virtual adjuster changing the focal region of the microdisplay assembly to place one or more virtual objects in the user focal region by moving the one or more light processing elements in the optical path under control of one or more processors further comprises the variable virtual adjuster rotating the rotatable support to change a lens of the set of lenses in the optical path for changing the focal region.

13. The augmented reality system of claim 10 , wherein the display unit projects the images generated by the range of displacements to the at least one eye location.

14. The augmented reality system of claim 10 , wherein the microdisplay generates a composite image including in-focu

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Bar-Zeev Avi
• Lewis John

Classifications

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B27/0101 Primary

  characterised by optical features (G02B27/0172 takes precedence) · CPC title

• G02B2027/0134

  of stereoscopic type · CPC title