Wavefront sensing in a head mounted display

Patent metadata

Field	Value
Publication number	US-10656707-B1
Application number	US-201815950017-A
Country	US
Kind code	B1
Filing date	Apr 10, 2018
Priority date	Apr 10, 2018
Publication date	May 19, 2020
Grant date	May 19, 2020

Abstract

A head mounted display (HMD) includes an eye sensor, an optics block, and a display. The eye sensor includes a detector and a tunable lens. The tunable lens has a micro-lens state (e.g., acts as a micro-lens array) and a neutral state. In the micro-lens state the eye sensor acts as an accommodation sensor, and in the neutral state the eye sensor acts as an eye tracking sensor. The eye sensor can alternate functioning as an eye tracking or an accommodation sensor by adjusting the state of the tunable lens. In alternate embodiments, the accommodation sensor is separate from the eye tracker and a beam splitter is used to split the reflected light toward the two sensors.

First claim

What is claimed is:

1. A head mounted display (HMD) comprising:
at least one processor;
an electronic display that:
displays a virtual scene to a user wearing the HMD, and
emits infrared (IR) light from a subset of pixels of the electronic display to deliver the IR light to a retina of an eye of the user;

an optics block that directs light from the electronic display to an exit pupil of the HMD;
an eye sensor including a tunable lens and a detector, the tunable lens alternates operation between a micro-lens state and a neutral state, in the micro-lens state, the eye sensor operates as a Shack-Hartmann wavefront sensor to determine a wavefront of the IR light after having passed through the eye of the user and, in the neutral state, the eye sensor operates as an eye tracking sensor; and
memory including instructions that, when executed by the at least one processor, cause the at least one processor to:
determine, at a first time, an eye position for the eye of the user for a frame of the virtual scene based on the IR light emitted from the electronic display being reflected from the eye of the user and received by the eye sensor with the tunable lens in the neutral state;
determine, at a second time, the wavefront of the IR light after the IR light passes through the crystalline lens of the eye of the user for the frame of the virtual scene based on the IR light emitted from the electronic display being reflected from the retina of the eye of the user and received by the eye sensor with the tunable lens in micro-lens state;
determine, for the frame of the virtual scene, a focus state for the optics block based on the wavefront of the IR light after the IR light passes through the crystalline lens of the eye of the user; and
display, based on the determined focus state, the frame of the virtual scene on the electronic display to the user wearing the HMD.

2. The HMD of claim 1 , wherein the optics block includes a hot mirror configured to redirect at least a portion of the IR light reflected from the retina of the eye of the user to the eye sensor.

3. The HMD of claim 1 , wherein the state of the tunable lens is time-multiplexed between the micro-lens state and the neutral state.

4. The HMD of claim 1 , wherein the IR light has an annular beam profile or is Bessel beam IR light.

5. The HMD of claim 4 , wherein the annular beam profile of the IR light or the Bessel beam IR light is generated using a vertical-cavity surface-emitting laser (VCSEL) in the electronic display.

6. The HMD of claim 1 , wherein the tunable lens in micro-lens state includes a plurality of micro-lenses, each micro-lens:
receiving a portion of the IR light reflected from a different portion of the retina; and
focusing the portion of the IR light to a focal location on the detector.

7. The HMD of claim 6 , wherein determining the wavefront in micro-lens state includes:
determining, for each micro-lens, differences between each focal location of the focused portion of the IR light and an expected focal location for each portion of the IR light across the tunable lens;
aggregating the differences between each focal location of the focused portion of the IR light and the expected focal location for each portion of the IR light across the tunable lens; and
determining the wavefront of the IR light after having passed through the eye based on the aggregated differences across the tunable lens.

8. The HMD of claim 1 , wherein the determined focus state includes at least one of synthetic blur or pre-distortion, the synthetic blur added to the virtual scene based on the shape of the crystalline lens of each eye and the pre-distortion correcting one or more optical aberrations associated with at least one of the eye of the user or the optics block.

9. A head mounted display (HMD) comprising:
at least one processor;
an electronic display that displays a virtual scene to a user wearing the HMD while emitting infrared (IR) light from a subset of the electronic display to deliver the IR light to a retina of an eye of the user;
an optics block configured to direct light from the electronic display to an exit pupil of the HMD; and
an eye sensor including a tunable lens and a detector multiplexing between:
1) a micro-lens state where the eye tracking sensor operates as an accommodation sensor to determine accommodation information for the user, wherein each micro-lens receives a different portion of the IR light reflected from a different portion of the retina and focuses the different portion of the IR light to a known focal location on the detector; and
2) a neutral state where the eye sensor operates as an eye tracking sensor;

memory including instructions that, when executed by the at least one processor, cause the at least one processor to:
at a first time,
determine, for each micro-lens, differences between each focal location of the focused portion of the IR light and an expected focal location for each portion of the IR light across the tunable lens;
aggregate the differences between each focal location of the focused portion of the IR light and the expected focal location for each portion of the IR light across the tunable lens; and
determine the wavefront of the IR light after having passed through the eye based on the aggregated differences across the tunable lens; and

track, at a second time, a position of the eye of the user.

10. The HMD of claim 9 , wherein the memory including the instructions that, when executed by the at least one processor, further cause the at least one processor to:
determine accommodation information for the eye of the user based on the determined wavefront;
determine a focus state for the optics block based on the accommodation information; and
display, based on the determined focus state, the virtual scene on the electronic display to the user wearing the HMD.

11. The HMD of claim 10 , wherein the determined focus state includes at least one of synthetic blur or pre-distortion, the synthetic blur added to the virtual scene based on the accommodation information and the pre-distortion correcting one or more optical aberrations associated with at least one of the eye of the user or the optics block.

12. The HMD of claim 9 , wherein the optics block includes a hot mirror configured to redirect at least a portion of the IR light reflected from the retina of the eye of the user to the eye sensor.

13. The HMD of claim 9 , wherein the IR light has an annular beam profile or is Bessel beam IR light.

14. The HMD of claim 1 , wherein the subset of pixels of the electronic display that deliver the IR light to the retina of the eye of the user is determined based on a portion of the electronic display that corresponds to where the user is looking in order to deliver the IR light to the retina.

15. The HMD of claim 9 , wherein the subset of pixels of the electronic display that deliver the IR light to the retina of the eye of the user is determined based on a portion of the electronic display that corresponds to where the user is looking in order to deliver the IR light to the retina.

Assignees

• Facebook Tech Llc

Inventors

• Sharma Robin
• Zannoli Marina
• Mackenzie Kevin James

Classifications

• G02B27/0176

  characterised by mechanical features · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G02B5/04

  Prisms · CPC title