Dynamic display calibration based on eye-tracking

Patent metadata

Field	Value
Publication number	US-10260864-B2
Application number	US-201615341822-A
Country	US
Kind code	B2
Filing date	Nov 2, 2016
Priority date	Nov 4, 2015
Publication date	Apr 16, 2019
Grant date	Apr 16, 2019

Abstract

Examples of a light field metrology system for use with a display are disclosed. The light field metrology may capture images of a projected light field, and determine focus depths (or lateral focus positions) for various regions of the light field using the captured images. The determined focus depths (or lateral positions) may then be compared with intended focus depths (or lateral positions), to quantify the imperfections of the display. Based on the measured imperfections, an appropriate error correction may be performed on the light field to correct for the measured imperfections. The display can be an optical display element in a head mounted display, for example, an optical display element capable of generating multiple depth planes or a light field display.

First claim

What is claimed is:

1. A display system comprising:
an eye-tracking camera;
a wearable display comprising a plurality of display color layers, each display color layer of the plurality of display color layers configured to output a different color;
non-transitory data storage configured to store a plurality of chromatic calibrations for the display, each chromatic calibration in the plurality of chromatic calibrations associated with a calibration position relative to the display, wherein each chromatic calibration in the plurality of chromatic calibrations corrects for a chromatic imperfection of the display comprising a chromatic mismatch between two or more display color layers of the plurality of display color layers; and
a hardware processor in communication with the eye-tracking camera, the display, and the non-transitory data storage, the hardware processor programmed to:
determine, based on information from the eye-tracking camera, an eye position, relative to the display, of the user of the display;
access, based at least partly on the determined eye position, one or more of the plurality of chromatic calibrations;
calculate, based at least in part on the one or more of the plurality of chromatic calibrations, a correction to apply to at least one of the two or more display color layers of the display to at least partially correct the chromatic imperfection of the display; and
apply the correction to the at least one of the two or more display color layers of the display.

2. The display system of claim 1 , wherein a number of calibration positions is 2, 3, 4, 5, 6, 7, 8, 9, or more.

3. The display system of claim 1 , wherein the calibration positions are distributed across the display in a grid.

4. The display system of claim 3 , wherein the grid comprises a 2×2, a 3×3, a 5×5, or a 9×9grid.

5. The display system of claim 1 , wherein the one or more of the plurality of chromatic calibrations comprises a chromatic calibration associated with a calibration position that is closest to the eye position.

6. The display system of claim 1 , wherein to calculate the correction, the hardware processor is programmed to interpolate or to extrapolate among the one or more of the plurality of chromatic calibrations, based at least in part on the calibration positions of the one or more of the plurality of chromatic calibrations and the determined eye position.

7. The display system of claim 1 , wherein the display comprises a first display associated with a first eye of the user and a second display associated with a second eye of the user, and the hardware processor is programmed to determine the eye position of the user relative to the first display and to apply the determined eye position for calculating the correction for the second display.

8. The display system of claim 1 , wherein the display comprises a first display associated with a first eye of the user and a second display associated with a second eye of the user, and wherein at least some of the plurality of calibrations represent an average calibration for the first display and the second display.

9. The display system of claim 1 , wherein the display comprises a light field display.

10. The display system of claim 1 , wherein the display comprises a stackable waveguide assembly comprising a plurality of waveguides, and a display color layer in the plurality of display color layers comprises at least one of the plurality of waveguides.

11. The display system of claim 1 , wherein the display is configured as a head-mounted wearable display system.

12. The display system of claim 1 , wherein the hardware processor is further programmed to correct for one or more spatial imperfection that comprise one or more of an in-plane translation, rotation, scaling, or warping error or an out-of-plane or focal depth error.

13. The display system of claim 1 , wherein each chromatic calibration in the plurality of chromatic calibrations comprises an adjustment in luminance between the two or more display color layers of the plurality of display color layers.

14. The display system of claim 13 , wherein:
the two or more display color layers of the plurality of display color layers comprise a first color layer having a first luminance and a second color layer having a second luminance, the first luminance greater than the second luminance, and
the adjustment in luminance comprises a reduction in the first luminance to a value of the second luminance.

15. A method for calibrating a wearable display, the method comprising:
under control of a dynamic calibration system performed by computer hardware:
determining an eye position for a user of the display, the display comprising a plurality of display color layers, each display color layer of the plurality of display color layers configured to output a different color;
accessing, based at least partly on the determined eye position, a chromatic calibration for the display, wherein the chromatic calibration is selected based upon an associated calibration position and the determined eye position, and wherein the chromatic calibration is configured to correct for a chromatic imperfection of the display comprising a chromatic mismatch between two or more display color layers of the plurality of display color layers;
calculating, based at least in part on the accessed chromatic calibration, a correction to apply to at least one of the two or more display color layers of the display to at least partially correct for the chromatic imperfection in the display, wherein calculating the correction comprises calibrating for the chromatic imperfection of the display; and
applying the correction to the at least one of the two or more display color layers of the display.

16. The method of claim 15 , wherein accessing the chromatic calibration comprises selecting one or more chromatic calibrations from a plurality of chromatic calibrations, wherein each chromatic calibration is associated with a different calibration position relative to the display.

17. The method of claim 16 , wherein the calibration positions are arranged in a grid across the display.

18. The method of claim 16 , wherein calculating the correction comprises interpolating or extrapolating among the one or more of the plurality of chromatic calibrations, based upon the associated calibration positions of the one or more of the plurality of chromatic calibrations and the determined eye position.

19. The method of claim 15 , further comprising accessing an image of an eye of the user of the display, and determining the eye position based at least in part upon the image of the eye.

20. A wearable display system comprising:
an inward-facing imaging system;
a display comprising a plurality of display color layers, each display color layer of the plurality of display color layers configured to output a different color;
non-transitory data storage configured to store a plurality of chromatic calibrations for the display, each chromatic calibration in the plurality of chromatic calibrations associated with a calibration position relative to the display and configured to correct for a chromatic imperfection of the display comprising a chromatic mismatch between two or more display color layers of the plurality of display color layers; and
a hardware processor in communication with the inward-facing imaging system, the display, and the non-transitory data storage, the hardware processor programmed to:
determine, using the inward-facing imaging system, an eye position relativ

Assignees

• Magic Leap Inc

Inventors

• Edwin Lionel E
• Yeoh Ivan L
• Miller Samuel A

Classifications

• G09G2320/028

  by changing the viewing angle properties, e.g. widening the viewing angle, adapting the viewing angle to the view direction · CPC title

• G09G2320/0626

  for control of overall brightness · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G09G2340/0464

  Positioning · CPC title

• G09G2320/0276

  for the purpose of adaptation to the characteristics of a display device, i.e. gamma correction · CPC title