Virtual, augmented, and mixed reality systems and methods

Patent metadata

Field	Value
Publication number	US-10748259-B2
Application number	US-201916697711-A
Country	US
Kind code	B2
Filing date	Nov 27, 2019
Priority date	Aug 22, 2016
Publication date	Aug 18, 2020
Grant date	Aug 18, 2020

Abstract

A virtual, augmented, or mixed reality display system includes a display configured to display virtual, augmented, or mixed reality image data, the display including one or more optical components which introduce optical distortions or aberrations to the image data. The system also includes a display controller configured to provide the image data to the display. The display controller includes memory for storing optical distortion correction information, and one or more processing elements to at least partially correct the image data for the optical distortions or aberrations using the optical distortion correction information.

First claim

The invention claimed is:

1. A virtual, augmented, or mixed reality display system comprising:
a display configured to display virtual, augmented, or mixed reality image data, the display comprising:
one or more optical components which introduce optical distortions or aberrations to the image data, and
a plurality of at least partially distinct optical paths corresponding to a plurality of colors of the image data; and

a display controller configured to provide the image data to the display, the display controller comprising:
memory for storing optical distortion correction information, the optical distortion correction information comprising separate optical distortion correction information for each of the plurality of colors of the image data, and
one or more processing elements to at least partially correct the image data for the optical distortions or aberrations using the optical distortion correction information.

2. The system of claim 1 , wherein the optical distortion correction information is used by the display controller to pre-distort the image data in a way that is at least partially inversely related to the optical distortions or aberrations.

3. The system of claim 2 , wherein the display controller is configured to pre-distort the image data which it provides to the display by determining a distortion-corrected pixel at a first location (x, y) based on one or more non-distortion-corrected pixels near a different second location (x′, y′) in non-distortion-corrected image data received by the display controller.

4. The system of claim 3 , wherein the optical distortion correction information is used to determine the second location (x′, y′).

5. The system of claim 4 , wherein (x′, y′) coordinates of the second location are fractional numbers.

6. The system of claim 5 , wherein the display controller is configured to determine the distortion-corrected pixel at the first location (x, y) by interpolating between a plurality of non-distortion-corrected pixels surrounding the second location (x′, y′).

7. The system of claim 6 , wherein the display controller uses bilinear interpolation.

8. The system of claim 1 , wherein the optical distortion correction information is stored in the form of one or more lookup tables.

9. The system of claim 8 , wherein the one or more lookup tables are stored in a compressed format, and wherein the display controller is configured to expand the one or more lookup tables before correcting for the optical distortions or aberrations using the optical distortion correction information.

10. The system of claim 1 , wherein the optical distortion correction information further comprises information for performing one or more image warp operations, and wherein the display controller is further configured to perform the one or more image warp operations on the image data.

11. A method in a virtual, augmented, or mixed reality display system, the method comprising:
providing virtual, augmented, or mixed reality image data to be shown on a display, the display comprising:
one or more optical components which introduce optical distortions or aberrations to the image data, and
a plurality of at least partially distinct optical paths corresponding to a plurality of colors of the image data;

storing optical distortion correction information, the optical distortion correction information comprises separate optical distortion correction information for each of the plurality of colors of the image data;
at least partially correcting the image data for the optical distortions or aberrations using the optical distortion correction information; and
displaying the image data to a user with the display.

12. The method of claim 11 , further comprising using the optical distortion correction information to pre-distort the image data in a way that is at least partially inversely related to the optical distortions or aberrations.

13. The method of claim 12 , further comprising pre-distorting the image data provided to the display by determining a distortion-corrected pixel at a first location (x, y) based on one or more non-distortion-corrected pixels near a different second location (x′, y′) in undistorted image data.

14. The method of claim 13 , further comprising using the optical distortion correction information to determine the second location (x′, y′).

15. The method of claim 14 , wherein (x′, y′) coordinates of the second location are fractional numbers.

16. The method of claim 15 , further comprising determining the distortion-corrected pixel at the first location (x, y) by interpolating between a plurality of non-distortion-corrected pixels surrounding the second location (x′, y′).

17. The method of claim 12 , wherein the optical distortion correction information further comprises information for performing one or more image warp operations, and further comprising performing the one or more image warp operations on the image data.

18. The method of claim 16 , further comprising using bilinear interpolation.

19. The method of claim 11 , further comprising storing the optical distortion correction information in the form of one or more lookup tables.

20. The method of claim 19 , further comprising storing the one or more lookup tables in a compressed format, and expanding the one or more lookup tables before correcting for the optical distortions or aberrations using the optical distortion correction information.

Assignees

• Magic Leap Inc

Inventors

• Rodriguez Jose Felix
• Perez Ricardo Martinez

Classifications

• G06T5/90

  Dynamic range modification of images or parts thereof · CPC title

• G06F1/3203

  Power management, i.e. event-based initiation of a power-saving mode · CPC title

• G09G2330/021

  Power management, e.g. power saving · CPC title

• G06T2210/44

  Morphing · CPC title

• G06T2207/10024

  Color image · CPC title