Systems and methods for head-mounted display adapted to human visual mechanism

Patent metadata

Field	Value
Publication number	US-10460704-B2
Application number	US-201615088816-A
Country	US
Kind code	B2
Filing date	Apr 1, 2016
Priority date	Apr 1, 2016
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

Systems and methods are provided for rendering of a dual eye-specific display. The system tracks the user's eye movements and/or positions, in some implementations, based on electroencephalography (EEG) of the user, to correctly label the central (foveal) and peripheral (extra-foveal) areas of the display. Foveal data is fully rendered while extra-foveal data is reduced in resolution and, in some implementations, shared between the two displays.

First claim

The invention claimed is:

1. A head-mounted display device comprising:
a first display for a left eye;
a second display for a right eye;
memory to store data to present via the first display and via the second display; and
one or more processors to:
determine, based on eye-tracking information of a user wearing the head-mounted display, a first foveal region of the first display, a first extra-foveal region of the first display, a second foveal region of the second display, and a second extra-foveal region of the second display, the eye-tracking information representative of at least one of an eye movement or an eye position of the user;
identify a first subset of the data as corresponding to first foveal region pixels associated with the first foveal region, a second subset of the data as corresponding to first extra-foveal region pixels associated with the first extra-foveal region, a third subset of the data as corresponding to second foveal region pixels associated with the second foveal region, and a fourth subset of the data as corresponding to second extra-foveal region pixels associated with the second extra-foveal region;
load the first subset at a first full resolution from the memory to a first foveal frame buffer;
load the second subset at a first reduced resolution from the memory to a first extra-foveal frame buffer;

load the third subset at a second full resolution from the memory to a second foveal frame buffer; and
load the fourth subset at a second reduced resolution from the memory to a second extra-foveal frame buffer.

2. The device of claim 1 , further including at least one of an infrared light projector or an electroencephalograph (EEG) device to generate the eye-tracking information.

3. The device of claim 1 , wherein the one or more processors are to adjust at least one of the first foveal region, the first extra-foveal region, the second foveal region, or the second extra-foveal region based on the eye-tracking information.

4. The device of claim 1 , wherein the one or more processors are to duplicate a pixel value in a group of pixels across at least one of 2×2 pixels, 3×3 pixels, 4×4 pixels, or 5×5 pixels.

5. The device of claim 1 , further including a Display Serial Interface in communication with the memory and in communication with at least one of the first display or the second display, the one or more processors to duplicate a pixel value after receiving the pixel value from the Display Serial Interface.

6. The device of claim 1 , further including a first Display Serial Interface (DSI) associated with the first foveal frame buffer and a second DSI associated with the second foveal frame buffer, the first and second DSIs to transfer, in parallel, the data from the first foveal frame buffer and the second foveal frame buffer.

7. The device of claim 1 , wherein the first extra-foveal frame buffer and second extra-foveal frame buffer are implemented as a shared extra-foveal frame buffer.

8. The device of claim 1 , wherein the one or more processors are to:
load the first foveal frame buffer at a first rate;
load the first extra-foveal frame buffer at a second rate;
load the second foveal frame buffer at a third rate; and
load the second extra-foveal frame buffer at a fourth rate.

9. The device of claim 1 , wherein the one or more processors are to apply alpha-blending (a) between depth-planes of the first foveal region pixels and depth-planes of the first extra-foveal region pixels and (b) between depth-planes of the second foveal region pixels and depth-planes of the second extra-foveal region pixels.

10. The device of claim 1 , wherein the one or more processors is to duplicate at least one of (a) a first pixel value of the first foveal region to represent the first extra-foveal region pixels or (b) a second pixel value of the second foveal region to represent the second extra-foveal region pixels in the first reduced resolution or the second reduced resolution, respectively.

11. A method to present images to eyes of a user of a head-mounted display, the method comprising:
determining, by executing an instruction with at least one processor, based on eye-tracking information of the user wearing the head-mounted display, a first foveal region of a first display, a first extra-foveal region of the first display, a second foveal region of a second display, and a second extra-foveal region of the second display, the eye-tracking information representative of at least one of an eye movement or an eye position of the user;
identifying, by executing an instruction with the at least one processor, a first subset of data as corresponding to first foveal region pixels associated with the first foveal region, a second subset of the data as corresponding to first extra-foveal region pixels associated with the first extra-foveal region, a third subset of the data as corresponding to second foveal region pixels associated with the second foveal region, and a fourth subset of the data as corresponding to second extra-foveal region pixels associated with the second extra-foveal region;
loading, by executing an instruction with the at least one processor, the first subset at a first full resolution from a memory to a first foveal frame buffer;
loading, by executing an instruction with the at least one processor, the second subset at a first reduced resolution from the memory to a first extra-foveal frame buffer;
loading, by executing an instruction with the at least one processor, the third subset at a second full resolution from the memory to a second foveal frame buffer; and
loading, by executing an instruction with the at least one processor, the fourth subset at a second reduced resolution from the memory to a second extra-foveal frame buffer.

12. The method of claim 11 , further including generating the eye-tracking information with at least one of an infrared light projector or an electroencephalograph (EEG).

13. The method of claim 11 , further including adjusting at least one of the first foveal region, the first extra-foveal region, the second foveal region, or the second extra-foveal region based on the eye-tracking information.

14. The method of claim 11 , further including duplicating a pixel value after retrieving the pixel value from a Display Serial Interface in communication with a memory and at least one of the first display or the second display.

15. The method of claim 11 , wherein the first extra-foveal frame buffer and the second extra-foveal frame buffer are implemented as a shared extra foveal frame buffer.

16. The method of claim 11 , further including:
loading the first foveal frame buffer at a first rate;
loading the first extra-foveal frame buffer at a second rate;
loading the second foveal frame buffer at a third rate; and
loading the second extra-foveal frame buffer at a fourth rate.

17. The method of claim 11 , further including applying alpha-blending (a) between depth-planes of the first foveal region pixels and depth-planes of the first extra-foveal region pixels and (b) between depth-planes of the second foveal region pixels and depth-planes of the second extra-foveal region pixels.

18. The method of claim 11 , further including transferring from at least two Display Serial Interfaces, in parallel, the first subset at the first full resolution from the first foveal frame buffer to the first display, the second subset at the first reduced resolution from the first extra-foveal frame buffer to the first display, the third subset at the second full resolution from the second foveal frame

Assignees

• Linear Algebra Tech Limited
• Movidius Ltd

Inventors

• Barry Brendan
• Moloney David

Classifications

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• G09G2354/00

  Aspects of interface with display user · CPC title

• H04N19/17

  the unit being an image region, e.g. an object · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G09G5/006

  Details of the interface to the display terminal (specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04) · CPC title