Display update time reduction for a near-eye display

Patent metadata

Field	Value
Publication number	US-10514541-B2
Application number	US-201213728892-A
Country	US
Kind code	B2
Filing date	Dec 27, 2012
Priority date	Dec 27, 2012
Publication date	Dec 24, 2019
Grant date	Dec 24, 2019

Abstract

Technology is described for reducing display update time for a near-eye display (NED) device. A point of focus in the NED field of view is identified, often based on natural user input data. A communication module of a computer system communicatively coupled to the NED device transmits lossless priority data, an example of which is user focal region image data, using one or more communication techniques for satisfying lossless transmission criteria. Allowed loss image data is identified based at least in part on its distance vector from a point of focus in the display field of view. An example of allowed loss image data is image data to be displayed outside the user focal region. The allowed loss image data is transmitted and extracted from received image data allowing for lossy transmission.

First claim

What is claimed is:

1. A method comprising:
determining a three-dimensional (3D) distance vector in a 3D field of view of a near-eye display device, the vector extending to a 3D point of focus of a user of the near-eye display device and from one of a predetermined reference point on the near-eye display device and an eye of the user, which eye is spaced apart from and not bonded to the near-eye display device, the 3D point of focus of the user being out in a corresponding 3D region of displayable scenery in the 3D display field of view, beyond the near-eye display device;
based on the determined distance vector that extends to the user's 3D point of focus and based on predetermined low perception criteria, determining whether at least some of image data of the corresponding 3D region of displayable scenery qualifies as lossless priority data which does not satisfy the low perception criteria;
based on the determined distance vector, determining whether at least some of other image data of the corresponding 3D region of displayable scenery qualifies as allowed loss data which satisfies the low perception criteria;
responsive to at least some of the image data qualifying as allowed loss data, first transmitting to the near-eye display device as first updating data, the allowed loss data, said first transmitting using one or more communication techniques allowing for lossy transmission; and
responsive to at least some of the other image data qualifying as lossless priority data, second transmitting to the near-eye display device as second updating data, the lossless priority image data using one or more communication techniques satisfying lossless transmission criteria.

2. The method of claim 1 wherein transmitting to the near-eye display device the lossless priority image data includes use of a computer system using one or more communication techniques satisfying lossless transmission criteria and the method further comprises:
encoding the lossless priority image data with one or more error correction techniques for satisfying lossless transmission criteria; and
wherein transmitting from the computer system to the near-eye display device the allowed loss data using one or more communication techniques allowing for lossy transmission further comprises
encoding at least some of the allowed loss image data by setting one or more data integrity header bits of packets and omitting redundant data for the at least some of the allowed loss image data with a level of error correction allowing for lossy transmission.

3. The method of claim 2 wherein the one or more communication techniques for satisfying lossless transmission criteria includes forward error correction (FEC).

4. The method of claim 1 wherein the lossless transmission criteria comprises a first bit error rate satisfying a probability criteria of error-free transmission of the lossless priority image data and allowing for lossy transmission comprises allowing for a second bit error rate which does not satisfy the probability criteria.

5. The method of claim 1 wherein transmitting from the computer system to the near-eye display device the allowed loss data using one or more communication techniques allowing for lossy transmission further comprises
transmitting the allowed loss image data over a communication channel not satisfying a noise criteria of the lossless transmission criteria.

6. The method of claim 1 wherein transmitting to the near-eye display device the lossless priority image data includes use of a computer system using one or more communication techniques satisfying lossless transmission criteria and the method further comprises:
modulating the lossless priority image data with a modulation technique satisfying a bit error rate criteria of the lossless transmission criteria; and
wherein transmitting from the computer system to the near-eye display device the allowed loss data using one or more communication techniques allowing for lossy transmission further comprises:
modulating the at least the portion of the allowed loss image data with a modulation technique not satisfying the bit error rate criteria of the lossless transmission criteria.

7. The method of claim 6 wherein modulating the lossless priority image data with the modulation technique satisfying the bit error rate criteria of the lossless transmission criteria further comprises
modulating the lossless priority image data with a first constellation encoding scheme of bits per symbol satisfying the bit error rate; and
wherein modulating the at least the portion of the allowed loss image data with the modulation technique not satisfying the bit error rate criteria of the lossless transmission criteria further comprises
modulating at least the portion of the allowed loss image data with a second constellation encoding scheme having more bits per symbol than the first constellation encoding scheme and which does not satisfy the bit error rate.

8. The method of claim 1 further comprising
the lossless priority image data comprises user focal region image data for display in a user focal region, and
the allowed loss image data comprises at least some secondary image data for display outside the user focal region; and
the secondary image data has a lower image resolution than the user focal region image data.

9. The method of claim 8 further comprising
the secondary image data has a different display update rate than the user focal region image data.

10. A method for reducing display update time for a near-eye display device comprising:
receiving image data representing a three-dimensional (3D) field of view of the near-eye display device;
identifying user focal region image data within the received image data, the received image data being configured for providing an update to a 3D display by the near-eye display device to spaced away eyes of a user, the eyes not being bonded with the near-eye display device, the identified user focal region image data corresponding to a 3D user focal region in the 3D field of view of the near-eye display device, the 3D user focal region being disposed beyond a side of the near-eye display device opposite to an eyes facing side of the near-eye display device
transmitting the identified user focal region image data to the near-eye display device using one or more communication techniques satisfying lossless transmission criteria;
identifying within the received image data representing the 3D field of view, at least some secondary image data as corresponding to a 3D secondary region outside the 3D user focal region;
transmitting the identified at least some secondary image data to the near-eye display device using one or more communication techniques allowing for lossy transmission; and
causing an updating of the 3D display by the near-eye display device through use of the transmitted user focal region image data and the transmitted at least some secondary image data.

11. The method of claim 10 wherein the at least some secondary image data has a lower image resolution than an image resolution of the user focal region image data.

12. The method of claim 10 further comprising:
identifying natural user input in image data captured by one or more image capture devices of the near-eye display device; and
determining the user focal region in the field of view of the near-eye display device based on the natural user input.

13. The method of claim 12 wherein identifying natural user input in the image data captured by one or more image capture devices of the near-eye display device further comprises capturing eye movements with an eye tracking camera.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Tantos Andras
• Fleck Rod G
• Perry Jedd
• Bohn David D

Classifications

• H04N21/234354

  by altering signal-to-noise ratio parameters, e.g. requantization · CPC title

• H04N19/67

  involving unequal error protection [UEP], i.e. providing protection according to the importance of the data · CPC title

• H04N19/167

  Position within a video image, e.g. region of interest [ROI] · CPC title

• H03M13/03

  Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words · CPC title

• H04N21/2383

  Channel coding {or modulation} of digital bit-stream, e.g. QPSK modulation (arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002) · CPC title