Scanning display system in head-mounted display for virtual reality

Patent metadata

Field	Value
Publication number	US-9824498-B2
Application number	US-201514627406-A
Country	US
Kind code	B2
Filing date	Feb 20, 2015
Priority date	Dec 30, 2014
Publication date	Nov 21, 2017
Grant date	Nov 21, 2017

Abstract

Methods, systems, and computer programs are presented for the presentation of images in a head-mounted display (HMD). One HMD includes a screen, a processor, inertial sensors, a motion tracker module, and a display adjuster module. The motion tracker tracks motion of the HMD based on inertial data from the inertial sensors, and the display adjuster produces modified display data for an image frame to be scanned to the screen if the motion of the HMD is greater than a threshold amount of motion. The display data includes pixel values to be scanned to rows in sequential order, and the modified display data includes adjusted pixel values for pixels in a current pixel row of the image frame to compensate for the distance traveled by the HMD during a time elapsed between scanning a first pixel row of the image frame and scanning the current pixel row of the image frame.

First claim

What is claimed is:

1. A head-mounted display (HMD) comprising:
a screen including a plurality of pixel rows;
a processor;
inertial sensors operable to generate inertial data;
a motion tracker module operable to be executed by the processor, the motion tracker module operable to track a motion of the head-mounted display (HMD) based on the inertial data; and
a display adjuster module operable to be executed by the processor, the display adjuster module operable to produce modified display data for an image frame to be scanned to the screen if the motion of the HMD is greater than a threshold amount of motion, the display data including pixel values to be scanned to the plurality of pixel rows in sequential order,
wherein the modified display data includes adjusted pixel values for pixels in a current pixel row of the image frame to compensate for a distance traveled by the HMD during a time elapsed between scanning a first pixel row of the image frame and scanning the current pixel row of the image frame,
wherein the adjusted pixel values are pixel values of virtual pixels located at an offset distance of the pixels in the current pixel row, the offset distance being based on the distance traveled in a direction of travel by the HMD during the time elapsed, wherein a new pixel value for a corresponding pixel in the current pixel row is interpolated from pixel data from a plurality of pixels that neighbor a corresponding virtual pixel,
wherein when the corresponding virtual pixel is located beyond an edge of the screen, the new pixel value is interpolated from available pixel values closest to the edge of the screen,
wherein the modified display data for the image frame is scanned to the screen to reduce distortion due to the motion of the HMD.

2. The HMD as recited in claim 1 , wherein the offset distance is based on a difference between a projection of the pixel in an absence of motion and a projection of the pixel to compensate for the motion.

3. The HMD as recited in claim 1 , further including:
a communications module operable to receive the display data from a computing device.

4. The HMD as recited in claim 3 , wherein the display adjuster module obtains a new pixel value from a current frame of the display data if available or the display adjuster module obtains the new pixel value from a previous frame of the display data if not available in the current frame.

5. The HMD as recited in claim 1 , further including:
a memory for storing one or more video frames previously presented on the display.

6. The HMD as recited in claim 1 , wherein the threshold amount of motion is an amount of motion where the offset distance is greater than a smallest distance between pixels.

7. The HMD as recited in claim 1 , wherein the display data includes pixel raster data, wherein the data is presented on the display one line of the pixel raster data at a time.

8. The HMD as recited in claim 1 , wherein the new pixel value for the corresponding pixel in the current row comprises a weighted average of the pixel data from the plurality of pixels that neighbor the corresponding virtual pixel.

9. A method comprising:
tracking motion of a head-mounted display (HMD) at the HMD;
receiving, at the HMD, display data from a computing device for presentation of a virtual reality on a screen including a plurality of pixel rows of the HMD;
modifying the display data to produce modified display data for an image frame to be scanned to the screen of the HMD if the motion of the HMD is greater than a threshold amount of motion, the display data including pixel values to be scanned to the plurality of pixel rows in sequential order, wherein the modified display data includes adjusted pixel values for pixels in a current pixel row of the image frame to compensate for a distance traveled by the HMD during a time elapsed between scanning a first pixel row of the image frame and scanning the current pixel row of the image frame, wherein the adjusted pixel values are pixel values of virtual pixels located at an offset distance of the pixels in the current pixel row, the offset distance being based on the distance traveled in a direction of travel by the HMD during the time elapsed, wherein a new pixel value for a corresponding pixel in the current pixel row is interpolated from pixel data from a plurality of pixels that neighbor a corresponding virtual pixel; and
presenting the modified display data on the display of the HMD,
wherein when the corresponding virtual pixel is located beyond an edge of the screen, the new pixel value is interpolated from available pixel values closest to the edge of the screen.

10. The method as recited in claim 9 , wherein the offset distance is associated with a difference between a projection of the pixel in an absence of motion and a projection of the pixel to compensate for the motion.

11. The method as recited in claim 9 , wherein tracking motion of the HMD is performed using inertial sensors, the inertial sensors including one or more of an accelerometer, or a magnetometer, or a gyroscope.

12. The method as recited in claim 9 , wherein the display is one of a virtual retinal display (VRD), or a light-emitting diode display (LED), or an electroluminescent display (ELD), or electronic paper, or o a plasma display panel (PDP), or a liquid crystal display (LCD), or an organic light-emitting diode display (OLED), or a cathode ray tube display (CRT).

13. The method as recited in claim 9 , wherein when the motion of the HMD is horizontal a pixel of the modified display data is changed with data corresponding to another pixel in a same line of the display data.

14. The method as recited in claim 9 , wherein when the motion of the HMD is vertical a pixel of the modified display data is changed with data corresponding to another pixel in a different line of the display data.

15. The method as recited in claim 9 , wherein presenting the modified display data includes starting presentation of raster pixel data of a display frame before the display frame is completely received by the HMD.

16. The method as recited in claim 9 , wherein the threshold amount of motion is an amount of motion that changes a location of a static pixel to a place in the display closer to another pixel, the static pixel being associated with an object that remains static in the virtual reality.

17. A non-transitory computer-readable storage medium storing a computer program, the computer-readable storage medium comprising:
program instructions for tracking motion of a head-mounted display (HMD) at the HMD;
program instructions for receiving, at the HMD, display data from a computing device for presentation of a virtual reality on a screen including a plurality of pixel rows of the HMD;
program instructions for modifying the display data to produce modified display data for an image frame to be scanned to the screen if the motion of the HMD is greater than a threshold amount of motion, the display data including pixel values to be scanned to the plurality of pixel rows in sequential order, wherein the modified display data includes adjusted pixel values for pixels in a current pixel row of the image frame to compensate for a distance traveled by the HMD during a time elapsed between scanning a first pixel row of the image frame and scanning the current pixel row of the image frame, wherein the adjusted pixel values are pixel values of virtual pixels located at an offset distance of the pixels in the current pixel row, the offset distance being based on the distance traveled in a direction of travel by t

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Mallinson Dominic S

Classifications

• H04N13/344

  with head-mounted left-right displays · CPC title

• G09G2360/18

  Use of a frame buffer in a display terminal, inclusive of the display panel · CPC title

• G09G2354/00

  Aspects of interface with display user · CPC title

• G02B2027/011

  comprising device for correcting geometrical aberrations, distortion · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title