Avoidance of color breakup in late-stage re-projection

Patent metadata

Field	Value
Publication number	US-9874932-B2
Application number	US-201514682860-A
Country	US
Kind code	B2
Filing date	Apr 9, 2015
Priority date	Apr 9, 2015
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

One embodiment provides a method to display video such as computer-rendered animation or other video. The method includes assembling a sequence of video frames featuring a moving object, each video frame including a plurality of subframes sequenced for display according to a schedule. The method also includes determining a vector-valued differential velocity of the moving object relative to a head of an observer of the video. At a time scheduled for display of a first subframe of a given frame, first-subframe image content transformed by a first transform is displayed. At a time scheduled for display of the second subframe of the given frame, second-subframe image content transformed by a second transform is displayed. The first and second transforms are computed based on the vector-valued differential velocity to mitigate artifacts.

First claim

The invention claimed is:

1. Enacted on a display device, a method to display video, comprising:
assembling a sequence of consecutive video frames featuring a moving object, each video frame including a plurality of subframes sequenced for display according to a schedule;
determining a vector-valued differential velocity of the moving object relative to a head of an observer of the video;
at a time scheduled for display of a first subframe of a given frame, displaying first-subframe image content transformed by a first transform within a first region of the first subframe and by a second transform within a second region of the first subframe, the first region being a region localized on the moving object, wherein the first transform is computed based on the vector-valued differential velocity; and
at a time scheduled for display of a second subframe of the given frame, displaying second-subframe image content transformed by a third transform within the first region of the second subframe and by a fourth transform within the second region of the second subframe, wherein the third transform is computed based on the vector-valued differential velocity, and
wherein displaying the first- and second-subframe image content includes illuminating first and second pixels, respectively of the first- and second-subframe image content, each corresponding to a given locus of the object within the first region, and wherein the first and third transforms are computed so that light from the first and second pixels is imaged at substantially a same location on a retina of the observer.

2. The method of claim 1 , wherein the video is displayed in a field-sequential color mode, and wherein the first subframe provides image content of a first color component and the second subframe provides image content of a second color component.

3. The method of claim 1 , wherein the video is displayed in a rolling-shutter mode, and wherein the first subframe provides image content limited to a first row of pixels and the second subframe provides image content limited to a second row of pixels.

4. The method of claim 1 , wherein determining the vector-valued differential velocity includes quantifying a head movement of the observer via motion sensor data.

5. The method of claim 1 , wherein determining the vector-valued differential velocity further comprises tracking a gaze of the observer to determine that the gaze of the observer is following the moving object.

6. The method of claim 1 , wherein determining the vector-valued differential velocity further comprises determining a vector-valued velocity of the moving object via an application displaying the moving object, and determining the vector-valued differential velocity based upon the vector-valued velocity of the moving object.

7. The method of claim 1 wherein displaying the first- and second-subframe image content further includes illuminating third and fourth pixels, respectively of the first- and second-subframe image content, each corresponding to a given locus of the object within the second region, and wherein the second and fourth transforms are computed so that light from the third and fourth pixels is imaged at substantially a same location on the retina of the observer.

8. A field-sequential color display system comprising:
a video-assembly engine configured to assemble a sequence of consecutive video frames featuring a moving object, each video frame including a plurality of color subframes sequenced for display according to a schedule;
a gaze-estimation engine configured to track a gaze of the observer to determine that the gaze of the observer is following the moving object;
a late-stage re-projection engine configured to compute a vector-valued differential velocity of the moving object relative to a movable head of an observer;
a regional correction engine configured to transform different regions of a given color subframe by different transforms, the different regions including a region localized on the moving object; and
a video-display engine configured to, at a time scheduled for display of a first color subframe of a given frame, display first-color image content transformed by a first transform in a region of the moving object, and, at a time scheduled for display of a second color subframe of the given frame, displaying second-color image content transformed by a second transform in the region of the moving object, the first and second transforms being computed based on the vector-valued differential velocity.

9. The system of claim 8 , wherein the video-display engine includes right and left display optics arranged for stereoscopic video display.

10. The system of claim 8 , wherein the video-display engine is configured to, in displaying the first- and second-subframe image content, illuminate first and second pixels, respectively of the first- and second-subframe image content, each corresponding to a given locus of the object, and wherein the first and second transforms are computed so that light from the first and second pixels is imaged at substantially a same location on a retina of the observer.

11. The system of claim 8 , wherein the video-assembly engine, the late-stage re-projection engine, and the video-display engine are components of a wearable display device configured to be worn on the head.

12. The system of claim 11 , further comprising a motion sensor configured to quantify a head movement of the observer.

13. The system of claim 11 , wherein the wearable display device includes a see-through optic configured to transmit real, external imagery which is presented to the observer in combination with the video.

14. The system of claim 8 , further comprising a gaze-estimation engine configured to track a gaze of the observer to determine that the gaze of the observer is following the moving object.

15. The system of claim 14 , wherein the gaze-estimation engine is configured to determine a vector-valued velocity of the moving object via gaze tracking.

16. The system of claim 8 , wherein determining the vector-valued differential velocity comprises determining a vector-valued velocity of the moving object from an application displaying the object.

17. A field-sequential color display system comprising:
a video-assembly engine configured to assemble a sequence of consecutive predetermined or computer rendered video frames featuring a moving object, each video frame including a plurality of color subframes sequenced for display according to a schedule;
a late-stage re-projection engine configured to compute a vector-valued differential velocity of the moving object relative to a movable head of an observer;
a regional correction engine configured to transform different regions of a given color subframe by different transforms, the different regions including a region localized on the moving object; and
a video-display engine configured to, at a time scheduled for display of a first color subframe of a given frame, display first-color image content transformed by a first transform in a region of the moving object, and, at a time scheduled for display of a second color subframe of the given frame, displaying second-color image content transformed by a second transform in the region of the moving object, the first and second transforms being computed based on the vector-valued differential velocity.

18. The system of claim 17 , wherein the moving object is selected from a plurality of moving objects in the video, and wherein the late-stage re-projection engine is configured to receive input identifying

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Crisler Matthew Calbraith
• Held Robert Thomas
• Latta Stephen
• Michail Ashraf Ayman
• Shetter Martin
• Tomlin Arthur

Classifications

• H04N13/0422

  Electricity · mapped topic

• H04N13/044

  Electricity · mapped topic

• H04N13/0459

  Electricity · mapped topic

• G02B27/017

  Head mounted · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title