Video frame playback scheduling

Patent metadata

Field	Value
Publication number	US-9998634-B2
Application number	US-201414530967-A
Country	US
Kind code	B2
Filing date	Nov 3, 2014
Priority date	Nov 3, 2014
Publication date	Jun 12, 2018
Grant date	Jun 12, 2018

Abstract

Systems and techniques are provided for video frame playback scheduling. A presentation time for a frame of a video may be received. An offset for the frame of video may be determined based on succeeding frames of the video and Vsync pulse times. The presentation time for the frame of the video may be adjusted based on the offset to generate an adjusted presentation time that is closer to a halfway point between a first Vsync pulse and a second Vsync pulse than the presentation time. A cumulative time correction may be tracked based on adjustments made to the presentation times for frames of the video. A magnitude of the cumulative time correction may be determined to exceed the magnitude of a positive or negative threshold. The cumulative time correction may be added to or subtracted from based on the determination that the magnitude exceeds the positive or negative threshold.

First claim

The invention claimed is:

1. A computer-implemented method performed by a data processing apparatus, the method comprising:
receiving a presentation time for a frame of a video;
determining an offset for the frame of video based on at least one succeeding frame of the video and at least one Vsync pulse time;
adjusting the presentation time for the frame of the video based on the offset to generate an adjusted presentation time that is closer to a halfway point between a first Vsync pulse and a second Vsync pulse than the presentation time;
tracking a cumulative time correction based on adjustments made to the presentation times for one or more frames of the video;
determining that a magnitude of the cumulative time correction exceeds the magnitude of a positive or negative threshold; and
adding or subtracting from the cumulative time correction based on the determination that the magnitude exceeds the positive or negative threshold.

2. The computer-implemented method of claim 1 , further comprising:
sending the frame of video to a renderer based on the adjusted presentation time.

3. The computer-implemented method of claim 1 , wherein the offset is based on 12 succeeding frames.

4. The computer-implemented method of claim 1 , wherein the offset is an average estimated offset.

5. The computer-implemented method of claim 4 , wherein the average estimated offset is determined by averaging, over the next N succeeding frames to the frame of the video, a presentation time plus a cumulative time correction for each frame, a tracked time difference between successive frames multiplied by the frame number within the set of frames being evaluated minus a tracked base time for a previous Vsync pulse, modulus a tracked time difference between successive Vsync pulses, wherein the tracked time difference between successive Vsync pulses is a Vsync period.

6. The computer-implemented method of claim 4 , wherein adjusting the presentation time further comprises:
dividing a Vsync period by two;
subtracting the offset from the result of dividing the Vsync period by two to determine a time correction;
applying the time correction to the presentation time of the frame of the video to generate the adjusted presentation time; and
aligning the adjusted presentation time halfway between the first Vsync pulse and second Vsync pulse, wherein the first Vsync pulse and the second Vsync pulse are consecutive.

7. The computer-implemented method of claim 1 , wherein the offset is a weighted average estimated offset.

8. The computer-implemented method of claim 1 , wherein at least half of a Vsync period is subtracted from the cumulative time correction when the cumulative time correction is positive and is greater than the positive threshold.

9. The computer-implemented method of claim 8 , wherein the at least half of a Vsync period is subtracted from the cumulative time correction to effect the display of a frame of the video that is designated to be displayed for more than a minimum number of Vsync pulses.

10. The computer-implemented method of claim 1 , wherein at least half of a Vsync period is added to the cumulative time correction when the cumulative time correction is negative and is less than the negative threshold.

11. The computer-implemented method of claim 10 , wherein the at least half of a Vsync period is added to the cumulative time correction to effect the display of a frame of the video that is designated to be displayed for less than a maximum number of Vsync pulses.

12. The computer-implemented method of claim 1 , wherein the positive threshold is between 25% of Vsync period and 100% of a Vsync period, and wherein the negative threshold is between negative 25% of a Vsync period and negative 100% of a Vsync period.

13. A computer-implemented system for video frame playback scheduling:
a media player adapted to decode frames from a video item;
a renderer adapted to receive frames of the video item decoded by the media player;
a Vsync generator adapted to generate Vsync pulses; and

a scheduler adapted to receive a presentation time for a frame of a video, determine an offset for the frame of video based on at least one succeeding frame of the video and at least one Vsync pulse time, adjust the presentation time for the frame of the video based on the offset to generate an adjusted presentation time that is closer to a halfway point between a first Vsync pulse and a second Vsync pulse than the presentation time, track a cumulative time correction based on adjustments to the presentation times for on or more frames of the video, and add or subtract at least half of a Vsync period to or from the cumulative time correction when the cumulative time correction has a magnitude greater than the magnitude of a positive or negative threshold.

14. The computer-implemented system of claim 13 , wherein the scheduler is adapted to adjust the presentation times for the frames of the video such that adjusted presentation times for the frames of the video are closer to halfway in between the expected times for successive Vsync pulses than the presentation times for the frames of the video.

15. The computer-implemented system of claim 13 , wherein the scheduler is adapted to subtract the at least half of a Vsync period from the cumulative time correction when the cumulative time correction is positive and exceeds the positive threshold.

16. The computer-implemented system of claim 13 , wherein the scheduler is adapted to add the at least half of a Vsync period to the cumulative time correction when the cumulative time correction is negative and is less than the negative threshold.

17. The computer-implemented system of claim 13 , wherein the scheduler is adapted to adjust the presentation times based on an average estimated offset.

18. The computer-implemented system of claim 17 , wherein the average estimated offset for a frame is based on a presentation time, a cumulative time correction, a tracked time difference between successive frames multiplied by the frame number, a tracked base time for a previous Vsync pulse, and a tracked time difference between successive Vsync pulses, for the next N succeeding frames to the frame.

19. A system comprising: one or more computers and one or more storage devices storing instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving a presentation time for a frame of a video;
determining an offset for the frame of video based on a presentation for at least one succeeding frame of the video and at least one Vsync pulse time;
adjusting the presentation time for the frame of the video based on the offset to generate an adjusted presentation time that is closer to a halfway point between a first Vsync pulse and a second Vsync pulse than the presentation time;
tracking a cumulative time correction based on adjustments made to the presentation times for one or more frames of the video;
determining that a magnitude of the cumulative time correction exceeds the magnitude of a positive or negative threshold; and
adding or subtracting from the cumulative time correction based on the determination that the magnitude exceeds the positive or negative threshold.

Assignees

• Google Llc

Inventors

• Molnar Lajos

Classifications

• H04N5/04 Primary

  Synchronising (for television systems using pulse code modulation H04N7/56) · CPC title

• G11B27/10

  Indexing; Addressing; Timing or synchronising; Measuring tape travel · CPC title

• H04N5/06

  Generation of synchronising signals · CPC title

• H04N5/93

  Regeneration of the television signal or of selected parts thereof · CPC title