Techniques for selecting frames for decode in media player

Patent metadata

Field	Value
Publication number	US-10199074-B2
Application number	US-201715612947-A
Country	US
Kind code	B2
Filing date	Jun 2, 2017
Priority date	Jun 2, 2017
Publication date	Feb 5, 2019
Grant date	Feb 5, 2019

Abstract

Techniques are disclosed for selecting frames for decode and display during different playback modes of a media player. Prediction dependencies may be estimated among frames from a sample table of a media item identifying dependency state among frames in the media item. Based on a playback rate of a media player, a collection of frames may be identified from the media item that have presentation times within a display refresh time of the player. A frame may be selected for decode and display during the display refresh time based on the estimated prediction dependencies. The selected frame may be decoded for display during the player display refresh time.

First claim

The invention claimed is:

1. A method comprising:
estimating, from a media item, coding prediction dependencies among frames of the media item;
based on a playback rate of a media player, identifying a collection of the frames from the media item that have presentation times within a display refresh time of the media player; and
selecting a frame of the collection for decode and display during the display refresh time based on the estimated prediction dependencies.

2. The method of claim 1 , wherein the estimation is based on a sample table for the media item identifying dependency state among frames in the media item.

3. The method of claim 2 , wherein the sample table includes, for each frame:
data identifying the presentation time of the respective frame;
a flag identifying whether the respective frame relies on another frame in the media item as a prediction reference;
a flag identifying whether the respective frame is a prediction reference for another frame in the media item.

4. The method of claim 1 , wherein the selection of a frame is based on an estimate of relative decoding costs among frames in the collection.

5. The method of claim 1 , wherein the selection of a frame is based on an estimate of presentation times among frames in the collection.

6. The method of claim 1 , wherein the selection comprises:
classifying each frame in the collection based on whether it depends from another frame in the collection; and
if there is only one frame in the collection that does not depend from another frame in the collection, selecting the frame for decode.

7. The method of claim 1 , wherein the selection comprises:
classifying each frame in the collection based on whether other frame(s) from the media item depend from it; and
discarding from selection frames that do not have other frame(s) from the media item depend from them.

8. The method of claim 1 , wherein the selection comprises:
classifying each frame in the collection based on whether frame(s) from other collection(s) depend from it; and
discarding from selection frames that do not have other frame(s) from other collection(s) depend from them.

9. The method of claim 1 , wherein the selection comprises, iteratively:
classifying each frame in the collection in one of two first states based on whether it depends from another frame in the collection;
classifying each frame in the collection in one of two second states based on whether other frame(s) from the media item depend from it; and
selecting one frame from the collection for decoding based on an evaluation of the frames' first state and second state.

10. The method of claim 1 , further comprising decoding the selected frame.

11. The method of claim 1 , further comprising repeating the identifying and selecting steps over a plurality of playback rates that vary between a normal playback rate and a slow motion playback rate.

12. The method of claim 1 , further comprising repeating the identifying and selecting steps over a plurality of playback rates that vary.

13. A non-transitory computer readable medium storing program instructions thereon that, when executed by a processing device, cause the device to:
estimate, from a media item, coding prediction dependencies among frames of the media item;
based on a playback rate of a media player, identify a collection of the frames from the media item that have presentation times within a display refresh time of the media player;
select a frame of the collection for decode and display during the display refresh time based on the estimated prediction dependencies.

14. The medium of claim 13 , wherein the estimation is based on a sample table for the media item identifying dependency state among frames in the media item.

15. The medium of claim 14 , wherein the sample table includes, for each frame:
data identifying the presentation time of the respective frame;
a flag identifying whether the respective frame relies on another frame in the media item as a prediction reference;
a flag identifying whether the respective frame is a prediction reference for another frame in the media item.

16. The medium of claim 13 , wherein the selection of a frame is based on an estimate of relative decoding costs among frames in the collection.

17. The medium of claim 13 , wherein the selection of a frame is based on an estimate of presentation times among frames in the collection.

18. The medium of claim 13 , wherein the selection comprises:
classifying each frame in the collection based on whether it depends from another frame in the collection; and
if there is only one frame in the collection that does not depend from another frame in the collection, selecting the frame for decode.

19. The medium of claim 13 , wherein the selection comprises:
classifying each frame in the collection based on whether other frame(s) from the media item depend from it; and
discarding from selection frames that do not have other frame(s) from the media item depend from them.

20. The medium of claim 13 , wherein the selection comprises:
classifying each frame in the collection based on whether frame(s) from other collection(s) depend from it; and
discarding from selection frames that do not have other frame(s) from other collection(s) depend from them.

21. The medium of claim 13 , wherein the selection comprises, iteratively:
classifying each frame in the collection in one of two first states based on whether it depends from another frame in the collection;
classifying each frame in the collection in one of two second states based on whether other frame(s) from the media item depend from it; and
selecting frame(s) from the collection for decoding based on an evaluation of the frames' first state and second state.

22. The medium of claim 13 , wherein the program instructions cause the device to repeat the identifications and selections over a plurality of playback rates that vary between a normal playback rate and a slow motion playback rate.

23. A system, comprising:
a display;
a processor; and
a memory storing program instructions thereon that, when executed by the processor, cause the processor to:
estimate, from a media item, coding prediction dependencies among frames of the media item;
based on a playback rate of the decoder, identify a collection of the frames from the media item that have presentation times within a display refresh time of the display;
select a frame of the collection for decode and display during the display refresh time based on the estimated prediction dependencies.

24. The system of claim 23 , wherein the estimation is based on a sample table for the media item identifying dependency state among frames in the media item.

25. The system of claim 24 , wherein the sample table includes, for each frame:
data identifying the presentation time of the respective frame;
a flag identifying whether the respective frame relies on another frame in the media item as a prediction reference;
a flag identifying whether the respective frame is a prediction reference for another frame in the media item.

26. The system of claim 23 , wherein the processor selects the frame based on an estimate of relative decoding costs among frames in the collection.

27. The system of claim 23 , wherein the processor selects the frame based on an estimate of presentation times among frames in the colle

Assignees

• Apple Inc

Inventors

• Decoodt Jerome
• Bushell John Samuel

Classifications

• H04N19/44 Primary

  Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder · CPC title

• H04N21/8455

  involving pointers to the content, e.g. pointers to the I-frames of the video stream · CPC title

• H04N21/4402

  involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)} · CPC title

• G11B27/005

  Reproducing at a different information rate from the information rate of recording (for television signals H04N5/783) · CPC title

• G11B27/34 Primary

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title