Power aware video decoding and streaming

Patent metadata

Field	Value
Publication number	US-10154258-B2
Application number	US-201313937678-A
Country	US
Kind code	B2
Filing date	Jul 9, 2013
Priority date	Jul 9, 2012
Publication date	Dec 11, 2018
Grant date	Dec 11, 2018

Abstract

Methods and systems are disclosed for a mobile device to decode video based on available power and/or energy. For example, the mobile device may receive a media description file (MDF) from for a video stream from a video server. The MDF may include complexity information associated with a plurality of video segments. The complexity information may be related to the amount of processing power to be utilized for decoding the segment at the mobile device. The mobile device may determine at least one power metric for the mobile device. The mobile device may determine a first complexity level to be requested for a first video segment based on the complexity information from the MDF and the power metric. The mobile device may dynamically alter the decoding process to save energy based on the detected power/energy level.

First claim

What is claimed:

1. A method for a mobile device to request a video stream from a video server, the method comprising:
receiving a media description file (MDF) for the video stream, wherein the video stream is divided into a plurality of video segments, and wherein the MDF comprises video decoding complexity information associated with a first video segment, and wherein the video decoding complexity information for the first video segment indicates relative video decoding power consumption information for a plurality of complexity levels at which the first video segment can be requested;
calculating an amount of power consumed by the mobile device to decode a previous video segment at a reference complexity level of the plurality of complexity levels;
determining an amount of energy remaining in a battery of the mobile device;
calculating, using the video decoding complexity information, a power reduction ratio for the first video segment relative to the previous video segment at the reference complexity level;
determining to request the first video segment at a first complexity level of the plurality of complexity levels based on the amount of energy remaining in the battery of the mobile device, the power reduction ratio, and the amount of power consumed by the mobile device to decode the previous video segment at the reference complexity level; and
requesting the first video segment at the first complexity level from the video server.

2. The method as in claim 1 , wherein the video decoding complexity information is provided per video segment in the MDF.

3. The method as in claim 1 , wherein the MDF corresponds to a media presentation description (MPD) file.

4. The method as in claim 1 , wherein the MDF further comprises bit rate information for the plurality of video segments, and the bit rate information indicates a plurality of bit rate levels at which at least the first video segment can be requested.

5. The method as in claim 1 , wherein the relative video decoding power consumption information for the plurality of complexity levels at which the first video segment can be requested indicates relative amounts of energy that would be used to decode the first video segment at each of the plurality of complexity levels, and wherein the plurality of complexity levels correspond to a video encoder varying one or more encoding parameters in order to encode the first video segment, and the one or more encoding parameters correspond to at least one of precision of motion information, motion compensation block size, transform size, in-loop filters, or a cost threshold for skipping coefficient coding.

6. The method as in claim 1 , further comprising estimating, based on the amount of energy remaining in the battery of the mobile device, a power available for decoding the first video segment, wherein determining to request the first video segment at the first complexity level of the plurality of complexity levels is further based on the estimated power available for decoding the first video segment.

7. The method as in claim 1 , wherein determining to request the first video segment at the first complexity level comprises determining the first complexity level such that decoding the first video segment results in using approximately an amount of energy allocated for decoding the first video segment.

8. The method as in claim 1 , wherein determining to request the first video segment at the first complexity level comprises selecting the first complexity level based on a power allocated for the previous video segment, a complexity level used to decode the previous video segment, and a power allocated for decoding the first video segment.

9. The method as in claim 1 , further comprising determining whether a region of the first video segment comprises high frequency components that exceed a high frequency threshold, and applying a first interpolation filter during motion compensation for the region on condition that the high frequency components exceed the high frequency threshold or applying a second interpolation filter for motion compensation for the region on condition that the high frequency components do not exceed the high frequency threshold, wherein the second interpolation filter is associated with a lower cutoff frequency that the first interpolation filter.

10. The method as in claim 1 , further comprising performing deblocking for reference pictures in the first video segment, and refraining from performing deblocking for non-reference pictures.

11. A mobile device for requesting a video stream from a video server based on power conditions, the mobile device comprising:
a transceiver configured to receive a media description file (MDF) for the video stream, wherein the video stream is divided into a plurality of video segments, and wherein the MDF comprises video decoding complexity information associated with a first video segment, and wherein the video decoding complexity information for the first video segment indicates relative video decoding power consumption information for a plurality of complexity levels at which the first video segment can be requested; and
a processor configured to:
calculate an amount of power consumed by the mobile device to decode a previous video segment at a reference complexity level of the plurality of complexity levels;
determine an amount of energy remaining in a battery of the mobile device;
calculate, using the video decoding complexity information, a power reduction ratio for the first video segment relative to the previous video segment at the reference complexity level; and
determine to request the first video segment at a first complexity level of the plurality of complexity levels based on the amount of energy remaining in the battery of the mobile device, the power reduction ratio, and the amount of power consumed by the mobile device to decode the previous video segment at the reference complexity level.

12. The mobile device as in claim 11 , wherein the processor is further configured to determine a parameter or method to apply for decoding the first video segment based on the video decoding complexity information indicated in the MDF.

13. The mobile device as in claim 11 , wherein the processor is further configured to determine an interpolation filter to apply for motion compensation of a given region based on a frequency analysis of a sampling of pixel within the given region.

14. The mobile device as in claim 13 , wherein the processor is further configured to utilize a first interpolation filter with a lower cutoff frequency than a second interpolation filter for higher layer temporal prediction for regions with high frequency components below a specified threshold, and use the second interpolation filter for lower layer temporal prediction even if high frequency components of the lower layer are below the threshold.

15. The mobile device as in claim 13 , wherein the processor is further configured to utilize a first interpolation filter with a lower cutoff frequency than a second interpolation filter for regions with high frequency components below a specified threshold except for at least inter blocks where a non-causal neighbor of the inter block comprise one or more intra coded blocks.

16. The mobile device as in claim 11 , wherein the processor is further configured to determine a resolution to request for a subsequent segment based on the relative video decoding power consumption information indicated in the MDF, the amount of energy remaining in the battery of the mobile device, and the amount of power consumed by the mobile device to de

Assignees

• Vid Scale Inc

Inventors

• He Yuwen
• Ye Yan
• He Yong
• Mcclellan George W
• Ryu Eun

Classifications

• H04N21/414

  Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance · CPC title

• H04N21/234309 Primary

  by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo (conversion of standards in analog television systems H04N7/01) · CPC title

• H04N19/44

  Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder · CPC title

• H04N21/23439

  for generating different versions · CPC title

• H04N21/41407 Primary

  embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57) · CPC title