Dynamic selection of candidate bitrates for video encoding

Patent metadata

Field	Value
Publication number	US-12225252-B2
Application number	US-202318179281-A
Country	US
Kind code	B2
Filing date	Mar 6, 2023
Priority date	Mar 6, 2023
Publication date	Feb 11, 2025
Grant date	Feb 11, 2025

Abstract

In some embodiments, a method generates a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video. Also, the method generates a second representation of a second relationship between bitrate and quality based on second features of a second portion of a video. The first representation is analyzed to determine a first list of bitrates for the first portion of video and the second representation is analyzed to determine a second list of bitrates for the second portion of video. The first list of bitrates is different from the second list of bitrates. The method outputs the first list of bitrates for use encoding the first portion of video and the second list of bitrates for use encoding the second portion of video.

First claim

What is claimed is:

1. A method comprising:
generating, by a computing device, a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video;
generating, by the computing device, a second representation of a second relationship between bitrate and quality based on second features of a second portion of the video;
analyzing, by the computing device, the first representation to determine a first list of bitrates for the first portion of video and analyzing the second representation to determine a second list of bitrates for the second portion of video, wherein the first list of bitrates is different from the second list of bitrates, wherein analyzing the first representation or analyzing the second representation comprises:
generating a list of potential bitrates based on the first representation or the second representation, and
refining the list of potential bitrates based to a quality associated with the potential bitrates to determine the first list of bitrates or the second list of bitrates, wherein refining the list of potential bitrates comprises:
adding a first potential bitrate to the list of potential bitrates; and

outputting, by the computing device, the first list of bitrates for encoding the first portion of video and the second list of bitrates for encoding the second portion of video.

2. The method of claim 1 , wherein:
generating the first representation comprises generating a first prediction of the first relationship for bitrate and quality, and
generating the second representation comprises generating a second prediction of the second relationship for bitrate and quality.

3. The method of claim 1 , wherein:
generating the first representation comprises:
inputting the first features into a prediction network; and
generating a first prediction of the first representation based on the first features, and

generating the second representation comprises:
inputting the second features into the prediction network; and
generating a second prediction of the second representation based on the second features.

4. The method of claim 1 , wherein refining the list of potential bitrates comprises:
removing a first potential bitrate from the list of potential bitrates.

5. The method of claim 4 , wherein removing the potential bitrate comprises:
determining a second potential bitrate;
comparing a first quality of the first potential bitrate to a second quality of the second potential bitrate; and
determining whether to remove the first potential bitrate based on the comparing.

6. The method of claim 5 , wherein the first potential bitrate is removed from the list of potential bitrates when a difference between the first quality and the second quality meets a threshold.

7. The method of claim 1 , wherein adding the potential bitrate comprises:
determining a second potential bitrate and a third potential bitrate that are in the list of potential bitrates;
comparing a first quality of the second potential bitrate to a second quality of the third potential bitrate; and
determining whether to add the first potential bitrate based on the comparing.

8. The method of claim 7 , wherein the first potential bitrate is added when a difference between the first quality and the second quality meets a threshold.

9. The method of claim 1 , wherein analyzing the first representation or analyzing the second representation comprises:
analyzing a plurality of first representations for a plurality of segments in the first portion of video or the second portion of video;
determining a first minimum bitrate and a first maximum bitrate for each of the plurality of first representations; and
determining a second minimum bitrate and a second maximum bitrate for the first portion of video or the second portion of video based on the first minimum bitrate and the first maximum bitrate for each of the plurality of first representations.

10. The method of claim 9 , wherein analyzing the first representation or analyzing the second representation comprises:
generating the list of potential bitrates based on the second minimum bitrate and the second maximum bitrate, wherein potential bitrates in the list of potential bitrates are analyzed to determine the first list of bitrates or the second list of bitrates.

11. The method of claim 10 , wherein analyzing the first representation or analyzing the second representation comprises:
determining potential removal candidate bitrates for each of the plurality of first representations, wherein a potential removal candidate bitrate is a potential removal from the list of potential bitrates; and
determining whether to remove a potential removal candidate bitrate based on the potential removal candidate bitrate being included as a potential removal candidate in one or more of each of the plurality of first representations.

12. The method of claim 10 , wherein analyzing the first representation or analyzing the second representation comprises:
determining potential added candidate bitrates for each of the plurality of first representations, wherein a potential added candidate is a potential addition to the list of potential bitrates; and
determining whether to add a potential bitrate based on the potential added candidate bitrate being included as a potential added candidate in one or more of each of the plurality of first representations.

13. The method of claim 1 , wherein:
the first list of bitrates includes a bitrate that is not included in the second list of bitrates.

14. The method of claim 1 , further comprising:
encoding the first portion of video using the first list of bitrates to generate a plurality of first encoded portions for the first portion of video; and
encoding the second portion of video using the second list of bitrates to generate a plurality of second encoded portions for the second portion of video.

15. The method of claim 14 , further comprising:
selecting an encoded segment from the plurality of first encoded portions for a first profile in a profile ladder; and
selecting an encoded segment from the plurality of second encoded portions for the first profile in the profile ladder.

16. A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for:
generating a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video;
generating a second representation of a second relationship between bitrate and quality based on second features of a second portion of the video;
analyzing the first representation to determine a first list of bitrates for the first portion of video and analyzing the second representation to determine a second list of bitrates for the second portion of video, wherein the first list of bitrates is different from the second list of bitrates, wherein analyzing the first representation or analyzing the second representation comprises:
generating a list of potential bitrates based on the first representation or the second representation, and
refining the list of potential bitrates based on a quality associated with the potential bitrates to determine the first list of bitrates or the second list of bitrates, wherein refining the list of potential bitrates comprises:
adding a first potential bitrate to the list of potential bitrates; and

outputting the first list of bitrates for use encoding the first portion of video and t

Assignees

• Beijing Yojaja Software Tech Development Co Ltd
• Disney Entpr Inc

Inventors

• Liu Chen
• Zhang Wenhao
• Labrozzi Scott
• Xue Yuanyi
• Huangfu Xuchang
• Liu Xiaobo

Classifications

• H04N21/8456

  by decomposing the content in the time domain, e.g. in time segments · CPC title

• H04N21/2662

  Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities · CPC title

• H04N21/23418

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• G06N20/00

  Machine learning · CPC title

• H04N19/184

  the unit being bits, e.g. of the compressed video stream · CPC title