Bitrate-adaptive segmentation for video transcoding

Patent metadata

Field	Value
Publication number	US-12256071-B2
Application number	US-202318507279-A
Country	US
Kind code	B2
Filing date	Nov 13, 2023
Priority date	Mar 16, 2022
Publication date	Mar 18, 2025
Grant date	Mar 18, 2025

Abstract

Bitrate-adaptive segmentation is performed for transcoding a video stream uploaded to an online video platform for hosting and later playback to platform users. The video stream is segmented into chunks based on prediction-based bit costs determined for frames of the video stream rather than based on scene changes detected within the video stream. The bitrate-adaptive segmentation includes determining inter-prediction bit costs and intra-prediction bit costs for frames of the video stream based on information indicated within a pass log based on a first pass encoding of the video stream, determining chunk boundaries for segmenting the video stream into a chunk based on the inter-prediction bit costs and the intra-prediction bit costs for the frames, and transcoding the chunk to produce a transcoded video stream.

First claim

What is claimed is:

1. A method, comprising:
determining, for each frame of multiple frames of a video stream to transcode, a prediction bit cost based on information indicated within a pass log for the video stream and one or more other frames of the multiple frames, wherein the one or more other frames are based on whether the frame is inter-predicted or intra-predicted;
segmenting the video stream into chunks determined based on the prediction bit costs associated with pairs of frames of the multiple frames that meet one or more chunk size thresholds; and
transcoding the chunks to produce a transcoded video stream.

2. The method of claim 1 , wherein, for an inter-predicted frame of the multiple frames, the information indicated within the pass log identifies an inter-prediction bit cost for the inter-predicted frame, and the prediction bit cost for the inter-predicted frame is determined based on the inter-prediction bit cost for the inter-predicted frame and an intra-prediction bit cost determined for the inter-predicted frame based on the one or more other frames.

3. The method of claim 2 , wherein the one or more other frames include a nearest intra-predicted frame to the inter-predicted frame in a display order of the video stream, and determining the prediction bit cost for the inter-predicted frame comprises:
predicting the intra-prediction bit cost for the intra-predicted frame based on an intra-predicted bit cost for the nearest intra-predicted frame.

4. The method of claim 1 , wherein, for an intra-predicted frame of the multiple frames, the information indicated within the pass log identifies an intra-prediction bit cost for the intra-predicted frame, and the prediction bit cost for the intra-predicted frame is determined based on the intra-prediction bit cost for the intra-predicted frame and an inter-prediction bit cost determined for the intra-predicted frame based on the one or more other frames.

5. The method of claim 4 , wherein the one or more other frames include multiple inter-predicted frames nearby the inter-predicted frame in a display order of the video stream, and determining the prediction bit cost for the intra-predicted frame comprises:
predicting the inter-prediction bit cost for the intra-predicted frame based on a mean value of inter-prediction bit costs for the multiple inter-predicted frames.

6. The method of claim 1 , wherein the information indicated within the pass log corresponds to prediction residual error data usable to infer the prediction-based bit cost for one or more frames of the multiple frames.

7. The method of claim 1 , wherein the multiple frames are stored in a lookahead buffer prior to the determination of the prediction bit costs for the multiple frames.

8. The method of claim 1 , wherein the multiple frames are identified within a sliding window.

9. The method of claim 1 , wherein the pass log is one of an encoder pass log or a transcoder mezzanine log.

10. A method, comprising:
determining prediction bit costs for frames of a video stream using a pass log for the video stream and information associated with one or more neighboring frames identified for individual ones of the frames based on whether the individual ones of the frames are inter-predicted or intra-predicted;
segmenting the video stream into chunks based on the prediction bit costs and one or more chunk size thresholds; and
transcoding the chunks to produce a transcoded video stream.

11. The method of claim 10 , wherein, for an inter-predicted frame of the frames, the one or more neighboring frames include a nearest intra-predicted frame according to a display order of the video stream, and determining a prediction bit cost for the inter-predicted frame comprises:
determining the prediction bit cost based on an inter-prediction bit cost identified for the inter-predicted frame within the pass log and an intra-prediction bit cost determined based on the nearest intra-predicted frame.

12. The method of claim 10 , wherein, for an intra-predicted frame of the frames, the one or more neighboring frames include multiple inter-predicted frames nearby the intra-predicted frame according to a display order of the video stream, and determining a prediction bit cost for the intra-predicted frame comprises:
determining the prediction bit cost based on an intra-prediction bit cost identified for the intra-predicted frame within the pass log and a mean value of inter-prediction bit costs determined for the multiple inter-predicted frames.

13. The method of claim 10 , wherein the pass log indicates prediction residual error data usable to infer the prediction-based bit costs for the frames.

14. The method of claim 10 , wherein the frames are stored in a lookahead buffer prior to the determination of the prediction bit costs for the frames.

15. A method, comprising:
segmenting a video stream into chunks based on prediction bit costs associated with pairs of frames of the video stream that meet one or more chunk size thresholds, wherein the prediction bit costs associated with a given pair of frames are based on a pass log for the video stream and include a first cost determined based on whether a first frame of the given pair of frames is inter-predicted or intra-predicted and a second cost determined based on whether a second frame of the given pair of frames is inter-predicted or intra-predicted; and
transcoding the chunks to produce a transcoded video stream.

16. The method of claim 15 , wherein the first frame is an inter-predicted frame, the method comprising:
determining the first cost based on an inter-prediction bit cost indicated within the pass log and an intra-prediction bit cost determined for a nearest intra-predicted frame of the first frame within a display order of the video stream.

17. The method of claim 15 , wherein the first frame is an intra-predicted frame, the method comprising:
determining the first cost based on an intra-prediction bit cost indicated within the pass log and an inter-prediction bit cost determined based on a mean value of inter-prediction bit costs for inter-predicted frames nearby the first frame within a display order of the video stream.

18. The method of claim 15 , wherein segmenting the video stream into the chunks comprises:
determining, for each of the pairs of frames, a weighted cost based on an intra-prediction bit cost for a first frame of the pair and a sum of inter-prediction bit costs for all frames between the first frame and a second frame of the pair;
selecting, as chunk boundaries, the pair of the frames corresponding to a lowest one of the weighted costs; and
segmenting the video stream into a chunk according to the chunk boundaries.

19. The method of claim 15 , wherein the frames of each of the pairs of frames are stored in a lookahead buffer.

20. The method of claim 15 , wherein the one or more chunk size thresholds include a minimum chunk size threshold and a maximum chunk size threshold.

Assignees

• Google Llc

Inventors

• Chen Di
• John Sam

Classifications

• H04N19/159

  Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction · CPC title

• H04N19/146

  Data rate or code amount at the encoder output · CPC title

• H04N19/192

  the adaptation method, adaptation tool or adaptation type being iterative or recursive · CPC title

• H04N19/179

  the unit being a scene or a shot · CPC title

• H04N19/119 Primary

  Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks · CPC title