Offloading video coding processes to hardware for better density-quality tradeoffs

Patent metadata

Field	Value
Publication number	US-2021105466-A1
Application number	US-202017127544-A
Country	US
Kind code	A1
Filing date	Dec 18, 2020
Priority date	Dec 18, 2020
Publication date	Apr 8, 2021
Grant date	—

Abstract

Techniques related to distributing the video encoding processing of an input video across hardware and software systems. Such techniques include evaluating the content of the video and determine whether or the encoding operation is best to be done on the hardware system only, software system only or a hybrid hardware and software system.

First claim

What is claimed is:

1 . A system for video coding comprising:
a memory to store at least a portion of input video for coding; and one or more processors coupled to the memory, the one or more processors to:
adaptively determine a hardware operation is to be applied for at least one of an encode of a group of video pictures of the input video, an encode of a particular picture of the input video, or an encode operation for at least a tile of the particular picture of the input video; and
perform the encode of the group of video pictures, the encode of the particular picture, or the encode operation for the tile of the particular picture via a function call to hardware and at least a second encode of a portion of a remainder of the input video via software to generate an output bitstream.

2 . The system of claim 1 , the one or more processors to:
access a target video quality level, a target operation latency, and functional capabilities of the hardware and the software; and select the hardware operation based on access to a table using the target quality level, the target latency, and the functional capabilities.

3 . The system of claim 1 , wherein the particular picture is in a group of pictures of the input video, and wherein the hardware operation is determined for the encode of the particular picture in response to one of the particular picture being in a higher temporal layer in the group of pictures or a first quality target for the particular picture being met by a hardware only picture encode quality metric corresponding to the hardware operation, wherein a second picture in the group of pictures is coded at least partially in software in response to the second picture being in a lower temporal layer in the group of pictures or a second quality target for the second picture not being met by the hardware only picture encode quality metric.

4 . The system of claim 3 , the one or more processors to:
encode the second picture using a multi-pass encode in response to multi-pass encode being required to meet the second quality target, wherein the multi-pass encode comprises a first pass using the hardware or second hardware and a subsequent pass at least partially performed in software.

5 . The system of claim 3 , the one or more processors to:
determine multi-pass encoding is not needed to meet the second quality target; and perform single pass encode of the second frame comprising software encode up to a particular encode stage and hardware encode for one or more subsequent stages to the particular encode stage.

6 . The system of claim 5 , wherein the particular encode stage comprises a mode decision stage and the one or more subsequent stages comprise a reconstruction stage, a filtering stage, and an entropy coding stage.

7 . The system of claim 1 , wherein the encode operation for the tile of the particular picture comprises one of a plurality of first encode operations for encode of the tile of the particular picture, the plurality of first encode operations are performed via hardware, the encode of the tile of the particular picture comprising one or more second encode operations performed via software.

8 . The system of claim 7 , wherein the first encode operations comprise one or more of motion compensated temporal filtering, global motion estimation, flat picture analysis, spatial analysis, open loop intra search, film grain synthesis, picture based interpolation, picture reconstruction, loop filtering, or entropy coding and the one or more second encode operations comprise mode decision.

9 . The system of claim 1 , wherein the hardware comprises one of fixed function circuitry or an application-specific integrated circuit and the second encode via software comprises running a plurality of lines of instruction code via a general purpose device.

10 . The system of claim 1 , the one or more processors to:
decode a received bitstream via a second function call to second hardware to generate the input video.

11 . The system of claim 10 , wherein the second encode via software is based at least in part on metadata generated by the decoding of the received bitstream.

12 . A method for video coding comprising:
receiving input video for coding; adaptively determining a hardware operation is to be applied for at least one of an encode of a group of video pictures of the input video, an encode of a particular picture of the input video, or an encode operation for at least a tile of the particular picture of the input video; and performing the encode of the group of video pictures, the encode of the particular picture, or the encode operation for the tile of the particular picture via hardware and at least a second encode of a portion of a remainder of the input video via software to generate an output bitstream.

13 . The method of claim 12 , further comprising:
accessing a target video quality level, a target operation latency, and functional capabilities of the hardware and the software; and selecting the hardware operation based on accessing a table using the target quality level, the target latency, and the functional capabilities.

14 . The method of claim 12 , wherein the particular picture is in a group of pictures of the input video, and wherein the hardware operation is determined for the encode of the particular picture in response to one of the particular picture being in a higher temporal layer in the group of pictures or a first quality target for the particular picture being met by a hardware only picture encode quality metric corresponding to the hardware operation, wherein a second picture in the group of pictures is coded at least partially in software in response to the second picture being in a lower temporal layer in the group of pictures or a second quality target for the second picture not being met by the hardware only picture encode quality metric.

15 . The method of claim 14 , further comprising:
encoding the second picture using multi-pass encoding in response to multi-pass encoding being required to meet the second quality target, wherein the multi-pass encoding comprises performing first pass using the hardware or second hardware and performing a subsequent pass at least partially in software.

16 . The method of claim 14 , further comprising:
determining multi-pass encoding is not needed to meet the second quality target; and performing single pass encode of the second frame comprising software encode up to a particular encode stage and hardware encode for one or more subsequent stages to the particular encode stage.

17 . At least one machine readable medium comprising a plurality of instructions that, in response to being executed on a device, cause the computing device to perform video coding by:
receiving input video for coding; adaptively determining a hardware operation is to be applied for at least one of an encode of a group of video pictures of the input video, an encode of a particular picture of the input video, or an encode operation for at least a tile of the particular picture of the input video; and performing the encode of the group of video pictures, the encode of the particular picture, or the encode operation for the tile of the particular picture via hardware and at least a second encode of a portion of a remainder of the input video via software to generate an output bitstream.

18 . The machine readable medium of claim 17 , further comprising instructions that, in response to being executed on the device, cause the device to perform video coding by:
acc

Assignees

• Intel Corp

Inventors

• Ganesh Brinda
• Jain Nilesh
• Mohan Sumit
• Kossentini Faouzi
• Boyce Jill
• Holland James
• Lei Zhijun
• Nouira Chekib
• Ben Amara Foued
• Tmar Hassene
• Possos Sebastian
• Hurst Craig

Classifications

• H04N19/91

  Entropy coding, e.g. variable length coding [VLC] or arithmetic coding · CPC title

• H04N19/42 Primary

  characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) · CPC title

• H04N19/184

  the unit being bits, e.g. of the compressed video stream · CPC title

• H04N19/174

  the region being a slice, e.g. a line of blocks or a group of blocks · CPC title

• H04N19/186

  the unit being a colour or a chrominance component · CPC title