Software hardware hybrid video encoder

Patent metadata

Field	Value
Publication number	US-10349069-B2
Application number	US-201213710794-A
Country	US
Kind code	B2
Filing date	Dec 11, 2012
Priority date	Dec 11, 2012
Publication date	Jul 9, 2019
Grant date	Jul 9, 2019

Abstract

In aspects of the present disclosure, a software encoder augments a hardware encoder by implementing portions of a video encoding task that are not supported by the hardware encoder while the hardware encoder implements other portions of the encoding task. The use of a software encoder to augment a hardware encoder in this manner can extend the useful life of a hardware encoder, allow the system to adapt to changes in video coding standards, and in some cases improve performance of encoding implemented by hardware or software alone. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

First claim

What is claimed is:

1. A video encoding method for encoding at least two video input streams, comprising:
downscaling an input high resolution video stream to generate a first video input stream of the at least two video input streams at a preprocessing module, wherein the first video input stream is compatible with a hardware encoder and is characterized by a lower resolution than the input high resolution video stream, wherein a second video input stream of the at least two video input streams is the input high resolution video stream;
performing a first portion of a video encoding task on one or more pictures of the first video input stream with the hardware encoder, wherein the hardware encoder encodes the first input video stream to generate a lower resolution encoded video stream;
delivering compatible motion information of the lower resolution encoded video stream from the hardware encoder to a buffer for use by the software encoder; and
performing a second portion of the video encoding task for one or more pictures in the second video input stream by executing a software encoder on a programmable processor, wherein the software encoder implements higher resolution video encoding utilizing the lower resolution encoded video stream as a prediction reference, wherein the software encoder obtains motion information from the lower resolution encoded video stream, wherein obtaining the motion information shortens a motion search performed by the software encoder during the higher resolution video encoding.

2. A video encoding method, comprising:
splitting a video input stream into a first video input stream and a second video input stream;
downscaling an input high resolution video stream to generate a first video input stream of the at least two video input streams at a preprocessing module, wherein the first video input stream is compatible with a hardware encoder and is characterized by a lower resolution than the input high resolution video stream, wherein a second video input stream of the at least two video input streams is the input high resolution video stream;
performing a first portion of a video encoding task on one or more pictures of the first video input stream with the hardware encoder, wherein the hardware encoder encodes the first input video stream to generate a lower resolution encoded video stream;
delivering compatible motion information of the lower resolution encoded video stream from the hardware encoder for use by the software encoder to a buffer; and
performing a second portion of the video encoding task for one or more pictures of the second video input stream by executing a software encoder on a programmable processor, wherein the software encoder implements higher resolution video encoding utilizing the lower resolution encoded video stream as a prediction reference, wherein the software encoder obtains motion information from the lower resolution encoded video stream, wherein obtaining the motion information shortens a motion search performed by the software encoder during the higher resolution video encoding.

3. The method of claim 2 , wherein the software encoder utilizes the encoded I and/or P pictures as references for the B picture encoding.

4. The method of claim 1 , wherein the one or more encoding features of the second portion of the video encoding task include utilization of two or more reference frames for the encoding of a frame in a video input stream.

5. The method of claim 1 , wherein the software encoder utilizes two or more encoded frames from the first video input stream as a reference frame for encoding each frame in the second video input stream.

6. The method of claim 1 , wherein the one or more features of the second portion of the video encoding task include encoding a multi-view profile, wherein the multiview profile is comprised of two or more video input streams.

7. The method of claim 6 , wherein the software encoder utilizes the encoded first video input stream as a reference for encoding the second video input stream.

8. The method of claim 6 , wherein the first video input stream and the second video input stream produce a three-dimensional image when combined.

9. The method of claim 6 , wherein the first video input stream is obtained from a first video camera and the second video input stream is obtained from a second video camera.

10. The method of claim 1 , wherein the one or more encoding features of the second portion of the video encoding task include multi-pass encoding.

11. The method of claim 10 , wherein an output from the first portion of the video encoding task is used by the software encoder as a prediction reference.

12. The method of claim 11 , wherein the second potion of the video encoding task further comprises alpha blending the second video input stream with the first video input stream.

13. The method of claim 12 , wherein the second video input stream is a computer generated texture.

14. The method of claim 1 , wherein the second portion of the video encoding task includes encoding the high-resolution video input stream by utilizing the information in the encoded lower-resolution video input stream for rate control.

15. The method of claim 1 , wherein the high-resolution video input stream is comprised of two or more video input streams.

16. The method of claim 15 , wherein each of the two or more video input streams are a view of a separate gamer in a multi-player video game.

17. The method of claim 1 , wherein the one or more features of the second portion of the video encoding task include encoding a high frame rate video input stream.

18. The method of claim 17 , wherein the first portion of the video encoding task comprises scaling down the high frame rate video input stream into a low frame rate video input stream, and wherein the second portion of a video encoding task comprises encoding the high frame rate video input stream by utilizing the encoded low frame rate video input stream as a prediction reference.

19. The method of claim 18 , wherein the low frame rate video input utilizes one frame for every four frames in the high frame rate video input stream.

20. The method of claim 17 , wherein the first portion of the video encoding task comprises scaling down the high frame rate video input stream into a low frame rate video input stream, and wherein the second portion of a video encoding task comprises encoding the high frame rate video input stream by utilizing the information in the encoded low frame rate video input stream for rate control.

21. The method of claim 18 , wherein the high frame rate video input stream is 120 frames per second.

22. The method of claim 1 , wherein the second portion of the video encoding task includes encoding a video input stream that is in a tree structure.

23. The method of claim 22 , wherein the first portion of the video encoding task comprises encoding one or more video frames that form a trunk of the tree structure, and wherein the second portion of the video encoding task comprises encoding one or more video frames that form one or more branches of the tree structure.

24. The method of claim 23 , wherein the software encoder utilizes the one or more encoded video frames of the trunk of the tree structure as a reference frame for one or more of the video frames of the one or more branches of the tree structure.

25. An video encoding apparatus for encoding at least two video input streams, c

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Wang Jason N
• Lee Hung-Ju

Classifications

• H04N19/42 Primary

  characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) · CPC title

• H04N19/156

  Availability of hardware or computational resources, e.g. encoding based on power-saving criteria · CPC title