Distortion-based video re-encoding

Patent metadata

Field	Value
Publication number	US-10972788-B1
Application number	US-201816118204-A
Country	US
Kind code	B1
Filing date	Aug 30, 2018
Priority date	Aug 30, 2018
Publication date	Apr 6, 2021
Grant date	Apr 6, 2021

Abstract

An input video stream of video content may be encoded and transmitted from a provider to an intermediary, which decodes and edits the video content, and then re-encodes and transmits the video content to end viewers via an output video stream. When re-encoding the video content, the intermediary may determine to selectively re-use and/or not re-use input motion vectors from the input video stream, for example based on an amount of distortion associated with editing of the video content. In some examples, input motion vectors may be re-used for re-encoding of certain portions (e.g., frames, parts of frames, etc.) of the output video stream and not re-used for re-encoding of other portions of the output video stream.

First claim

What is claimed is:

1. A computing system for distortion-based video processing in live video streams comprising:
one or more processors; and
one or more memories having stored therein instructions that, upon execution by the one or more processors, cause the computing system to perform operations comprising:
decoding a first portion of input video content included in an input live video stream, wherein one or more edits are applied to the first portion of input video content, wherein a first portion of output video content includes the first portion of input video content with the one or more edits applied thereto;
determining a first amount of distortion associated with the one or more edits to the first portion of input video content, wherein the first amount of distortion is determined based at least in part on one or more differences between the first portion of input video content and the first portion of output video content;
comparing the first amount of distortion to a threshold amount of distortion;
based at least in part on the comparing, determining whether or not to use one or more first motion vectors from the input live video stream to encode the first portion of output video content in an output live video stream; and
encoding the first portion of output video content in the output live video stream.

2. The computing system of claim 1 , wherein the one or more first motion vectors from the input live video stream are used to encode the first portion of output video content in the output live video stream, and wherein no motion vectors from the input live video stream are used to encode one or more other portions of output video content in the output live video stream.

3. The computing system of claim 1 , wherein the one or more first motion vectors from the input live video stream are not used to encode the first portion of output video content in the output live video stream, and wherein one or more other motion vectors from the input live video stream are used to encode one or more other portions of output video content in the output live video stream.

4. The computing system of claim 1 , wherein the distortion of the first portion of input video content comprises an indication of a result of a facial recognition process.

5. A computer-implemented method for distortion-based video processing comprising:
decoding a first portion of input video content included in an input video stream, wherein one or more edits are applied to the first portion of input video content, wherein a first portion of output video content includes the first portion of input video content with the one or more edits applied thereto;
determining a first amount of distortion associated with the one or more edits to the first portion of input video content, wherein the first amount of distortion is determined based at least in part on one or more differences between the first portion of input video content and the first portion of output video content;
comparing the first amount of distortion to a threshold amount of distortion;
based at least in part on the comparing, determining whether or not to use one or more first motion vectors from the input video stream to encode the first portion of output video content in an output video stream; and
encoding the first portion of output video content in the output video stream.

6. The computer-implemented method of claim 5 , wherein the first amount of distortion is determined further based in part on one or more differences between a reference portion of output video content and the first portion of output video content.

7. The computer-implemented method of claim 5 , wherein the threshold amount of distortion is a threshold percentage of changed color pixel values.

8. The computer-implemented method of claim 5 , wherein the threshold amount of distortion is a threshold peak signal-to-noise ratio (PSNR) amount.

9. The computer-implemented method of claim 5 , wherein the first portion of input video content comprises a whole video frame.

10. The computer-implemented method of claim 5 , further comprising determining, based at least in part on the comparing, whether or not to use additional information associated with the input video stream to encode the first portion of output video content in the output video stream.

11. The computer-implemented method of claim 10 , wherein the additional information comprises at least one of an inter-frame encoding mode, an intra-frame encoding mode, a macroblock size, or a skip macroblock determination.

12. The computer-implemented method of claim 5 , wherein the one or more first motion vectors from the input video stream are used to encode the first portion of output video content in the output video stream, and wherein no motion vectors from the input video stream are used to encode one or more other portions of output video content in the output video stream.

13. The computer-implemented method of claim 5 , wherein the one or more first motion vectors from the input video stream are not used to encode the first portion of output video content in the output video stream, and wherein one or more other motion vectors from the input video stream are used to encode one or more other portions of output video content in the output video stream.

14. The computer-implemented method of claim 5 , wherein, when a determination is made to use the one or more first motion vectors, the encoding of the first portion of output video content is performed with an identical motion vector that was used in the input video stream.

15. The computer-implemented method of claim 5 , wherein, when a determination is made to use the one or more first motion vectors, a motion vector that was used in the input video stream is modified, and the encoding of the first portion of output video content is performed with a modified version of the motion vector that was used in the input video stream.

16. The computer-implemented method of claim 5 , wherein the one or more first motion vectors are applied as a reference to search for a different motion vector for use in the encoding.

17. One or more non-transitory computer-readable storage media having stored thereon instructions that, upon execution by a computing device, cause the computing device to perform operations comprising:
decoding a first portion of input video content included in an input video stream, wherein one or more edits are applied to the first portion of input video content, wherein a first portion of output video content includes the first portion of input video content with the one or more edits applied thereto;
determining a first amount of distortion associated with the one or more edits to the first portion of input video content, wherein the first amount of distortion is determined based at least in part on one or more differences between the first portion of input video content and the first portion of output video content;
comparing the first amount of distortion to a threshold amount of distortion;
based at least in part on the comparing, determining whether or not to use one or more first motion vectors from the input video stream to encode the first portion of output video content in an output video stream; and
encoding the first portion of output video content in the output video stream.

18. The one or more non-transitory computer-readable storage media of claim 17 , wherein the one or more first motion vectors from the input video stream are used to encode the first portion of output video content in the output video stream, and wher

Assignees

• Amazon Tech Inc

Inventors

• Wang Qia

Classifications

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• G06V40/16

  Human faces, e.g. facial parts, sketches or expressions · CPC title

• H04N19/40

  using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream · CPC title

• H04N21/64784

  Data processing by the network (data processing in packet switching systems H04L12/56; flow control in packet networks H04L47/10; intermediate storage or scheduling H04L49/90; provisioning of proxy services in data packet switching networks H04L67/56) · CPC title

• H04N21/440254 Primary

  by altering signal-to-noise parameters, e.g. requantization · CPC title