Methods and systems for video processing

Patent metadata

Field	Value
Publication number	US-11527068-B2
Application number	US-202117198177-A
Country	US
Kind code	B2
Filing date	Mar 10, 2021
Priority date	Oct 18, 2016
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

A method for processing an online video stream may include determining a transmission performance of a network for a queue of video frames, wherein each video frame in the queue may be associated with a priority level. The method may also include determining a maximum discarding level based on the transmission performance of the network. The method may further include removing a target video frame of which the associated priority level is lower than or equal to the maximum discarding level from the queue.

First claim

What is claimed is:

1. A video processing system for encoding video streams, comprising:
at least one camera configured to generate a plurality of video frames; and
one or more video processing devices, configured to encode the plurality of video frames,
wherein during operation, the one or more video processing devices:
obtain a candidate video frame the plurality of video frames, wherein the candidate video frame is encoded as an independently decodable video frame;
determine a complexity of the candidate video frame;
obtain a first quantization parameter for performing the encoding; and
determine whether a first encoded size of the candidate video frame exceeds a predetermined threshold based on the complexity of the candidate video frame and the first quantization parameter;
when the first encoded size exceeds the predetermined threshold:
determine a second quantization parameter, so that a second encoded size of the candidate video frame based on the second quantization parameter is lower than or equal to the predetermined threshold; and

encode the candidate video frame based on the second quantization parameter, wherein the one or more video processing devices further:
designate the encoded candidate video frame as a current video frame;
determine a transmission performance of a network for a queue of video frames, each video frame in the queue associated with a priority level;
determine a maximum discarding level based on the transmission performance of the network; and
remove, from the queue, target video frames of which the associated priority level is lower than or equal to the maximum discarding level, wherein the priority level of each video frame in the queue is based at least on an interest level associated with a specified event scene included in that video frame, wherein the interest level represents a degree of attention to the specified event scene.

2. The video processing system of claim 1 , wherein the complexity of the candidate video frame includes a sum of absolute differences (SAD) of the candidate video frame, and the complexity of the candidate video frame is determined based on a SAD of at least one preceding video frames.

3. The video processing system of claim 1 , wherein the one or more video processing devices further:
encode the candidate video frame based on the first quantization parameter when the first encoded size is lower than or equal to the predetermined threshold.

4. The video processing system of claim 1 , wherein the predetermined threshold is based on a network transmission rate and a maximum network transmission wait time.

5. A method configured to encode video streams via at least one video processing device, comprising:
determining, by the at least one video processing device, a complexity of a candidate video frame, wherein the candidate video frame is encoded as an independently decodable video frame;
obtaining, by the at least one video processing device, a first quantization parameter for performing the encoding; and
determining, by the at least one video processing device, whether a first encoded size of the candidate video frame exceeds a predetermined threshold based on the complexity of the candidate video frame and the first quantization parameter;
when the first encoded size exceeds the predetermined threshold:
determining, by the at least one video processing device, a second quantization parameter, so that a second encoded size of the candidate video frame based on the second quantization parameter is lower than or equal to the predetermined threshold; and

encoding, by the at least one video processing device, the candidate video frame based on the second quantization parameter, wherein the method further comprise:
designating the encoded candidate video frame as a current video frame;
determining a transmission performance of a network for a queue of video frames, each video frame in the queue associated with a priority level;
determining a maximum discarding level based on the transmission performance of the network; and
removing, from the queue, target video frames of which the associated priority level is lower than or equal to the maximum discarding level, wherein the priority level of each video frame in the queue is based at least on an interest level associated with a specified event scene included in that video frame, and the interest level represents a degree of attention to the specified event scene.

6. The method of claim 5 , wherein the complexity of the candidate video frame includes a sum of absolute differences (SAD) of the candidate video frame, and the complexity of the candidate video frame is determined based on a SAD of at least one preceding video frames.

7. The method of claim 5 , further comprising:
encoding the candidate video frame based on the first quantization parameter when the first encoded size is lower than or equal to the predetermined threshold.

8. The method of claim 5 , wherein the predetermined threshold is based on a network transmission rate and a maximum network transmission wait time.

9. A non-transitory computer readable medium, storing instructions, the instructions when executed by a processor, causing the processor to execute operations comprising:
determining a complexity of a candidate video frame, wherein the candidate video frame is encoded as an independently decodable video frame;
obtaining a first quantization parameter for performing the encoding;
determining whether a first encoded size of the candidate video frame exceeds a predetermined threshold based on the complexity of the candidate video frame and the first quantization parameter;
when the first encoded size exceeds the predetermined threshold:
determining a second quantization parameter, so that a second encoded size of the candidate video frame based on the second quantization parameter is lower than or equal to the predetermined threshold; and
encoding the candidate video frame based on the second quantization parameter, wherein the operations further comprise:
designating the encoded candidate video frame as a current video frame;
determining a transmission performance of a network for a queue of video frames, each video frame in the queue associated with a priority level;
determining a maximum discarding level based on the transmission performance of the network; and
removing, from the queue, target video frames of which the associated priority level is lower than or equal to the maximum discarding level,

wherein the priority level of each video frame in the queue is based at least on an interest level associated with a specified event scene included in that video frame, and the interest level represents a degree of attention to the specified event scene.

10. The non-transitory computer readable medium of claim 9 , wherein the complexity of the candidate video frame includes a sum of absolute differences (SAD) of the candidate video frame, and the complexity of the candidate video frame is determined based on a SAD of at least one preceding video frames.

11. The non-transitory computer readable medium of claim 9 , further comprising:
encoding the candidate video frame based on the first quantization parameter when the first encoded size is lower than or equal to the predetermined threshold.

12. The non-transitory computer readable medium of claim 9 , wherein the predetermined threshold is based on a network transmission rate and a maximum network transmission wait time.

13. The non-transitory computer readable medium of claim 9 , wherein the candidate video frame is encoded as an independently decodable video frame.

Assignees

• Zhejiang Dahua Technology Co

Inventors

• Cao Xinchao
• Lin Jucai

Classifications

• H04N19/159 Primary

  Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction · CPC title

• H04N21/64769

  for rate control (flow control in packet networks H04L47/10) · CPC title

• G06V20/41 Primary

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• H04N21/234381

  by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping · CPC title

• H04N19/587

  involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence · CPC title