Systems and methods for hybrid video encoding

Patent metadata

Field	Value
Publication number	US-10187649-B2
Application number	US-201615064744-A
Country	US
Kind code	B2
Filing date	Mar 9, 2016
Priority date	Mar 10, 2015
Publication date	Jan 22, 2019
Grant date	Jan 22, 2019

Abstract

Systems and methods are provided for hybrid video encoding. An example method includes: acquiring image information; extracting a background image based at least in part on the image information; detecting whether the background image is stable; and performing encoding switching between a background-based encoding method and a non-background encoding method based at least in part on the detection.

First claim

What is claimed is:

1. A processor-implemented method for hybrid video encoding, the method comprising:
acquiring, using one or more processors, frame information;
extracting, using the one or more processors, a background image based at least in part on the frame information;
calculating, using the one or more processors, a confidence degree of the background image;
determining, using the one or more processors, whether the confidence degree is smaller than a predetermined threshold value;
in response to the confidence degree being smaller than the predetermined threshold value, the one or more processors switching to a non-background encoding method; and
in response to the confidence degree being not smaller than the predetermined threshold value, the one or more processors
acquiring foreground points for one or more foreground images,
acquiring frame difference points for one or more frames,
detecting, within a predetermined time period, a foreground image with foreground points smaller than a first threshold,
detecting, within the predetermined time period, a frame difference image with frame difference points smaller than a second threshold,
in response to a foreground image with foreground points smaller than the first threshold and a frame difference image with frame difference points smaller than the second threshold being detected within the predetermined time period, switching to the background-based encoding method, and
in response to a foreground image with foreground points smaller than the first threshold and a frame difference image with frame difference points smaller than the second threshold not being detected within the predetermined time period, switching to the non-background encoding method.

2. The method of claim 1 , wherein the background-based encoding method includes:
acquiring an original frame at a current moment;
acquiring a foreground image based at least in part on the original frame;
acquiring a background image based at least in part on the original frame; and
acquiring a region of variation based at least in part on the background image.

3. The method of claim 2 , wherein acquiring a background image based at least in part on the original frame includes:
comparing frame difference images of a previous frame and a current frame;
determining whether an image of the current frame is a stable image that does not contain foreground information;
in response to the image of the current frame being a stable image that does not contain foreground information, taking the current frame as the background frame; and
in response to the image of the current frame being not a stable image that does not contain foreground information, acquiring the original frame at the current moment.

4. The method of claim 1 , wherein:
the confidence degree of the background image corresponds to a whole-image confidence degree;
the whole-image confidence degree represents a confidence degree of the background image;
the whole-image confidence degree is obtained through calculating, in a background-confidence-degree image, a proportion of macro blocks of which confidence degrees are greater than a threshold value.

5. The method of claim 1 , wherein switching between the background-based encoding method and the non-background encoding method is further based at least in part on PTZ coordinates, including:
detecting PTZ coordinates in the image information;
in response to a change of the PTZ coordinates not being detected, switching to the background-based encoding method; and
in response to a change of the PTZ coordinates being detected, switching to the non-background encoding method.

6. The method of claim 5 , wherein in response to a change of the PTZ coordinates not being detected, switching to the background-based encoding method includes:
performing encoding switching based at least in part on the background image.

7. The method of claim 1 , wherein switching between the background-based encoding method and the non-background encoding method is further based at least in part on bit-rate changes, including:
recording one or more bit rates within a first time period;
detecting whether a change of the bit rates between consecutive times exceeds a third threshold value;
in response to the change of the bit rates between consecutive times not exceeding the third threshold value, continuing recording the bit rates within the first time period;
in response to the change of the bit rates between consecutive times exceeding the third threshold value, detecting whether a duration time during which the change of the bit rates exceeds the third threshold value is longer than a fourth threshold value;
in response to the duration time during which the change of the bit rates exceeds the third threshold value being longer than the fourth threshold value, performing encoding switching based at least in part on a background model; and
in response to the second time period during which the change of the bit rates exceeds the third threshold value being not longer than the fourth threshold value, continuing recording the bit rates within the first time period.

8. A system for hybrid video encoding, comprising:
one or more processors embedded in a digital camera, the one or more processors including:
a background modeling processor configured to extract a background image based at least in part on frame information;
a background detection processor configured to detect whether the background image is stable;
an encoding switching processor configured to perform encoding switching between a background-based encoding method and a non-background encoding method based at least in part on the detection; and
an encoder configured to perform encoding operations based at least in part on an encoding-switching command from the encoding switching processor;
the background detection processor being configured to,
calculate a confidence degree of the background image,
determine whether the confidence degree is smaller than a predetermined threshold value,
in response to the confidence degree being smaller than the predetermined threshold value, cause the encoding switching processor to switch to a non-background encoding method,
in response to the confidence degree being not smaller than the predetermined threshold value,
acquire foreground points for one or more foreground images,
acquire frame difference points for one or more frames,
detect, within a predetermined time period, a foreground image with foreground points smaller than a first threshold,
detect, within the predetermined time period, a frame difference image with frame difference points smaller than a second threshold,
in response to a foreground image with foreground points smaller than the first threshold and a frame difference image with frame difference points smaller than the second threshold being detected within the predetermined time period, cause the encoding switching processor to switch to the background-based encoding method, and
in response to a foreground image with foreground points smaller than the first threshold and a frame difference image with frame difference points smaller than the second threshold not being detected within the predetermined time period, cause the encoding switching processor to switch to the non-background encoding method.

9. The system of claim 8 , wherein the background detection processor includes:
an image preprocessing processor configured to preprocess an original frame to obtain one or more preset image formats;
a foreground image extraction processor configured to perform background modeling to extract a foreground image that includes one or more moving objects from the fram

Assignees

• Hangzhou Hikvision Digital Tec
• Hangzhou Hiksvision Digital Tech Co Ltd

Inventors

• Tong Junyan
• Pu Shiliang
• Shen Linjie
• Ma Cheng
• Hu Zhigang
• Ren Ye
• Yu Hai

Classifications

• H04N19/103

  Selection of coding mode or of prediction mode · CPC title

• H04N19/23 Primary

  with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic · CPC title

• H04N19/146

  Data rate or code amount at the encoder output · CPC title

• H04N19/167

  Position within a video image, e.g. region of interest [ROI] · CPC title

• H04N19/142

  Detection of scene cut or scene change · CPC title