Video analytics accuracy using transfer learning

Patent metadata

Field	Value
Publication number	US-2024037778-A1
Application number	US-202318361340-A
Country	US
Kind code	A1
Filing date	Jul 28, 2023
Priority date	Jul 30, 2022
Publication date	Feb 1, 2024
Grant date	—

Abstract

Systems and methods are provided for increasing accuracy of video analytics tasks in real-time by acquiring a video using video cameras, and identifying fluctuations in the accuracy of video analytics applications across consecutive frames of the video. The identified fluctuations are quantified based on an average relative difference of true-positive detection counts across consecutive frames. Fluctuations in accuracy are reduced by applying transfer learning to a deep learning model initially trained using images, and retraining the deep learning model using video frames. A quality of object detections is determined based on an amount of track-ids assigned by a tracker across different video frames. Optimization of the reduction of fluctuations includes iteratively repeating the identifying, the quantifying, the reducing, and the determining the quality of object detections until a threshold is reached. Model predictions for each frame in the video are generated using the retrained deep learning model.

First claim

What is claimed is:

1 . A method for increasing accuracy of video analytics tasks in real-time, comprising:
acquiring a video using one or more video cameras, and identifying fluctuations in the accuracy of video analytics applications across consecutive frames of the video; quantifying the identified fluctuations by determining an average relative difference of true-positive detection counts across the consecutive frames; reducing the fluctuations in accuracy by applying transfer learning to a deep learning model initially trained using images, and retraining the deep learning model using video frames captured for a plurality of different scenarios; determining a quality of object detections based on an amount of track-ids assigned by a tracker across different video frames; optimizing the reducing the fluctuations by iteratively repeating the identifying fluctuations, the quantifying the identified fluctuations, the reducing the fluctuations in accuracy, and the determining a quality of object detections until a threshold is reached; and generating model predictions for each frame in the video using the retrained deep learning model for the video analytics tasks.

2 . The method as recited in claim 1 , further comprising performing object and person detection across the different video frames of the video in real-time with increased detection speed and accuracy by using the deep learning model retrained using the video frames.

3 . The method as recited in claim 1 , further comprising performing object and person counts for a particular area of interest based on the captured video using the retrained deep learning model, and generating a list of the predicted object and person counts in the area of interest.

4 . The method as recited in claim 1 , wherein the retraining of the deep learning model using transfer learning includes extracting a plurality of video frames from the video and pre-processing the extracted frames to match input requirements of the deep learning model.

5 . The method as recited in claim 1 , wherein the fluctuations in accuracy are reduced by adjusting a confidence threshold in the deep learning model based on a difficulty level associated with detection in particular frames of the video.

6 . The method as recited in claim 1 , wherein the fluctuations in accuracy result from an adversarial effect caused by automatic, dynamic camera parameter changes in a video camera.

7 . The method as recited in claim 1 , wherein the fluctuations in accuracy are quantified by determining the average relative difference of the true-positive object detection counts across the consecutive frames by:
∥ tp ( i )− tp ( i+ 1)∥/mean( gt ( i , gt ( + 1),
where i represents a video frame, tp(i) represents a true positive object detection count on frame i, and gt(i) represents a ground-truth object count on frame i, on a moving window of 2 frames.

8 . The method as recited in claim 1 , wherein the fluctuations in accuracy are quantified by determining the average relative difference of the true-positive object detection counts across the consecutive frames by:



max
⁢

(

tp
⁡
(
i
)

,
…

,

tp
⁡
(

i
+
9

)

)

-

min
⁡
(

tp
⁡
(
i
)

,
…

,

tp
⁡
(

i
+
9

)

)



mean
(

gt
⁡
(
i
)

,
…

,

gt
⁡
(

i
+
9

)

)

,

where i represents a video frame, tp(i) represents a true positive object detection count on frame i, and gt(i) represents a ground-truth object count on frame i, on a moving window of 10 frames.

9 . A system for increasing accuracy of video analytics tasks in real-time, comprising:
a processor operatively coupled to a non-transitory computer-readable storage medium, the processor being configured for:
acquiring a video using one or more video cameras, and identifying fluctuations in the accuracy of video analytics applications across consecutive frames of the video;
quantifying the identified fluctuations by determining an average relative difference of true-positive detection counts across the consecutive frames;
reducing the fluctuations in accuracy by applying transfer learning to a deep learning model initially trained using images, and retraining the deep learning model using video frames captured for a plurality of different scenarios;
determining a quality of object detections based on an amount of track-ids assigned by a tracker across different video frames;
optimizing the reducing the fluctuations by iteratively repeating the identifying fluctuations, the quantifying the identified fluctuations, the reducing the fluctuations in accuracy, and the determining a quality of object detections until a threshold is reached; and
generating model predictions for each frame in the video using the retrained deep learning model for the video analytics tasks.

10 . The system as recited in claim 9 , wherein the processor is further configured for performing object and person detection across the different video frames of the video in real-time with increased detection speed and accuracy by using the deep learning model retrained using the video frames.

11 . The system as recited in claim 9 , wherein the proces

Assignees

• Nec Lab America Inc

Inventors

• Rao Kunal
• Coviello Giuseppe
• Sankaradas Murugan
• Po Oliver
• Chakradhar Srimat
• Paul Sibendu

Classifications

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06V10/25 Primary

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V2201/07

  Target detection · CPC title