Adaptive perceptual quality based camera tuning using reinforcement learning

Patent metadata

Field	Value
Publication number	US-2024089592-A1
Application number	US-202318466296-A
Country	US
Kind code	A1
Filing date	Sep 13, 2023
Priority date	Sep 14, 2022
Publication date	Mar 14, 2024
Grant date	—

Abstract

Systems and methods are provided for dynamically tuning camera parameters in a video analytics system to optimize analytics accuracy. A camera captures a current scene, and optimal camera parameter settings are learned and identified for the current scene using a Reinforcement Learning (RL) engine. The learning includes defining a state within the RL engine as a tuple of two vectors: a first representing current camera parameter values and a second representing measured values of frames of the current scene. Quality of frames is estimated using a quality estimator, and camera parameters are adjusted based on the quality estimator and the RL engine for optimization. Effectiveness of tuning is determined using perceptual Image Quality Assessment (IQA) to quantify a quality measure. Camera parameters are adaptively tuned in real-time based on learned optimal camera parameter settings, state, quality measure, and set of actions, to optimize the analytics accuracy for video analytics tasks.

First claim

What is claimed is:

1 . A method for dynamically tuning camera parameters in a video analytics system (VAS) to optimize analytics accuracy, comprising:
capturing a current scene using a video-capturing camera; learning optimal camera parameter settings for the current scene using a Reinforcement Learning (RL) engine by defining a state within the RL engine as a tuple of a first vector representing current camera parameter values and a second vector representing measured values of captured frames of the current scene, and defining sets of actions for modifying parameter values and maintaining the current parameter values; estimating a quality of the captured frames using a perceptual no-reference quality estimator, and tuning the camera parameter settings based on the quality estimator and the RL engine to optimize analytics accuracy of the VAS; evaluating an effectiveness of the tuning by perceptual Image Quality Assessment (IQA) to quantify a quality measure; iteratively adaptively tuning the camera parameter settings in real-time using the RL engine, responsive to changes in the scene, based on the learned optimal camera parameter settings, the state, the quality measure, and the set of actions, to further optimize the analytics accuracy until a threshold condition is reached.

2 . The method of claim 1 , wherein the current scene captured by the video-capturing camera is identified using a scene classification algorithm configured to classify the scene into one of a plurality of predefined categories.

3 . The method of claim 1 , wherein the Reinforcement Learning (RL) engine utilizes a State-action-reward-state-action (SARSA) algorithm configured to enhance the learning and improve speed and accuracy of identification of the optimal camera parameter settings.

4 . The method of claim 1 , wherein the camera parameter settings are tuned responsive to a perceptual no-reference quality determination indicating a quality of the captured frames being below a quality threshold level.

5 . The method of claim 1 , wherein a dynamic adjustment of a constant E is integrated into the RL engine to balance between exploration and exploitation, the dynamic adjustment being based on progress of the learning by the RL engine.

6 . The method of claim 1 , wherein the video analytics system (VAS) is implemented in a security surveillance system configured for object detection and recognition, and the tuning of the camera parameter settings is iteratively executed at a predetermined interval to optimize the object detection and recognition.

7 . The method of claim 1 , wherein the tuning the camera parameter settings includes real-time adaptation to changes in lighting conditions, weather, or other environmental factors affecting the captured scene, the camera parameter settings including one or more of brightness, contrast, sharpness, and color-saturation.

8 . The method of claim 1 , wherein rewards or penalties received by the RL engine are based on predefined criteria, including maximizing image clarity and minimizing noise.

9 . A system for optimizing analytics accuracy in a Video Analytics System (VAS) by dynamically tuning camera parameters, comprising:
a video-capturing camera configured to capture a current scene; a processor operatively coupled to a computer-readable storage medium, the processor being configured for: learning optimal camera parameter settings for the current scene using a Reinforcement Learning (RL) engine by defining a state within the RL engine as a tuple of a first vector representing current camera parameter values and a second vector representing measured values of captured frames of the current scene, and defining sets of actions for modifying parameter values and maintaining the current parameter values; estimating a quality of the captured frames using a perceptual no-reference quality estimator, and tuning the camera parameter settings based on the quality estimator and the RL engine to optimize analytics accuracy of the VAS; evaluating an effectiveness of the tuning by perceptual Image Quality Assessment (IQA) to quantify a quality measure; iteratively adaptively tuning the camera parameter settings in real-time using the RL engine, responsive to changes in the scene, based on the learned optimal camera parameter settings, the state, the quality measure, and the set of actions, to further optimize the analytics accuracy until a threshold condition is reached.

10 . The system of claim 9 , wherein the current scene captured by the video-capturing camera is identified using a scene classification algorithm configured to classify the scene into one of a plurality of predefined categories.

11 . The system of claim 9 , wherein the camera parameter settings are tuned responsive to a perceptual no-reference quality determination indicating a quality of the captured frames being below a quality threshold level.

12 . The system of claim 9 , wherein a dynamic adjustment of a constant E is integrated into the RL engine to balance between exploration and exploitation, the dynamic adjustment being based on progress of the learning by the RL engine.

13 . The system of claim 9 , wherein the video analytics system (VAS) is implemented in a security surveillance system configured for object detection and recognition, and the tuning of the camera parameter settings is iteratively executed at a predetermined interval to optimize the object detection and recognition.

14 . The system of claim 9 , wherein the tuning the camera parameter settings includes real-time adaptation to changes in lighting conditions, weather, or other environmental factors affecting the captured scene, the camera parameter settings including one or more of brightness, contrast, sharpness, and color-saturation.

15 . A computer program product for optimizing analytics accuracy in a Video Analytics System (VAS) by dynamically tuning camera parameters in real-time, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
capturing a current scene using a video-capturing camera; learning optimal camera parameter settings for the current scene using a Reinforcement Learning (RL) engine by defining a state within the RL engine as a tuple of a first vector representing current camera parameter values and a second vector representing measured values of captured frames of the current scene, and defining sets of actions for modifying parameter values and maintaining the current parameter values; estimating a quality of the captured frames using a perceptual no-reference quality estimator, and tuning the camera parameter settings based on the quality estimator and the RL engine to optimize analytics accuracy of the VAS; evaluating an effectiveness of the tuning by perceptual Image Quality Assessment (IQA) to quantify a quality measure; iteratively adaptively tuning the camera parameter settings in real-time using the RL engine, responsive to changes in the scene, based on the learned optimal camera parameter settings, the state, the quality measure, and the set of actions, to further optimize the analytics accuracy until a threshold condition is reached.

16 . The computer program product of claim 15 , wherein the Reinforcement Learning (RL) engine utilizes a State-action-reward-state-action (SARSA) algorithm configured to enhance the learning and improve speed and accuracy of identification of the optimal camera parameter se

Assignees

• Nec Lab America Inc

Inventors

• Rao Kunal
• Paul Sibendu
• Coviello Giuseppe
• Sankaradas Murugan
• Po Oliver
• Chakradhar Srimat

Classifications

• H04N23/64 Primary

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/61

  based on recognised objects · CPC title

• H04N23/60 Primary

  Control of cameras or camera modules · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title