Video monitoring apparatus, method of controlling the same, computer-readable storage medium, and video monitoring system

Patent metadata

Field	Value
Publication number	US-11048947-B2
Application number	US-201715856577-A
Country	US
Kind code	B2
Filing date	Dec 28, 2017
Priority date	Jan 13, 2017
Publication date	Jun 29, 2021
Grant date	Jun 29, 2021

Abstract

According to the present invention, switching of the monitoring images matching the intention of the observer can be automatically performed for images from a plurality of image capturing apparatus, and the load about the job of the observer can be reduced. The image monitoring apparatus includes an estimating unit configured to estimate attention degrees of a user for a plurality of images acquired from the plurality of image capturing apparatuses, a designating unit configured to designate one of the acquired images as an image to be displayed in accordance with an instruction from the user, a learning unit configured to cause the estimating unit to learn so as to increase an attention degree of the designated image, and a selecting unit configured to select one of the plurality of images based on an attention degree of each estimated image.

First claim

What is claimed is:

1. A video monitoring apparatus comprising:
one or more processors; and
one or more memories coupled to the one or more processors, the one or more memories having stored thereon instructions which, when executed by the one or more processors, cause the apparatus to:
acquire, from a video monitoring camera, a video containing a plurality of images;
estimate, by a neural network, respective degrees of attention by a user for the plurality of images based on their image features;
display, on a video monitoring display device, one of the plurality of images based on the respective degrees of attention;
change, by the apparatus, the displayed one of the plurality of images with another one of the plurality of images in accordance with a designation of the another one by the user; and
learn, by the neural network, so as to increase a degree of attention to be estimated for the designated another one of the plurality of images.

2. The apparatus according to claim 1 , wherein the degree of attention to be estimated for the designated image is set higher than a degree of attention to be estimated for an image which is not designated.

3. The apparatus according to claim 1 , wherein the degree of attention is estimated based on a set parameter, and the set parameter is updated so as to increase the degree of attention for the designated image.

4. The apparatus according to claim 3 , wherein the learning continues until the number of times of updating of the parameter reaches a preset number of times.

5. The apparatus according to claim 1 , wherein the leaning continues until the user inputs a predetermined instruction.

6. The apparatus according to claim 1 , wherein the learning is performed based on time of an image selection operation by the user.

7. The apparatus according to claim 1 , wherein an image is divided into a plurality of areas; and
degrees of attention to be estimated for respective divided areas are integrated.

8. The apparatus according to claim 7 , wherein the degrees of attention for the areas are integrated to a highest degree of attention for the areas.

9. The apparatus according to claim 7 , wherein the degrees of attention for the areas are integrated to an average of the degrees of attention for the areas.

10. The apparatus according to claim 1 , wherein the degree of attention estimated for an image and an identifier of the video monitoring camera that captured the image are included in learning data for the learning.

11. The apparatus according to claim 1 , wherein a parameter for the neural network is obtained from the learning.

12. The apparatus according to claim 11 , wherein the parameter for the neural network is updated so as to increase the difference between the degree of attention for the designated image and the degree of attention for the undesignated image.

13. The apparatus according to claim 1 , wherein the learning is performed in response to the acquisition of a predetermined times.

14. The apparatus according to claim 1 , wherein the degree of attention is estimated from a time-space image obtained by coupling a plurality of image frames.

15. A method of controlling a video monitoring apparatus to perform steps, the steps comprising:
acquiring, from a video monitoring camera, a video containing a plurality of images;
estimating, by a neural network, respective degrees of attention by a user for the plurality of images based on their image features;
displaying, by the video monitoring apparatus, on a video monitoring display device, one of the plurality of images based on the respective degrees of attention;
changing, by the video monitoring apparatus, the displayed one of the plurality of images with another one of the plurality of images in accordance with a designation of the another one by the user; and
learning, by the neural network so as to increase a degree of attention to be estimated for the designated another one of the plurality of images.

16. A non-transitory computer-readable storage medium storing a program which, when executed by a computer, causes the computer to execute steps of a method of controlling a video monitoring apparatus, the method comprising:
acquiring, from a video monitoring camera, a video containing a plurality of images;
estimating, by a neural network, respective degrees of attention by a user for the plurality of images based on their image features;
displaying, by the video monitoring apparatus, on a video monitoring display device, one of the plurality of images based on the respective degrees of attention;
changing, by the video monitoring apparatus, the displayed one of the plurality of images with another one of the plurality of images in accordance with a designation of the another one by the user; and
learning, by the neural network so as to increase a degree of attention to be estimated for the designated another one of the plurality of images.

17. A video monitoring system comprising:
a plurality of video monitoring cameras; and
a video monitoring apparatus communicably connected to the plurality of video monitoring cameras and configured to display images captured by the plurality of video monitoring cameras, wherein the video monitoring apparatus comprises:
one or more processors; and
one or more memories coupled to the one or more processors, the one or more memories having stored thereon instructions which, when executed by the one or more processors, cause the apparatus to:
acquire, from a video monitoring camera, a video containing a plurality of images;
estimate, by a neural network, respective degrees of attention by a user for the plurality of images based on their image features;
display, on a video monitoring display device, one of the plurality of images based on the respective degrees of attention;
change, by the apparatus, the displayed one of the plurality of images with another one of the plurality of images in accordance with a designation of the another one by the user; and
learning, by the neural network so as to increase a degree of attention to be estimated for the designated another one of the plurality of images.

18. The apparatus according to claim 1 , wherein the one of the plurality of images is displayed in a main screen with displaying, at least one of the plurality of images other than the image displayed in the main screen, in sub screens.

19. The apparatus according to claim 1 , wherein the plurality of images are acquired from a plurality of image capturing apparatuses.

Assignees

• Canon Kk

Inventors

• Yano Kotaro
• Kawai Tomoaki

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• A47G9/1027 Primary

  Details of inflatable pillows · CPC title