Information processing system, method for controlling information processing system, and storage medium

Patent metadata

Field	Value
Publication number	US-11308158-B2
Application number	US-201916404941-A
Country	US
Kind code	B2
Filing date	May 7, 2019
Priority date	Sep 20, 2018
Publication date	Apr 19, 2022
Grant date	Apr 19, 2022

Abstract

Provided is an information processing system, including a processor, a storage device and a display device. The storage device stores images captured by imaging devices. If an image of a moving object is selected as a search query, then the processor executes a similarity search on the images stored in the storage device, with the selected image as the search query. The display device displays images attained as results of the similarity search. If an image of a moving object is added as a search query, then the processor executes a similarity search with all selected images as search queries. The display device updates images to be displayed such that images attained as results of the similarity search performed with all the selected images as the search queries are displayed.

First claim

What is claimed is:

1. An information processing system, comprising:
a processor;
a storage device coupled to the processor; and
a display device coupled to the processor,
wherein the storage device stores a plurality of images captured by a plurality of imaging devices,
wherein if an image of a moving object is selected as a search query, then the processor executes a similarity search on the plurality of images stored in the storage device, with the selected image as the search query,
wherein the display device displays a plurality of images attained as results of the similarity search,
wherein if an image of a moving object is added as a search query, then the processor executes a similarity search with all selected images as search queries, and
wherein the display device updates images to be displayed such that a plurality of images attained as results of the similarity search performed with all the selected images as the search queries are displayed,
wherein the processor detects moving objects from the images captured by the imaging devices and stored in the storage device,
wherein the processor determines whether the moving objects detected in images captured by a same imaging device at differing times are a same moving object,
wherein, if identification information of the imaging device and a time are inputted, the display device displays a plurality of images, among the images stored in the storage device, of the same moving object as the moving object of the selected image that was detected in the plurality of images captured by the imaging device identified by the inputted identification information during a time period including the inputted time,
wherein if one or more of the plurality of images displayed by the display device are added as search queries, then the processor executes a similarity search with all the selected images as the search queries, and
wherein the display device:
displays an image of a map that displays a position of the imaging device that captured the image selected as the search query, and
if an additional image of the moving object is selected as a search query, then updates the image of the map so as to add to the display the position of the imaging device that captured the additionally selected image.

2. The information processing system according to claim 1 ,
wherein the display device displays the plurality of images attained as the results of the similarity search in chronological order of being captured.

3. The information processing system according to claim 2 ,
wherein the display device displays information indicating a time at which each image attained as the results of the similarity search was captured.

4. The information processing system according to claim 3 ,
wherein the information indicating the time at which each said image was captured is a difference between a current time and the time at which each said image was captured.

5. The information processing system according to claim 1
wherein when any one of the plurality of images attained as results of the similarity search is selected, the display device displays the plurality of images of the same moving object as the moving object in the selected image that was detected in the plurality of images captured during a time period including a time at which the image selected by the imaging device was captured, and
wherein if one or more of the plurality of images displayed by the display device are added as search queries, then the processor executes a similarity search with all the selected images as the search queries.

6. The information processing system according to claim 1 ,
wherein the display device displays all of the images selected as the search queries in chronological order of being captured.

7. The information processing system according to claim 1 ,
wherein the processor calculates, for each of the selected images, a score indicating a degree of similarity of a feature value thereof to feature values of images of a moving object detected in the images stored in the storage device, and outputs images with the highest scores as search results.

8. The information processing system according to claim 1 ,
wherein the processor performs clustering on feature values of the plurality of selected images so as to classify the feature values into a plurality of clusters, calculates a score indicating a degree of similarity of a representative value of each cluster to feature values of images of a moving object detected in images stored in the storage device, and outputs the images with the highest scores as the search results.

9. A method for controlling an information processing system having a processor, a storage device coupled to the processor, and a display device coupled to the processor,
wherein the storage device stores a plurality of images captured by a plurality of imaging devices, and
wherein the method comprises the steps of:
executing, by the processor, if an image of a moving object is selected as a search query, a similarity search on the plurality of images stored in the storage device, with the selected image as the search query;
displaying, by the display device, a plurality of images attained as results of the similarity search;
executing, by the processor, if an image of a moving object is added as a search query, a similarity search with all selected images as search queries;
updating, by the display device, images to be displayed such that a plurality of images attained as results of the similarity search performed with all the selected images as the search queries are displayed;
detecting, by the processor, moving objects from the images captured by the imaging devices and stored in the storage device,
determining, by the processor, whether the moving objects detected in images captured by a same imaging device at differing times are a same moving object,
if identification information of the imaging device and a time are inputted, displaying, by the display device, a plurality of images, among the images stored in the storage device, of the same moving object as the moving object of the selected image that was detected in the plurality of images captured by the imaging device identified by the inputted identification information during a time period including the inputted time;
if one or more of the plurality of images displayed by the display device are added as search queries, executing, by the processor, a similarity search with all the selected images as the search queries;
displaying, by the display device, an image of a map that displays a position of the imaging device that captured the image selected as the search query; and
if an additional image of the moving object is selected as a search query, updating, by the display device, the image of the map so as to add to the display the position of the imaging device that captured the additionally selected image.

10. A non-transitory computer-readable storage medium that stores programs that control the computer,
wherein the computer has a processor, a storage device coupled to the processor, and a display device coupled to the processor,
wherein the storage device stores a plurality of images captured by a plurality of imaging devices, and
wherein the program causes the processor to:
execute, if an image of a moving object is selected as a search query, a similarity search on the plurality of images stored in the storage device, with the selected image as the search query;
output, to the display device, data for displaying a plurality of images attained as results of the similarity search;
execute, if an image of a moving object is added as a search query, a similarity search

Assignees

• Hitachi Ltd

Inventors

• Matsubara Daisuke
• Nukaga Nobuo

Classifications

• G06F16/54

  Browsing; Visualisation therefor · CPC title

• G06F16/532 Primary

  Query formulation, e.g. graphical querying · CPC title

• G06F16/75 Primary

  Clustering; Classification · CPC title

• G06F16/55 Primary

  Clustering; Classification · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title