Method and system for interfacing with a user to facilitate an image search for a person-of-interest

Patent metadata

Field	Value
Publication number	US-10810255-B2
Application number	US-201715705076-A
Country	US
Kind code	B2
Filing date	Sep 14, 2017
Priority date	Sep 14, 2017
Publication date	Oct 20, 2020
Grant date	Oct 20, 2020

Abstract

Methods, systems, and techniques for interfacing with a user to facilitate an image search for a person-of-interest. A face thumbnail of the person-of-interest, a body thumbnail of the person-of-interest, and image search results of the person-of-interest are concurrently displayed on a display to help a user identify the person-of-interest who appears in one or more of a collection of video recordings. The user may provide feedback to the system regarding whether image search results show the person-of-interest, which feedback may be used to refine the image search results.

First claim

The invention claimed is:

1. A method for interfacing with a user to facilitate an image search for a person-of-interest, the method comprising:
concurrently displaying, on a display, a face thumbnail of the person-of-interest, a body thumbnail of the person-of-interest, and image search results of the person-of-interest;
displaying a selected video recording concurrently with the face thumbnail, body thumbnail, and image search results, wherein the person-of-interest is displayed in the selected video recording;
displaying a boundary box over the person-of-interest in the selected video recording;
receiving search commencement user input indicating that a search for the person-of-interest is to commence;
in response to the search commencement user input:
using as the body thumbnail at least a portion of an image frame from the selected video recording contained within the boundary box;
searching a collection of video recordings for the person-of-interest, wherein the collection of video recordings comprises the selected video recording; and
using as part of the face thumbnail a face of the person-of-interest identified during the searching of the collection of video recordings;

determining an appearance likelihood plot for the person-of-interest, wherein the appearance likelihood plot depicts a likelihood that the person-of-interest appears in the collection of video recordings over a time span; and
displaying, concurrently with the image search results, the appearance likelihood plot.

2. The method of claim 1 , wherein the time span is divided into time intervals, and wherein:
determining the appearance likelihood plot comprises determining, for each of the time intervals, a likelihood that the person-of-interest appears in the collection of video recordings for the time interval; and
displaying the appearance likelihood plot comprises, for each of the time intervals, displaying a likelihood indicator representing the likelihood that the person-of-interest appears in the collection of video recordings for the time interval.

3. The method of claim 2 , wherein, for each of the time intervals, the likelihood that the person-of-interest appears in the collection of video recordings for the time interval is determined as a maximum likelihood that the person-of-interest appears in any one of the collection of video recordings for the time interval.

4. The method of claim 3 , wherein the appearance likelihood plot comprises a bar graph comprising a single bar for each of the time intervals.

5. The method of claim 1 , further comprising displaying an appearance likelihood plot resizable selection window over the appearance likelihood plot, wherein the resizable selection window overlays at least a portion of the time span and the image search results are selected only from the collection of video recordings corresponding to the portion of the time span.

6. The method of claim 5 , further comprising:
concurrently displaying a timeline with the appearance likelihood plot, wherein the timeline covers a time span comprising at least part of a duration of the collection of video recordings;
displaying a timeline resizable selection window over the timeline;
receiving window resizing user input indicating that the user is resizing one of the selection windows; and
in response to the window resizing user input, adjusting the other of the selection windows such that both of the selection windows cover identical time spans.

7. The method of claim 1 , wherein the image search results comprise a first image and a second image, the body thumbnail comprises at least a portion of the first image, the face thumbnail comprises at least a portion of the second image, and further comprising:
displaying a first and a second indicator over the first and second images indicating that the first and second images are bases for the body and face thumbnails, respectively; and
displaying the first and second indicators on the appearance likelihood plot at positions corresponding to times the first and second images appear in the collection of video recordings.

8. The method of claim 7 , further comprising concurrently displaying with the appearance likelihood plot:
a timeline, wherein the timeline covers a time span comprising at least part of a duration of the collection of video recordings; and
the first and second indicators on the timeline at positions corresponding to times the first and second images appear in the collection of video recordings.

9. A method for interfacing with a user to facilitate an image search for a person-of-interest, the method comprising:
concurrently displaying, on a display, a face thumbnail of the person-of-interest, a body thumbnail of the person-of-interest, and image search results of the person-of-interest, wherein each of the image search results is positioned on the display according to a likelihood that the image search result comprises the person-of-interest;
receiving match confirmation user input regarding whether at least one of the image search results depicts the person-of-interest;
determining, using the match confirmation user input, whether positioning of the image search results is to be changed in response to the match confirmation user input; and
when the positioning of the image search results is to be changed, updating the positioning of the image search results in response to the match confirmation user input.

10. The method of claim 1 , further comprising displaying a selected video recording concurrently with the face thumbnail, body thumbnail, and image search results.

11. The method of claim 10 , wherein the person-of-interest is displayed in the selected video recording, and further comprising:
displaying a boundary box over the person-of-interest in the selected video recording;
receiving search commencement user input indicating that a search for the person-of-interest is to commence; and
in response to the search commencement user input, using as the body thumbnail at least a portion of an image frame from the selected video recording contained within the boundary box.

12. The method of claim 11 , further comprising, in response to the search commencement user input:
searching a collection of video recordings for the person-of-interest, wherein the collection of video recordings comprises the selected video recording; and
using as part of the face thumbnail a face of the person-of-interest identified during the searching of the collection of video recordings.

13. The method of claim 12 , wherein the collection of video recordings comprises video recordings generated concurrently using different video cameras.

14. The method of claim 12 , wherein the image search results comprise a first image and a second image, the body thumbnail comprises at least a portion of the first image, the face thumbnail comprises at least a portion of the second image, and the first and second images are different.

15. The method of claim 12 , wherein searching the collection of video recordings for the person-of-interest comprises:
performing a face search on the collection of video recordings to identify the person-of-interest; and
performing a body search on the collection of video recordings to identify the person-of-interest,

wherein the image search results comprise results from the face search and results from the body search.

16. The method of claim 1 , wherein the image search results comprise a first image and a second image, the body thumbnail comprises at least a portion of the first image, the face thumbnail comprises at least

Assignees

• Avigilon Corp

Inventors

• Alcantara Tulio
• Doumbouya Moussa
• Sjue Eric
• Valbonesi Hannah
• Weston William Christopher

Classifications

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06F16/5838

  using colour · CPC title

• G06F16/784 Primary

  the detected or recognised objects being people · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title