Method and system for interfacing with a user to facilitate an image search for an object-of-interest

Patent metadata

Field	Value
Publication number	US-11526549-B2
Application number	US-201916355470-A
Country	US
Kind code	B2
Filing date	Mar 15, 2019
Priority date	Mar 23, 2018
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

Methods, systems, and techniques for performing a facet search include receiving facet search commencement user input indicating that a search for a facet is to commence; in response to the facet search commencement user input, searching one or more video recordings for the facet; and displaying, on a display, facet image search results depicting the facet, wherein the facet image search results are selected from the one or more video recordings. An artificial neural network may be used for the facet search, and that network may be trained by generating a facet image training set that comprises training images, with the training images depicting a type of facet common to the training images; and training, by using the facet image training set, that neural network to classify the type of facet when a sample image comprising the type of facet is input to that network.

First claim

The invention claimed is:

1. A method comprising:
generating a facet image training set that comprises training images, wherein the training images depict a type of facet shared by the training images, wherein the training images depict the type of facet in conjunction with a type of object shared by the training images;
training, by using the facet image training set, an artificial neural network to classify the type of facet when one or more sample images comprising the type of facet is input to the artificial neural network;
classifying the one or more sample images using the artificial neural network to assess whether the one or more sample images depict the type of facet;
after the one or more sample images have been classified, searching the one or more sample images for the type of facet using different states of the artificial neural network as trained by different users, the searching using the different states occurring independent of each other; and
determining search results based on a weighting of different images found during the searching of the one or more sample images using the different states.

2. The method of claim 1 , wherein the artificial neural network comprises a convolutional neural network.

3. The method of claim 1 , wherein training the artificial neural network comprises recording state data of the artificial neural network corresponding to the different states of the artificial neural network during the training by the different users.

4. The method of claim 3 , wherein the state data is indexed to index data comprising at least one of the type of facet, identification credentials of a user who is performing the training, the training images, cameras used to capture the training images, timestamps of the training images, and a time when the training commenced.

5. The method of claim 3 , further comprising:
receiving index data corresponding to an earlier state of the artificial neural network; and
reverting to the earlier state of the artificial neural network by loading the state data indexed to the index data corresponding to the earlier state.

6. The method of claim 3 , wherein the searching performed using more than one of the different states results in intermediate search results respectively corresponding to the more than one of the different states, and wherein:
the weighting of the different images is based on how frequently the different images occur in the intermediate search results.

7. The method of claim 1 , wherein the type of facet comprises age, gender, a type of clothing, a color of clothing, a pattern displayed on clothing, a hair color, a footwear color, or clothing accessories.

8. The method of claim 1 , wherein the type of facet comprises color, make, model, or configuration.

9. The method of claim 1 , wherein at least one of the training images comprises an image chip derived from an image captured by a camera.

10. The method of claim 1 , wherein classifying the one or more sample images using the artificial neural network to assess whether the one or more sample images depicts the type of facet comprises generating and storing metadata indicating whether the one or more sample images depicts the type of facet, and wherein searching the one or more sample images for the type of facet is performed using the metadata.

11. The method of claim 1 , further comprising:
receiving facet search commencement user input indicating that a search for a facet is to commence, wherein the searching is performed in response to receiving the facet search commencement user input; and
displaying, on a display, facet image search results depicting the facet, wherein the facet image search results are selected from one or more video recordings, wherein the facet image search results depict the facet in conjunction with a type of object-of-interest shared by the facet image search results.

12. A method comprising:
receiving facet search commencement user input indicating that a search for a facet is to commence, wherein the facet comprises a data structure that includes a descriptor and a tag, provided in a pair, that describe a particular visual characteristic of an object-of-interest;
in response to the facet search commencement user input, searching one or more video recordings for the facet using different states of an artificial neural network as trained by different users, the searching using the different states occurring independent of each other;
displaying, on a display, facet image search results depicting the facet, wherein the facet image search results are selected from the one or more video recordings, wherein the facet image search results depict the facet in conjunction with a type of object-of-interest shared by the facet image search results;
prior to receiving the facet search commencement user input, displaying a list of facets appearing in object-of-interest search results; and
receiving, as the facet search commencement user input, a selection of the facet comprising the list of facets.

13. The method of claim 12 , further comprising, after displaying the facet image search results:
receiving object-of-interest search commencement user input indicating that a search for the object-of-interest is to commence;
in response to the object-of-interest search commencement user input, searching the one or more video recordings for the object-of-interest; and
displaying, on the display, the object-of-interest search results depicting the object-of-interest.

14. The method of claim 13 , wherein the one or more video recordings that arc searched arc the one or more video recordings from which arc selected the facet image search results, wherein the object-of-interest search results are selected from the one or more video recordings from which are selected the facet image search results, and wherein the object-of-interest search results depict the object-of-interest and the facet.

15. The method of claim 13 , further comprising, after displaying the object-of-interest search results:
receiving updated facet search commencement user input indicating that an updated facet search is to commence;
in response to the updated facet search commencement user input, searching the one or more video recordings from which are selected the object-of-interest search results for a different type or number of facets than were searched in the search for the facet; and
displaying, on the display, updated facet search results depicting the different type or number of facets and the object-of-interest, wherein the updated facet search results are selected from the one or more video recordings from which are selected the object-of-interest search results.

16. The method of claim 12 , further comprising, before displaying the facet image search results:
receiving the object-of-interest search commencement user input indicating that a search for the object-of-interest is to commence;
in response to the object-of-interest search commencement user input, searching one or more video recordings for the object-of-interest; and
displaying, on the display, the object-of-interest search results depicting the object-of-interest, wherein the object-of-interest search results are selected from the one or more video recordings,
wherein the facet search commencement user input is received after the object-of-interest search results are displayed, and the one or more video recordings that are searched for the facet comprise the one or more video recordings from which are selected the object-of-interest search results.

17. The method of any claim

Assignees

• Avigilon Corp
• Motorola Solutions Inc

Inventors

• Alcantara Tulio De Souza
• Cheng Courtney Anne Adame
• Donaldson Roger David
• Flanagan David
• Jessen Ken
• Sjue Eric

Classifications

• G06F16/7837 Primary

  using objects detected or recognised in the video content · CPC title

• G06F16/743 Primary

  a collection of video files or sequences · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06V10/945

  User interactive design; Environments; Toolboxes · CPC title

• G06N3/045

  Combinations of networks · CPC title