Hierarchical sampling for object identification

Patent metadata

Field	Value
Publication number	US-11423248-B2
Application number	US-202017061262-A
Country	US
Kind code	B2
Filing date	Oct 1, 2020
Priority date	Oct 1, 2019
Publication date	Aug 23, 2022
Grant date	Aug 23, 2022

Abstract

Aspects of the present disclosure include methods, systems, and non-transitory computer readable media that perform the steps of receiving a first plurality of snapshots, generating a first plurality of descriptors each associated with the first plurality of snapshots, grouping the first plurality of snapshots into at least one cluster based on the plurality of descriptors, selecting a representative snapshot for each of the at least one cluster, generating at least one second descriptor for the representative snapshot for each of the at least one cluster, wherein the at least one second descriptor is more complex than the first plurality of descriptors, and identifying a target by applying the at least second descriptor to a second plurality of snapshots.

First claim

What is claimed is:

1. A method of hierarchical sampling, comprising:
receiving a first plurality of snapshots;
generating a first plurality of descriptors each associated with the first plurality of snapshots;
grouping the first plurality of snapshots into at least one cluster based on the plurality of descriptors;
selecting a representative snapshot for each of the at least one cluster;
generating at least one second descriptor for the representative snapshot for each of the at least one cluster, wherein the at least one second descriptor is more complex than the first plurality of descriptors; and
identifying a target by applying the at least second descriptor to a second plurality of snapshots.

2. The method of claim 1 , further comprising, prior to receiving the first plurality of snapshots:
receiving a third plurality of snapshots;
generating a third plurality of descriptors each associated with the third plurality of snapshots, wherein the third plurality of descriptors are less complex than the first plurality of descriptors;
grouping the third plurality of snapshots into a plurality of clusters based on the third plurality of descriptors; and
selecting a second plurality of representative snapshots as the first plurality of snapshots.

3. The method of claim 1 , further comprising receiving a plurality of tracks, wherein the first plurality of snapshots is from one of the plurality of tracks.

4. The method of claim 1 , further comprising:
classifying the representative snapshot for each of the at least one cluster;
aggregating classification scores of the representative snapshot for each of the at least one cluster;
determining a class based on the aggregated classification scores; and
wherein the at least one descriptor is a class-specific descriptor.

5. The method of claim 1 , further comprising, prior to selecting the representative snapshot, estimating a mean average precision (MAP) for each snapshot in the at least one cluster.

6. The method of claim 5 , wherein selecting the representative snapshot for each of the at least one cluster comprises selecting a snapshot in the at least one cluster having a highest estimated MAP.

7. The method of claim 6 , further comprising:
computing similarity indices of remaining snapshots in the at least one cluster; and
removing a subset of snapshots in the at least one cluster having similarity indices exceeding a predetermined threshold.

8. The method of claim 5 , wherein estimating a MAP comprises using a neural network to estimate the MAP.

9. A non-transitory computer readable medium comprising instructions stored therein that, when executed by a processor of a system, cause the processor to:
receive a first plurality of snapshots;
generate a first plurality of descriptors each associated with the first plurality of snapshots;
group the first plurality of snapshots into at least one cluster based on the plurality of descriptors;
select a representative snapshot for each of the at least one cluster;
generate at least one second descriptor for the representative snapshot for each of the at least one cluster, wherein the at least one second descriptor is more complex than the first plurality of descriptors; and
identify a target by applying the at least second descriptor to a second plurality of snapshots.

10. The non-transitory computer readable medium of claim 9 , further comprising instructions stored therein that, when executed by the processor of the system, cause the processor to, prior to receiving the first plurality of snapshots:
receive a third plurality of snapshots;
generate a third plurality of descriptors each associated with the third plurality of snapshots, wherein the third plurality of descriptors are less complex than the first plurality of descriptors;
group the third plurality of snapshots into a plurality of clusters based on the third plurality of descriptors; and
select a second plurality of representative snapshots as the first plurality of snapshots.

11. The non-transitory computer readable medium of claim 9 , further comprising instructions stored therein that, when executed by the processor of the system, cause the processor to receive a plurality of tracks, wherein the first plurality of snapshots is from one of the plurality of tracks.

12. The non-transitory computer readable medium of claim 9 , further comprising instructions stored therein that, when executed by the processor of the system, cause the processor to:
classify the representative snapshot for each of the at least one cluster;
aggregate classification scores of the representative snapshot for each of the at least one cluster;
determine a class based on the aggregated classification scores; and
wherein the at least one descriptor is a class-specific descriptor.

13. The non-transitory computer readable medium of claim 9 , further comprising instructions stored therein that, when executed by the processor of the system, cause the processor to, prior to select the representative snapshot, estimate a Mean Average Precision for each snapshot in the at least one cluster.

14. The non-transitory computer readable medium of claim 13 , wherein the instructions for selecting the representative snapshot for each of the at least one cluster comprises instructions that, when executed by the processor of the system, cause the processor to select a snapshot in the at least one cluster having a highest estimated mean average precision (MAP).

15. The non-transitory computer readable medium of claim 14 , further comprising instructions stored therein that, when executed by the processor of the system, cause the processor to:
compute similarity indices of remaining snapshots in the at least one cluster; and
remove a subset of snapshots in the at least one cluster having similarity indices exceeding a predetermined threshold.

16. The non-transitory computer readable medium of claim 13 , wherein the instructions for estimating a MAP comprises instructions that, when executed by the processor of the system, cause the processor to use a neural network to estimate the MAP.

17. A system, comprising:
memory that stores instructions; and
a processor configured to execute the instructions to:
receive a first plurality of snapshots;
generate a first plurality of descriptors each associated with the first plurality of snapshots;
group the first plurality of snapshots into at least one cluster based on the plurality of descriptors;
select a representative snapshot for each of the at least one cluster;
generate at least one second descriptor for the representative snapshot for each of the at least one cluster, wherein the at least one second descriptor is more complex than the first plurality of descriptors; and
identify a target by applying the at least second descriptor to a second plurality of snapshots.

18. The system of claim 17 , wherein the processor is further configured to execute the instructions to, prior to receiving the first plurality of snapshots:
receive a third plurality of snapshots;
generate a third plurality of descriptors each associated with the third plurality of snapshots, wherein the third plurality of descriptors are less complex than the first plurality of descriptors;
group the third plurality of snapshots into a plurality of clusters based on the third plurality of descriptors; and
select a second plurality of representative snapshots as the first plurality of snapshots.

19. The system of claim 17 , wherein the processor is further c

Assignees

• Johnson Controls Tyco IP Holdings LLP

Inventors

• Amores Llopis Jaume
• Lee Young M
• Westmacott Ian C
• Falik Yohay
• Rozner Amit

Classifications

• G06V20/47 Primary

  Detecting features for summarising video content · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/2137

  based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps · CPC title

• G06V2201/07

  Target detection · CPC title