Systems and methods for identifying a target object in an image

Patent metadata

Field	Value
Publication number	US-10229347-B2
Application number	US-201715594611-A
Country	US
Kind code	B2
Filing date	May 14, 2017
Priority date	May 14, 2017
Publication date	Mar 12, 2019
Grant date	Mar 12, 2019

Abstract

There is provided a method of identifying objects in an image, comprising: extracting query descriptors from the image, comparing each query descriptor with training descriptors for identifying matching training descriptors, each training descriptor is associated with a reference object identifier and with relative location data (distance and direction from a center point of a reference object indicated by the reference object identifier), computing object-regions of the digital image by clustering the query descriptors having common center points defined by the matching training descriptors, each object-region approximately bounding one target object and associated with a center point and a scale relative to a reference object size, wherein the object-regions are computed independently of the identifier of the reference object associated with the object-regions, wherein members of each cluster point toward a common center point, and classifying the target object of each object-region according to the reference object identifier of the cluster.

First claim

What is claimed is:

1. A computed implemented method of identifying a plurality of target objects in a digital image, the method comprising:
receiving a digital image including a plurality of target objects;
extracting a plurality of query descriptors from respective a plurality of locations in the digital image;
comparing each one of said plurality of query descriptors with a plurality of training descriptors for identifying a plurality of matching training descriptors, each one of the plurality of training descriptors is associated with one of a plurality of reference object identifiers and with relative location data comprising an estimated distance and an estimated direction from a center point of a reference object indicated by the respective associated reference object identifier from the plurality of reference object identifiers;
computing a plurality of object-regions of the digital image by clustering the query descriptors having common center points defined by the matching training descriptors, each object-region approximately bounding one target object of the plurality of target objects of the digital image, each object-region is associated with another common center point of said common center points and with a scale relative to a reference object size,
wherein each of the plurality of object-regions is computed independently of the respective reference object identifier associated with said each of the plurality of object-regions and is computed by:
aggregating the relative location data of the matching training descriptors to generate a Kernel Density Estimate (KDE) for a plurality of posterior probability maps of the center point and scale of each respective reference object of the plurality of reference object identifiers;
aggregating the posterior probability maps into a plurality of probability map clusters;
extracting each of the plurality of object-regions with inter-scale normalization and non-maximal suppression according to location of the common center point and the scale of each respective cluster of the plurality of probability map clusters, wherein each of the plurality of object-regions is defined according to the center point and the scale of the reference object of the plurality of reference objects associated with the respective cluster of the plurality of probability map clusters; and

classifying the bound target object of each object-region of the plurality of object-regions according to the reference object identifier of a respective cluster according to a statistically significant correlation requirement between a common center point of the respective cluster and the center point of the reference object associated with the reference object identifier of the respective cluster.

2. The method of claim 1 , wherein the plurality of probability map clusters are each represented as an x-y-s-3D-heatmap having a center point at a location with coordinates defined by an x axis (x) and a y-axis(y) and defined by a scale (s).

3. The method of claim 1 , wherein each one of the plurality of training descriptors is associated with an estimated scale of one level of a plurality of levels of a Gaussian pyramid, and wherein the scale of the object-region is computed based on the estimated scale of the one level of the plurality of levels of the Gaussian pyramid of the identified matching training descriptors, wherein the object-regions are computed by clustering according to the scale defined by the matching training descriptors.

4. The method of claim 1 , further comprising:
selecting, for each bound target object of each object-region, a group of candidate reference object identifiers based on a cluster of query descriptors associated with the respective object-region;
computing a probability of each member of the group of candidate reference object identifiers being the respective target object; and
classifying the bound target object of each object-region of the plurality of object-regions according to the member of the group with the highest computed probability.

5. The method of claim 4 , wherein the probability of each member of the group of candidate reference object identifiers is computed based on one or more of the following components: a data fidelity value that penalizes distance between the query descriptor and the plurality of matching training descriptors, a penalty for deviation in expected spatial location between the common center point and the center point of the reference object associated with the plurality of matching training descriptors, and discrepancy in scale between the target object and the reference object associated with the plurality of matching training descriptors.

6. The method of claim 1 , wherein the plurality of training descriptors are extracted from a training set comprising a single training sample image of each of the plurality of reference objects associated with each of the plurality of reference object identifiers.

7. The method of claim 6 , wherein the plurality of image descriptions are extracted from each single training sample image of each of the plurality of reference objects in a sampling pattern that is denser relative to the pattern of the plurality of locations in the digital image.

8. The method of claim 1 , wherein the plurality of training descriptors are indexed with a sub-linear search data structure, and the comparing is performed by searching for the matching training descriptor of the extracted query descriptors within the sub-linear search data structure.

9. The method of claim 1 , wherein the extracted query descriptors and each of the plurality of training descriptors is based on scale invariant feature transform (SIFT).

10. The method of claim 1 , wherein the comparing is performed by finding a set of Euclidean nearest neighbors of the respective extracted query descriptors, wherein each member of the set of Euclidean nearest neighbors is one of the plurality of matching training descriptors.

11. The method of claim 10 , wherein the set of Euclidean nearest neighbors are identified for a first subset of the extracted query descriptors, wherein a second subset of extracted query descriptors are unmatched, wherein for each member of the second subset of extracted query descriptors that are unmatched, a matching training descriptor is computed such that the difference between the center point of the relative location data of the identified matching training descriptors and the center point of the relative location data of the computed training descriptor matched to the unmatched second subset of query descriptors is equal to the difference between the location relative location of the query descriptor matched to the identified matching training descriptor and the location relative location of the unmatched second subset of query descriptors for which the matching training descriptor is computed.

12. The method of claim 11 , wherein each member of the second subset of extracted query descriptors that are unmatched is paired to a closest single already matched query descriptor of the first subset of extracted query descriptors according to a distance requirement.

13. The method of claim 11 , further comprising iterating the computing of the matching training descriptor for each unmatched query descriptor of the second subset of query descriptors, wherein the probability of extending the matching training descriptor from nth closest members of the first subset of query descriptors is mathematically represented as (1−p)pn−1, where p denotes the probability of independently ignoring each previously matched query descriptor.

14. The method of claim 1 , further comp

Assignees

• IBM

Inventors

• Harary Sivan
• Karlinsky Leonid
• Marder Mattias
• Shtok Joseph
• Tzadok Asaf

Classifications

• G06V10/84

  using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06N20/00 Primary

  Machine learning · CPC title