Image object category recognition method and device

Patent metadata

Field	Value
Publication number	US-10013636-B2
Application number	US-201415032460-A
Country	US
Kind code	B2
Filing date	Sep 26, 2014
Priority date	Nov 4, 2013
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

The present invention relates to an image object category recognition method and device. The recognition method comprises an off-line autonomous learning process of a computer, which mainly comprises the following steps: image feature extracting, cluster analyzing and acquisition of an average image of object categories. In addition, the method of the present invention also comprises an on-line automatic category recognition process. The present invention can significantly reduce the amount of computation, reduce computation errors and improve the recognition accuracy significantly in the recognition process.

First claim

The invention claimed is:

1. An image object category recognition method, comprising the following steps:
an image feature extracting step (S 1 ) of extracting feature points of all sampling images in N known categories with a feature point extracting method, and establishing a known category—sampling image—feature point correspondence, where N is a natural number greater than 1, and each category comprises at least one sampling image;
a cluster analyzing step (S 2 ) of performing cluster analysis on all of the feature points extracted using a clustering algorithm, and dividing the feature points into N subsets;
an object category determining step (S 3 ) of determining an object category C n for each of the subsets;
a common feature acquiring step (S 4 ) of acquiring common features among the images in each object category C n with a search algorithm, where C n is the n th object category, and n is a positive integer no more than N,
wherein the step S 4 comprises at least the following sub-steps:
S 401 of searching a set of common feature points sharing common features among images included in each object category C n by means of the search algorithm; and
S 402 of additionally mapping sampling images having the largest number of common feature points among the set of common feature points from the each object category C n based upon the set of common feature points searched by means of the known category—sampling image—feature point correspondence, and using the sampling images as average images of the object category C n ; and
after the step S 4 , an on-line image recognizing and categorizing step S 5 of recognizing and categorizing an image to be categorized by comparing the image to be categorized with at least one of the common feature points among the images in each object category C n and the key feature points in the average images in each object category C n .

2. The method according to claim 1 , wherein the step S 1 comprises at least the following sub-steps:
S 101 of extracting the feature points among low-level visual features of each of the sampling images;
S 102 of acquiring a vector description for each of the feature points; and
S 103 of establishing a known category—sampling image—feature point correspondence.

3. The method according to claim 2 , wherein the step S 2 comprises at least the following sub-steps:
S 201 of clustering all of the feature points extracted into a predetermined number of clusters by means of the clustering algorithm; and
S 202 of constructing the clusters into a k-tree structure, wherein k is a positive integer, and k∈(1, N).

4. The method according to claim 3 , wherein the step S 3 comprises at least the following sub-steps:
S 301 of counting a number of the feature points that belong to different known categories for each subset of the N subsets; and
S 302 of determining a known category that includes the largest number of feature points as the object category C n .

5. The method according to claim 1 , wherein a number of common feature points (K(C n )) in the set of common feature points is determined based upon a number of feature points of an image having the smallest number of feature points in the object category C n .

6. The method according to claim 1 , wherein the on-line image recognizing and categorizing step S 5 comprises:
S 502 of performing the same image feature extracting processing on the image to be categorized as the step S 1 to extract feature points of the image to be categorized;
S 503 of comparing the feature points extracted from the image to be categorized with each of the common features of the each object category C n among the n object categories to compute similarity between the image to be categorized and each object category respectively; and
S 504 of attributing the image to be categorized to an object category C n having the greatest similarity.

7. The method according to claim 1 , wherein the on-line image recognizing and categorizing step S 5 comprises:
S 502 of performing the same image feature extracting processing on the image to be categorized as the step S 1 to extract feature points of the image to be categorized;
S 503 ′ of comparing each of the feature points extracted from the image to be categorized with each feature point in average images of the object category one by one to calculate similarity between the image to be categorized and each average image of the object category respectively; and
S 504 of attributing the image to be categorized to an object category C n having the greatest similarity.

8. The method according to claim 1 , wherein prior to the step S 1 , further comprising an image preprocessing step for each image, the image preprocessing step comprising:
S 001 of scaling proportionally the image;
S 002 of performing a filtering processing on the image scaled proportionally to remove noise; and
S 003 of performing a graying processing on the image filtering processed.

9. The method according to claim 1 , wherein,
the feature point extracting method is a SIFT algorithm, with which SIFT key feature points of each image and SIFT descriptors of each key feature point are extracted;
the clustering algorithm is a k-means algorithm, and the key feature points are divided into the N subsets by constructing the k-tree, where k is a positive integer and k∈(1, N); and
the search algorithm is a KNN nearest neighbor search algorithm.

10. An image object category recognition device, comprising:
an image feature extracting unit configured to extract feature points of all sampling images in N known categories with a feature point extracting method, and establish a known category—sampling image—feature point correspondence, where N is a natural number greater than 1, each category comprises at least one sampling image;
a cluster analyzing unit configured to perform cluster analysis on all of the feature points extracted using a clustering algorithm, and dividing the feature points into N subsets;
a determining unit for determining an object category C n for each of the subsets;
an acquiring unit for acquiring common features among the images included in each object category C n with a search algorithm, where C n is the n th object category, and n is a positive integer no more than N,
wherein the acquiring unit includes at least the following sub-modules:
a searching module for searching a set of common feature points sharing common features among images included in each object category C n by means of the search algorithm; and
a mapping module for mapping sampling images having the largest number of common feature points among the set of common feature points from the each object category C n by means of the known category—sampling image—feature point correspondence, and using the sampling images as average images of the object category C n ;

an on-line image recognizing and categorizing unit for recognizing and categorizing an image to be categorized by comparing the image to be categorized with at least one of the common feature points among the images in each object category C n and the key feature points in the average images in each object category C n .

11. The device according to claim 10 , wherein the determining unit comprises at least the following sub-modules:
a counting module for counting a number of the feature points that belong to different known categories for each subset of the N subsets; and
a determining module for determining a known category that includes the largest number of feature points as the object category C n .

12. The method according to claim 1 , wherein the on-line i

Assignees

• Beijing Jingdong Shangke Information Technology Co Ltd
• Beijing Jingdong Century Trading Co Ltd
• Beijing Jingdong Shangke Information Technology Co Ltd

Inventors

• Gan Yongzhou
• Deng Zhengping

Classifications

• G06V10/7625

  Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms · CPC title

• G06V10/771 Primary

  Feature selection, e.g. selecting representative features from a multi-dimensional feature space · CPC title

• G06F18/23213

  with fixed number of clusters, e.g. K-means clustering · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/231

  Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram · CPC title