Recognition Procedure for Identifying Multiple Items in Images

Patent metadata

Field	Value
Publication number	US-2015049902-A1
Application number	US-201313967294-A
Country	US
Kind code	A1
Filing date	Aug 14, 2013
Priority date	Aug 14, 2013
Publication date	Feb 19, 2015
Grant date	—

Abstract

The disclosure includes a system and method for identifying multiple items in an image. A image recognition application receives a query image of items, computes features of the query image, for each feature finds an indexed image with closest matched features in a database, determines that the shape of the matched features is geometrically consistent, determines whether the query image matches the indexed image, responsive to the query image matching the indexed image, returns inliers, determines a region of interest where the match was found in the query image, removes inliers from the set of features to reduce the set of features and returns all matches found when the query image fails to match the indexed image.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
receiving, with one or more processors, a query image of items; computing, with the one or more processors, features of the query image; comparing, with the one or more processors, the set of features of the query image to features of indexed images in an electronic database; determining, with the one or more processors, that a comparison of the features forms a shape that is geometrically consistent; determining, with the one or more processors, whether the query image matches the indexed image; responsive to the query image matching the indexed image, returning inliers; determining, with the one or more processors, a region of interest where the match was found in the query image; removing inliers from the set of features to reduce the set of features; for each feature, finding the indexed image with the closest matched features in the database until the query image fails to match the indexed image; and returning all matches found.

2 . The method of claim 1 , wherein the features comprise location, orientation and an image descriptor.

3 . The method of claim 1 , wherein the inliers are a set of matched features.

4 . The method of claim 1 further comprising:
determining whether the query image matches the indexed image;
blurring out the matched region in the query image; and
computing features of the query image until the query image fails to match the indexed image.

5 . The method of claim 1 , wherein the items are products and further comprising generating a planogram of indexed products.

6 . The method of claim 5 , further comprising analyzing products based on the query images and the planogram.

7 . The method of claim 6 , further comprising generating statistics about a location of products and frequency of purchase.

8 . A system comprising:
one or more processors; a controller stored on a memory and executable by the one or more processors, the controller configured to receive a query image of items; a feature identifier coupled to the controller and configured to compute features of the query image; a matching engine coupled to the feature identifier and configured to compare the set of features of the query image to features of indexed images in an electronic database, determine that a comparison of the features forms a shape that is geometrically consistent and determine whether the query image matches the indexed image; wherein the feature identifier returns inliers, determines a region of interest where the match was found in the capture image, removes inliers from the set of features to reduce the set of features; and wherein for each feature, the matching engine finds the indexed image with the closest matched features in the database until the query image fails to match the indexed image and returns all matches found.

9 . The system of claim 8 , wherein the features comprise location, orientation and an image descriptor.

10 . The system of claim 8 , wherein the inliers are a set of matched features.

11 . The system of claim 8 , wherein the matching engine is further configured to determine whether the query image matches the indexed image, further comprising a blurring engine that blurs out the matched region in the query image and wherein the feature identifier computes features of the query image until the query image fails to match the indexed image.

12 . The system of claim 8 , wherein the items are products and further comprising an analyzer that is configured to generate a planogram of indexed products.

13 . The system of claim 12 , wherein the analyzer is further configured to analyze products based on the query images and the planogram.

14 . The system of claim 13 , wherein the analyzer is further configured to generate statistics about a location of products and frequency of purchase.

15 . A computer program product comprising a computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:
receive a query image of items; compute features of the query image; compare the set of features of the query image to features of indexed images in an electronic database; determine that a comparison of the features from a shape that is geometrically consistent; determine whether the query image matches the indexed images; responsive to the query image matching the indexed image, return inliers; determine a region of interest where the match was found in the query image; remove inliers from the set of features to reduce the set of features; for each feature, find the indexed image with the closest matched features in the database until the query image fails to match the indexed image; and return all matches found.

16 . The computer program product of claim 15 , wherein the features comprise location, orientation and an image descriptor.

17 . The computer program product of claim 15 , wherein the inliers are a set of matched features.

18 . The computer program product of claim 15 , wherein the computer readable program when executed on the computer causes the computer to also:
determine whether the query image matches the indexed image; blur out the matched region in the query image; and compute features of the query image until the query image fails to match the indexed image.

19 . The computer program product of claim 15 , wherein the items are products and wherein the computer readable program when executed on the computer causes the computer to also generate a planogram of indexed products.

20 . The computer program product of claim 19 , wherein the computer readable program when executed on the computer causes the computer to also analyze products based on the query images and the planogram.

Assignees

• Moraleda Jorge
• Pavani Sri Kaushik
• Garg Anshul
• Ricoh Co Ltd

Inventors

• Moraleda Jorge
• Pavani Sri Kaushik
• Garg Anshul

Classifications

• G06V30/424 Primary

  Postal images, e.g. labels or addresses on parcels or postal envelopes · CPC title

• G06V30/19167

  Active pattern learning · CPC title

• G06F18/2178

  based on feedback of a supervisor · CPC title

• G06V10/449

  Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters · CPC title

• G06K9/6202 Primary

  Physics · mapped topic