Three-dimensional indexing protocol

Patent metadata

Field	Value
Publication number	US-10460287-B2
Application number	US-201615218904-A
Country	US
Kind code	B2
Filing date	Jul 25, 2016
Priority date	Aug 14, 2013
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

The disclosure includes a system and method for indexing synthetically modified images of a high quality image. An image recognition application receives images of a product, crops background regions from the images, scales the image based on a minimum value among width and height of the image and generates multiple image sizes, blurs the images, brightens the image and indexes the images as being associated with the product. The images can be of box-shaped packages that include four or six images or cylindrical packages that include, for example, eight images of the packages. The images can be indexed in a k-dimensional tree for faster retrieval.

First claim

What is claimed is:

1. A method comprising:
generating, using one or more processors, a planogram of indexed products;
receiving, by the one or more processors, a captured image of a plurality of products;
identifying, by the one or more processors, the plurality of products in the captured image using the indexed products by:
computing a first set of features for the captured image including a location, an orientation, and an image descriptor for the first set of features;
comparing the first set of features for the captured image to features of a first indexed product to determine whether the first set of features for the captured image can be transformed to the features of the first indexed product by a combination of translation, rotation, and scaling;
determining the captured image matches the first indexed product responsive to determining that the first set of features for the captured image can be transformed to the features of the first indexed product by the combination of translation, rotation, and scaling;
blurring a region of a first product in the captured image to generate a partially blurred image, the first product matching the first indexed product;
computing a second set of features for the partially blurred image and comparing the second set of features for the partially blurred image to features of a second indexed product to determine whether the second set of features can be transformed to the features of the second indexed product by a combination of translation, rotation, and scaling;
determining the partially blurred image matches the second indexed product responsive to determining that the second set of features for the partially blurred image can be transformed to the features of the second indexed product by the combination of translation, rotation, and scaling; and
returning matches found for the plurality of products in the captured image;

analyzing, by the one or more processors, the plurality of products using the captured image and the planogram of indexed products; and
notifying, by the one or more processors, a user of a condition based on the analysis.

2. The method of claim 1 , wherein analyzing the plurality of products further comprises determining, by the one or more processors, whether a change in inventory of the first product satisfies a threshold.

3. The method of claim 2 , wherein determining whether the change in inventory of the first product satisfies a threshold includes determining whether the first product is out of stock.

4. The method of claim 3 , wherein notifying the user includes an instruction to restock the first product.

5. The method of claim 1 , wherein analyzing the plurality of products further comprises checking whether locations of the plurality of products in the captured image match expected locations based on the planogram of indexed products.

6. The method of claim 1 , wherein analyzing the plurality of products further comprises generating sales statistics for the first product.

7. The method of claim 1 , further comprising:
receiving, by the one or more processors, an image of the first product;
receiving, by the one or more processors, information relating to the first product; and
indexing, by the one or more processors, the image of the first product and the information relating to the first product as being associated with the first product.

8. A system comprising:
one or more processors; and
a memory, the memory storing instructions, which when executed cause the one or more processors to:
generate a planogram of indexed products;
receive a captured image of a plurality of products;
identify the plurality of products in the captured image using the indexed products by:
computing a first set of features for the captured image including a location, an orientation, and an image descriptor for the first set of features;
comparing the first set of features for the captured image to features of a first indexed product to determine whether the first set of features for the captured image can be transformed to the features of the first indexed product by a combination of translation, rotation, and scaling;
determining the captured image matches the first indexed product responsive to determining that the first set of features for the captured image can be transformed to the features of the first indexed product by the combination of translation, rotation, and scaling;
blurring a region of a first product in the captured image to generate a partially blurred image, the first product matching the first indexed product;
computing a second set of features for the partially blurred image and comparing the second set of features for the partially blurred image to features of a second indexed product to determine whether the second set of features can be transformed to the features of the second indexed product by a combination of translation, rotation, and scaling;
determining the partially blurred image matches the second indexed product responsive to determining that the second set of features for the partially blurred image can be transformed to the features of the second indexed product by the combination of translation, rotation, and scaling; and
returning matches found for the plurality of products in the captured image;

analyze the plurality of products using the captured image and the planogram of indexed products; and
notify a user of a condition based on the analysis.

9. The system of claim 8 , wherein to analyze the plurality of products, the instructions further cause the one or more processors to determine whether a change in inventory of the first product satisfies a threshold.

10. The system of claim 9 , wherein the change in inventory of the first product satisfying the threshold indicates the first product is out of stock.

11. The system of claim 10 , wherein to notify the user the instructions further cause the one or more processors to include an instruction to restock the first product.

12. The system of claim 8 , wherein to analyze the plurality of products the instructions further cause the one or more processors to check whether locations of the plurality of products in the captured image match expected locations based on the planogram of indexed products.

13. The system of claim 8 , wherein to analyze the plurality of products the instructions further cause the one or more processors to generate sales statistics for the first product.

14. The system of claim 8 , wherein the instructions further cause the one or more processors to:
receive, by the one or more processors, an image of the first product;
receive, by the one or more processors, information relating to the first product; and
index, by the one or more processors, the image of the first product and the information relating to the first product as being associated with the first product.

15. A computer program product comprising a non-transitory computer usable medium including a computer readable program, wherein the computer readable program, when executed on a computer causes the computer to:
generate a planogram of indexed products;
receive a captured image of a plurality of products;
identify the plurality of products in the captured image using the indexed products by:
computing a first set of features for the captured image including a location, an orientation, and an image descriptor for the first set of features;
comparing the first set of features for the captured image to features of a first indexed product to determine whether the first set of features for the captured image can be transformed to the features of th

Assignees

• Pavani Sri Kaushik
• Garg Anshul
• Ricoh Co Ltd

Inventors

• Pavani Sri Kaushik
• Garg Anshul

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title

• G06F16/5854

  using shape and object relationship · CPC title

• G06Q40/12 Primary

  Accounting · CPC title

• G06T7/0004

  Industrial image inspection · CPC title