Computer vision product recognition

Patent metadata

Field	Value
Publication number	US-10521844-B2
Application number	US-201414553062-A
Country	US
Kind code	B2
Filing date	Nov 25, 2014
Priority date	Nov 25, 2014
Publication date	Dec 31, 2019
Grant date	Dec 31, 2019

Abstract

A system for computer-aided visual recognition of products may be used by a customer operating a mobile computing device having a camera. A customer can direct the camera at a targeted product for which the customer desires to purchase or learn information. Image recognition operations can be carried out to compare the captured image against images from a prepopulated product image store of known products. Upon a positive match, information regarding the identified product may be gathered and transmitted to the customer and displayed on the mobile computing device. The customer may be presented with various options regarding the identified product, including adding the product to a shopping list, a to-do list, a wish list, or other types of lists.

First claim

What is claimed is:

1. A computer-implemented method for identifying a product, comprising:
receiving, at a processing device of a product recognition server, one or more signals from a mobile computing device corresponding to consecutively captured images of a targeted product or a packaging of the targeted product, wherein the targeted product comprises the targeted product offered for sale by a retailer or recently consumed by a user, or an advertisement of the targeted product, wherein the mobile computing device comprises smart glasses;
analyzing, at the processing device, the consecutively captured images of the targeted product by comparing each image of the consecutively captured images to a plurality of images of retail products offered for sale by the retailer;
receiving, with the processing device, a match score between the consecutively captured images of the targeted product and the plurality of images of retail products offered by the retailer;
when the match score is lower than a pre-determined minimum match score threshold, determining, at the processing device, that the consecutively captured images of the targeted product does not match the plurality of images of retail products and that the retailer does not offer the targeted product for sale; and
when the match score is higher than the pre-determined minimum match score threshold:
determining, at the processing device, that the consecutively captured images of the targeted product matches at least one image of the plurality of images of retail products offered for sale by the retailer;
confirming, at the processing device, a product selection of the targeted product after identifying the targeted product within a pre-determined number of the consecutively captured images to minimize false positive identifications of the targeted product;
obtaining, from a product database, information associated with the targeted product;
transmitting, from the processing device, the information to the smart glasses by a graphical user interface within a field of view of the user; and
displaying, on a display of the smart glasses, by the graphical user interface, at least one of a text, graphic, or video including the information.

2. The method of claim 1 , wherein the information comprises a retail price of the targeted product.

3. The method claim 1 , wherein the information comprises an availability status of the targeted product at a selected retail store.

4. The method of claim 3 , wherein the selected retail store is selected based on physical proximity to the mobile computing device.

5. The method of claim 1 , wherein analyzing, at the processing device, the consecutively captured images of the targeted product to the plurality of images of retail products further comprises retrieving the plurality of images of retail products from a product image store.

6. The method of claim 1 , wherein analyzing, at the processing device, the consecutively captured images of the targeted product to the plurality of images of retail products further comprises applying a feature detection algorithm to the consecutively captured images of the targeted product.

7. The method of claim 6 , wherein applying the feature detection algorithm to the consecutively captured images of the targeted product comprises carrying out a Speeded-Up Robust Features (SURF) feature detection algorithm.

8. The method of claim 1 , wherein analyzing, at the processing device the consecutively captured images of the targeted product to the plurality of images of retail products further comprises disregarding multiple ones of the consecutively captured images when an undesired product is captured by the consecutively captured images.

9. The method of claim 1 , further comprising:
receiving from the mobile computing device, a signal of the one or more signals indicating an input to add the targeted product to a list; and
transmitting an instruction to a customer database to add the targeted product to a list associated with an operator of the mobile computing device.

10. The method of claim 9 , wherein the list comprises a shopping list.

11. A method for identifying a product, comprising:
receiving, at a processing device or a product recognition server, one or more signals from a camera of a mobile computing device corresponding to capturing consecutively captured images of a targeted product or a packaging of the targeted product, wherein the targeted product comprises the targeted product offered for sale by a retailer or recently consumed by a user, or an advertisement of the targeted product, wherein the mobile computing device comprises smart glasses;
analyzing, at the processing device, the consecutively captured images of the targeted product by comparing each image of the consecutively captured images to a plurality of images of retail products offered for sale by the retailer;

receiving, with the processing device, a match score between the consecutively captured images of the targeted product and the plurality of images of retail products offered by the retailer;
when the match score is lower than a pre-determined minimum match score threshold, determining, at the processing device, that the consecutively captured images of the targeted product does not match the plurality of images of retail products and that the retailer does not offer the targeted product for sale; and
when the match score is higher than the pre-determined minimum match score threshold:
determining, at the processing device, that the consecutively captured images of the targeted product matches at least one image of the plurality of images of retail products offered for sale by the retailer;
confirming, at the processing device, a product selection of the targeted product after identifying the targeted product within a pre-determined number of the consecutively captured images to minimize false positive identifications of the targeted product;
obtaining, from a product database, information associated with the targeted product;

transmitting, from the processing device, the information to the smart glasses by a graphical user interface within a field of view of the user; and
displaying on a display of the smart glasses, by the graphical user interface, at least one of a text, graphic, or video including the information.

12. A computer-implemented system for identifying a product comprising:
one or more processors;
a product recognition server comprising a memory device and a processing device, the memory device storing computer-readable instructions directing the processing device to:
receive one or more signals from a mobile computing device corresponding to consecutively captured images of a targeted product or a packaging of the targeted product, wherein the targeted product comprises the targeted product offered for sale by a retailer or recently consumed by a user, or an advertisement of the targeted product, wherein the mobile computing device comprises smart glasses;
analyze the consecutively captured images of the targeted product by comparing each image of the consecutively captured images to a plurality of images of retail products offered for sale by the retailer;
receive a match score between the consecutively captured images of the targeted product and the plurality of images of retail products offered by the retailer;
when the match score is lower than a pre-determined minimum match score threshold, determine that the consecutively captured images of the targeted product does not match the plurality of images of retail products and that the retailer does not offer the targeted product for sale; and
when the match score is higher than the

Assignees

• Wal Mart Stores Inc
• Walmart Apollo Llc

Inventors

• Gupta Ankit
• Shah Saurav
• Sowdaboina Venkata Pranay Kumar

Classifications

• G06Q30/0623 Primary

  by investigating goods or services · CPC title

• G06Q30/0633

  Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087) · CPC title

• G06Q30/0639

  Locating goods or services, e.g. based on physical position of the goods or services within a shopping facility · CPC title

• G06F16/5838

  using colour · CPC title