Object retrieval

Patent metadata

Field	Value
Publication number	US-10013633-B1
Application number	US-201715453726-A
Country	US
Kind code	B1
Filing date	Mar 8, 2017
Priority date	Dec 23, 2014
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

Various approaches enable a user to capture image information (e.g., still images or video) about an object of interest such as the sole of a shoe or other piece of footwear (e.g., a sandal) and receive information about items that are determined to match footwear based at least in part on the image information. For example, an image analyze service or other similar service can analyze the images to determine a type of shoe included within the images based at least in part on patterns of other distinguishing features of the sole of the shoe. The image analysis service can aggregate the results and can provide information about the results as a set of matches or results to be displayed to a user in response to a visual search query. The information can include, for example, descriptions, contact information, availability, location data, pricing information, and other such information.

First claim

What is claimed is:

1. A computing device, comprising:
a camera configured to capture an image;
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the computing device to:
receive the image acquired by the camera, the image including a representation of a footwear sole;
analyze the image to determine at least one portion of that image that includes a view of at least a portion of a surface pattern of the footwear sole, the portion of the surface pattern of the footwear sole including repeated surface pattern markings separated by spacing, at least a first portion of the repeated surface pattern markings being located in a toe portion of the footwear sole, a second portion of the repeated surface pattern markings being located in a heel portion of the footwear sole, and a distinguishing feature separate from the surface pattern markings on the footwear sole including at least one of a logo or text;
determine feature information representative of at least a portion of the surface pattern or markings; and
return matching footwear information based at least in part on the feature information.

2. The computing device of claim 1 , wherein the instructions when executed to return matching footwear information further cause the computing device to:
compare the feature information to stored feature information representative of soles of footwear to determine a shoe type corresponding to the representation of the footwear sole.

3. The computing device of claim 2 , wherein the instructions when executed to return matching footwear information further cause the computing device to:
determine matching stored feature information with a respective similarity score that at least meets a matching threshold.

4. The computing device of claim 3 , wherein the instructions when executed to return matching footwear information further cause the computing device to:
identify, based at least in part on the matching feature information, at least one respective type of footwear represented in the image.

5. The computing device of claim 4 , wherein the instructions when executed to return matching footwear information further cause the computing device to:
provide, for display on the computing device, at least one of a description, contact information, availability, location data, or pricing information of the respective type of footwear.

6. A computing device, comprising:
a camera configured to capture an image;
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the computing device to:
receive the image acquired by the camera, the image including a representation of a surface pattern on a portion of a footwear sole, wherein the surface pattern comprises repeated markings separated by spacing, at least a first portion of the repeated markings being located in a toe portion of the footwear sole, a second portion of the repeated markings being located in a heel portion of the footwear sole, and a distinguishing feature separate from the repeated markings on the footwear sole including at least one of a logo or text;
analyze the image to determine at least one feature representative of a portion of the surface pattern;
compare the at least one feature to stored information to determine a type of footwear represented in the image; and
return matching footwear information based at least in part on the type of footwear represented in the image.

7. The computing device of claim 6 , wherein the instructions when executed to return matching footwear information further cause the computing device to:
identify, based at least in part on matching feature information to stored feature information, at least one respective type of footwear represented in the image.

8. The computing device of claim 7 , wherein the instructions when executed to return matching footwear information further cause the computing device to:
provide on a display of the computing device, in response to receiving the image, the identified at least one respective type of footwear for a user action.

9. The computing device of claim 6 , wherein the instructions when executed to analyze the image further cause the computing device to:
generate one or more codewords based at least in part on the analyzed image.

10. The computing device of claim 9 , wherein the instructions when executed to analyze the image further cause the computing device to:
generate a codebook based at least upon the one or more codewords.

11. The computing device of claim 9 , wherein the one or more codewords represent a particular appearance of the surface pattern on a sole of the footwear.

12. The computing device of claim 6 , wherein the instructions when executed to analyze the image further cause the computing device to:
analyze one or more intensity gradients of a region in the image to identify one or more potential edge locations.

13. The computing device of claim 6 , wherein the instructions when executed to analyze the image further cause the computing device to:
analyze one or more character portions using a character recognition algorithm to recognize at least one character associated with the one or more character portions.

14. The computing device of claim 6 , wherein the surface pattern includes a spacing between different regions of a sole of the footwear, an arrangement of the repeated markings, a distance between the markings, or a number of the markings.

15. The computing device of claim 6 , wherein a sole of the footwear further includes at least one of colors or art.

16. A method, comprising:
receiving an image acquired by a camera of a computing device, the image including a representation of a surface pattern on a portion of a footwear sole, wherein the surface pattern comprises repeated markings separated by spacing, at least a first portion of the repeated markings being located in a toe portion of the footwear sole, a second portion of the repeated markings being located in a heel portion of the footwear sole, and a distinguishing feature separate from the repeated markings on the footwear sole including at least one of a logo or text;
analyzing the image with an image analysis service associated with a content provider to determine at least one feature representative of a portion of the surface pattern;
generating one or more codewords using at least one of an edge detection algorithm, a feature detection algorithm, or a gradient detection algorithm;
comparing the at least one feature to stored information associated with the content provider to determine a type of shoe represented in the image, wherein the at least one feature is mapped to a particular codeword of the one or more codewords; and
returning matching footwear information based at least in part on the type of shoe represented in the image.

17. The method of claim 16 , further comprising:
identifying, based at least in part on matching feature information to stored feature information, at least one respective type of footwear represented in the image.

18. The method of claim 17 , further comprising:
providing on a display of the computing device, in response to receiving the image, the identified at least one respective type of footwear for a user action.

19. The method of claim 16 , wherein analyzing the image further comprises:
generating the one or more codewords based at least in part on the analyzed image.

20. The method of claim 19 , wherein gener

Assignees

• A9 Com Inc

Inventors

• Manmatha Raghavan
• Chuang Wei-Hong

Classifications

• G06T7/194 Primary

  involving foreground-background segmentation · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/421

  by analysing segments intersecting the pattern · CPC title

• G06T2210/16

  Cloth · CPC title