Context-based image recognition for consumer market research

Patent metadata

Field	Value
Publication number	US-9569692-B2
Application number	US-201414530129-A
Country	US
Kind code	B2
Filing date	Oct 31, 2014
Priority date	Oct 31, 2014
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

Example methods, apparatus, systems to perform context-based image recognition for consumer market research are disclosed. Disclosed example methods include comparing a first subset of feature points in a first region of interest of a reference image with a first subset of feature points of a first image residing within a first region corresponding spatially to the first region of interest and located at a first location in the first image to determine a first match value. Disclosed example methods also include, in response to determining the first match value satisfies a first threshold, comparing a second subset of feature points in a second region of interest of the reference image with a second subset of the feature points of the first image residing within a second region of the first image corresponding spatially to the second region of interest to determine a second match value.

First claim

What is claimed is:

1. A method to determine whether a first image depicts a product, the method comprising:
receiving the first image, the first image captured by a mobile device;
comparing, by executing an instruction with a processor, a first subset of feature points in a first region of interest of a reference image with a first subset of feature points of the first image residing within a first region corresponding spatially to the first region of interest and located at a first location in the first image to determine a first match value;
in response to determining the first match value satisfies a first threshold, comparing, by executing an instruction with the processor, a second subset of feature points in a second region of interest of the reference image with a second subset of the feature points of the first image residing within a second region of the first image corresponding spatially to the second region of interest to determine a second match value; and
in response to determining the second match value satisfies a second threshold lower than the first threshold, associating, by executing an instruction with the processor, the first image with product information associated with the reference image.

2. The method as defined in claim 1 , wherein the second region of the first image is located at a second location in the first image, the second location being selected relative to the first location based on a spatial relationship between the first region of interest and the second region of interest in the reference image.

3. The method as defined in claim 2 , further including, in response to determining the first match value does not satisfy the first threshold:
selecting a third subset of feature points of the first image within a third region of the first image corresponding spatially to the first region of interest and located at a third location in the first image to determine a third match value; and
in response to determining the third match value satisfies the first threshold, comparing, with the processor, the second subset of feature points in the second region of interest of the reference image with a fourth subset of feature points of the first image residing within a fourth region corresponding spatially to the second region of interest and located at a fourth location in the first image to determine a fourth match value, the fourth location being selected relative to the third location based on the spatial relationship between the first region of interest and the second region of interest in the reference image.

4. The method as defined in claim 1 , wherein the comparing of the first subset of feature points in the first region of interest of the reference image with the first subset of feature points of the first image includes:
calculating distances between the first subset of feature points in the first region of interest of the reference image and the first subset of feature points of the first image to determine a percentage of the first subset of feature points in the first region of interest of the reference image that match corresponding feature points from the first subset of feature points of the first image; and
calculating the first match value based on the percentage.

5. The method as defined in claim 1 , wherein the reference image is associated with a plurality of regions of interest having respective assigned significance values, and further including:
selecting the first region of interest having a highest significance value out of the plurality of regions of interest associated with the reference image; and
setting the first threshold to correspond to a first significance value associated with the first region of interest.

6. The method as defined in claim 1 , wherein the reference image is a first reference image associated with the product, the product is associated with a plurality of different reference images, and further including:
in response to determining the second match value satisfies the second threshold:
incrementing a first index value associated with the first reference image;
decrementing index values associated with respective ones of the plurality of reference images other than the first reference image; and

discarding ones of the plurality of reference images associated with index values that do not satisfy a third threshold.

7. The method as defined in claim 1 , further including improving accuracy of an image matcher by utilizing different first and second thresholds adjusted to a number of false positive matches between the reference images and captured images depicting products to be identified.

8. The method as defined in claim 1 , further including:
in response to determining the second match value satisfies the second threshold, sending the product information associated with the reference image to the mobile device.

9. An apparatus to determine whether a first digital image depicts a product, the apparatus comprising:
a feature point extractor to determine feature points in the first digital image; and
a reference comparator to:
compare a first subset of feature points in a first region of interest of a digital reference image with a first subset of the feature points of the first digital image residing within a first region corresponding spatially to the first region of interest and located at a first location in the first digital image to determine a first match value;
in response to determining the first match value satisfies a first threshold, compare a second subset of feature points in a second region of interest of the digital reference image with a second subset of the feature points of the first digital image residing within a second region of the first digital image corresponding spatially to the second region of interest to determine a second match value; and
in response to determining the second match value satisfies a second threshold lower than the first threshold, associate the first digital image with product information associated with the digital reference image, at least one of the feature point extractor or the reference comparator including hardware.

10. The apparatus as defined in claim 9 , wherein the second region of the first digital image is located at a second location in the first image, the second location being selected relative to the first location based on a spatial relationship between the first region of interest and the second region of interest in the reference image.

11. The apparatus as defined in claim 9 , wherein to compare the first subset of feature points in the first region of interest of the reference image with the first subset of feature points of the first image, the reference comparator is further to:
calculate distances between the first subset of feature points in the first region of interest of the reference image and the first subset of feature points of the first digital image to determine a percentage of the first subset of feature points in the first region of interest of the reference image that match corresponding feature points from the first subset of feature points of the first image; and
calculate the first match value based on the percentage.

12. The apparatus as defined in claim 9 , wherein the reference image is associated with a plurality of regions of interest having respective assigned significance values, and the reference comparator is further to:
select the first region of interest having a highest significance value out of a plurality of regions of interest associated with the reference image; and
set the first threshold to correspond to a first significance value associated with the first region of interest

Assignees

• Nielsen Co Us Llc

Inventors

• Xie Wei
• Bivins Michael Allen
• Jones Steve
• Deng Kevin Keqiang
• Schiller Brian
• Sullivan Jonathan
• Terrazas Alejandro
• Sheppard Michael

Classifications

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06Q30/0201 Primary

  Market modelling; Market analysis; Collecting market data · CPC title

• G06F16/50

  of still image data · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title