Semi-supervised price tag detection

Patent metadata

Field	Value
Publication number	US-9911033-B1
Application number	US-201615256648-A
Country	US
Kind code	B1
Filing date	Sep 5, 2016
Priority date	Sep 5, 2016
Publication date	Mar 6, 2018
Grant date	Mar 6, 2018

Abstract

A method comprising: training a price tag detector, comprising a gross feature detector and a classifier, to automatically detect a price tag in an image, by: a) training the gross feature detector using supervised learning with labeled images, and b) training the classifier using a two-phase hybrid learning process comprising: c) applying an initial supervised learning using the labeled images, yielding a semi-trained version of the classifier, and d) applying a subsequent unsupervised learning using unlabeled images, yielding a fully trained version of the classifier, wherein applying the unsupervised learning comprises: for each unlabeled image: i) detecting multiple price tag hypotheses using the gross feature detector, ii) classifying each price tag hypothesis using the semi-trained classifier, ii) rating each classification based contextual data extracted from the unlabeled image, iv) retraining the semi-trained classifier with the rated classifications, and repeating steps ii) through iv) until the reclassification converges.

First claim

What is claimed is:

1. A method comprising using at least one hardware processor for:
training a price tag detector that comprises a gross feature detector and a classifier, to automatically detect a price tag in an image, by:
a) training the gross feature detector using a supervised learning process with a set of labeled images, and
b) training the classifier using a two-phase hybrid learning process comprising:
applying an initial supervised learning phase using the set of labeled images, yielding a semi-trained version of the classifier, and
applying a subsequent unsupervised learning phase using a set of unlabeled images, yielding a fully trained version of the classifier,
wherein the applying of the unsupervised learning phase comprises, for each unlabeled image:
i) detecting multiple price tag hypotheses using the gross feature detector,
ii) classifying each price tag hypothesis using the semi-trained classifier,
iii) rating each classification based contextual data extracted from the unlabeled image,
iv) retraining the semi-trained classifier with the rated classifications, and
repeating steps ii) through iv) until the reclassification converges thereby yielding the trained classifier.

2. The method of claim 1 , wherein the gross feature detector comprises a Viola Jones Cascade algorithm, and wherein the labeled images comprise multiple categories of aspect ratios and price tag formats, wherein the gross feature detector comprises an aggregation of multiple feature detection algorithms, each trained on a different one of the categories.

3. The method of claim 1 , wherein the classifier is a support vector machine (SVM), wherein the price tag hypotheses comprise any of a price tag detection and a false positive detection, and wherein each price tag hypothesis is formulated as a Fisher vector by:
transforming each price tag hypothesis to a canonical size,
calculating one or more local descriptors for each price tag hypothesis using a dense scale-invariant feature transform (SIFT),
concatenating the one or more local descriptors to form a single concatenated vector,
training a Gaussian mixture model (GMM) for each local descriptor, and
using the GMMs to calculate a Fisher representation for the concatenated vector,
wherein classifying each price tag hypothesis using the semi-trained classifier comprises classifying the Fisher vectors using the SVM.

4. The method of claim 1 , wherein the contextual data is selected from the group consisting of: a color attribute, a texture attribute, a depth attribute, a flatness attribute, a relative position of any of the detected features, a detected edge, a detected product, a detected alphanumeric character, a color of the alphanumeric character, a size of the alphanumeric character, a font of the alphanumeric character, and a background color.

5. The method of claim 4 , further comprising using the at least one hardware processor for extracting the contextual data by using an algorithm selected from the group consisting of: an optical character recognition (OCR) algorithm, a product recognition algorithm, an edge detection algorithm, a depth detection algorithm, and a contour detection algorithm.

6. The method of claim 1 , further comprising: capturing an image, and using the at least one hardware processor for detecting a price tag in the captured image using the price tag detector.

7. A computer program product comprising a non-transitory computer-readable storage medium having program code embodied thereon, the program code executable by at least one hardware processor to:
train a price tag detector that comprises a gross feature detector and a classifier, to automatically detect a price tag in an image, by:
a) training the gross feature detector using a supervised learning process with a set of labeled images, and
b) training the classifier using a two-phase hybrid learning process comprising:
applying an initial supervised learning phase using the set of labeled images, and yielding a semi-trained version of the classifier, and
applying a subsequent unsupervised learning phase using a set of unlabeled images, yielding a fully trained version of the classifier,
wherein applying the unsupervised learning phase comprises:
for each unlabeled image:
i) detecting multiple price tag hypotheses using the gross feature detector,
ii) classifying each price tag hypothesis using the semi-trained classifier,
iii) rating each classification based contextual data extracted from the unlabeled image,
iv) retraining the semi-trained classifier with the rated classifications, and

repeating steps ii) through iv) until the reclassification converges thereby yielding the trained classifier.

8. The computer program product of claim 7 , wherein the gross feature detector comprises a Viola Jones Cascade algorithm, and wherein the labeled images comprise multiple categories of aspect ratios and price tag formats, wherein the gross feature detector comprises an aggregation of multiple feature detection algorithms, each trained on a different one of the categories.

9. The computer program product of claim 7 , wherein the price tag hypotheses comprise any of a price tag detection and a false positive detection, and wherein the the program code is further executable to:
represent each price tag hypothesis as a Fisher vector by:
transforming each price tag hypothesis to a canonical size,
calculating one or more local descriptors for each price tag hypothesis using a dense scale-invariant feature transform (SIFT),
concatenating the one or more local descriptors to form a single concatenated vector, training a Gaussian mixture model (GMM) for each local descriptor, and
using the GMMs to calculate a Fisher representation for the concatenated vector,

wherein classifying each price tag hypothesis using the semi-trained classifier comprises classifying the Fisher vectors using the SVM.

10. The computer program product of claim 7 , wherein the contextual data is selected from the group consisting of: a color attribute, a texture attribute, a depth attribute, a flatness attribute, a relative position of any of the detected features, a detected edge, a detected product, a detected alphanumeric character, a color of the alphanumeric character, a size of the alphanumeric character, a font of the alphanumeric character, and a background color.

11. The computer program product of claim 10 , wherein the contextual data is extracted by using an algorithm selected from the group consisting of: an optical character recognition (OCR) algorithm, a product recognition algorithm, an edge detection algorithm, a depth detection algorithm, and a contour detection algorithm.

12. A system, comprising:
at least one hardware processor; and
a non-transitory memory device having embodied thereon program code executable by said at least one hardware processor to:
train a price tag detector that comprises a gross feature detector and a classifier, to automatically detect a price tag in an image, by:
a) training the gross feature detector using a supervised learning process with a set of labeled images, and
b) training the classifier using a two-phase hybrid learning process comprising:
applying an initial supervised learning phase using the set of labeled images, and yielding a semi-trained version of the classifier, and
applying a subsequent unsupervised learning phase using a set of unlabeled images, yielding a fully trained version of the classifier,
wherein applying the unsupervised learning phase comprises:
for each unlabeled image:
i) detecting multiple price tag hypotheses using the gross feature detector,
ii) classifying e

Assignees

• IBM

Inventors

• Harary Sivan
• Marder Mattias

Classifications

• G06V30/18152 Primary

  Extracting features based on a plurality of salient regional features, e.g. "bag of words" · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06T7/33

  using feature-based methods · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title