Product labeling review

Patent metadata

Field	Value
Publication number	US-12067797-B2
Application number	US-202117408181-A
Country	US
Kind code	B2
Filing date	Aug 20, 2021
Priority date	Aug 20, 2020
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

A label processing engine receives, as inputs, raw data representative of a label and baseline data, detects a raw data object within the raw data, classifies the raw data object, and localizes the raw data object within the raw data, detects a baseline data object within the baseline data, classifies the baseline data object, and localizes the baseline data object within the baseline data. The engine recognizes corresponding text within the raw data object and the baseline data object and extracts the corresponding text within the raw data object and the baseline data object, reassembles the corresponding text of the raw data object and the baseline data object into respective lines of text, compares the respective lines of text with one another, and issues a notification based on the comparison.

First claim

The invention claimed is:

1. A system for analyzing a product label, the system comprising:
a label processing engine configured to:
receive a first input including raw data representative of the product label and a second input including baseline data;
detect a raw data object within the raw data, classify the raw data object into a first class of a plurality of classes by associating the raw data object with the first class, and localize the raw data object within the raw data;
detect a baseline data object within the baseline data, classify the baseline data object into a second class of the plurality of classes by associating the baseline data object with the second class, and localize the baseline data object within the baseline data;
recognize corresponding text within the raw data object and the baseline data object and extract the corresponding text within the raw data object and the baseline data object;
reassemble the corresponding text of the raw data object and the baseline data object into respective lines of text;
compare the respective lines of text with one another; and
one of,
issue a first notification indicating that the respective lines of text match in response to determining that the respective lines of text match, and
issue a second notification indicating that the respective lines of text do not match in response to determining that the respective lines of text do not match,

wherein the label processing engine is further configured to perform contour detection to generate a plurality of candidate bound regions within data of each of the raw data input and the baseline data input and classify as a nutrition facts box one of the plurality of candidate bound regions based on a magnitude of an overlap between the one of the plurality of candidate bound regions and a proposed region generated by an customized object detection model of the label processing engine.

2. The system of claim 1 , wherein to extract the corresponding text within the raw data object and the baseline data object comprises to identify whether the corresponding text comprises one of a text sequence, a number, a percentage, and a measurement unit.

3. The system of claim 1 , wherein, prior to detecting, the label processing engine is further configured to convert each of the raw data and the baseline data from a first format into a second format, and wherein the first format comprises a portable document format and the second format comprises a high-resolution image format.

4. The system of claim 1 , wherein the raw data comprises a nutrition label and the baseline data comprises a label information document.

5. The system of claim 1 , wherein the raw data object is a nutrition facts box and the baseline data object is a baseline nutrition facts box.

6. The system of claim 1 , wherein the baseline data is tabulated.

7. The system of claim 6 , wherein the label processing engine is further configured to, prior to detecting the baseline data object, convert the tabulated baseline data from a Java-based tabulated format into a JavaScript Object Notation (JSON) format.

8. The system of claim 1 , wherein prior to classifying the raw data object, the label processing engine is configured to identify one of a region and a market associated with the raw data object, and wherein classifying the raw data object into the first class includes classifying the raw data object according to the identified one of the region and the market.

9. A method for processing a product label, the method comprising:
receiving, by a label processing engine, a first input including raw data representative of the label and a second input including baseline data;
detecting a raw data object within the raw data, classifying the raw data object into a first class of a plurality of classes by associating the raw data object with the first class, and localizing the raw data object within the raw data;
detecting a baseline data object within the baseline data, classifying the baseline data object into a second class of the plurality of classes by associating the baseline data object with the second class, and localizing the baseline data object within the baseline data;
recognizing corresponding text within the raw data object and the baseline data object and extracting the corresponding text within the raw data object and the baseline data object;
reassembling the corresponding text of the raw data object and the baseline data object into respective lines of text;
comparing the respective lines of text with one another;
one of,
issuing a first notification indicating that the respective lines of text match in response to determining that the respective lines of text match, and
issuing a second notification indicating that the respective lines of text do not match in response to determining that the respective lines of text do not match; and

performing, via the label processing engine, contour detection to generate a plurality of candidate bound regions within data of each of the raw data input and the baseline data input and classify as a nutrition facts box one of the plurality of candidate bound regions based on a magnitude of an overlap between the one of the plurality of candidate bound regions and a proposed region generated by an customized object detection model of the label processing engine.

10. The method of claim 9 , wherein extracting the corresponding text within the raw data object and the baseline data object comprises identifying whether the corresponding text comprises one of a text sequence, a number, a percentage, and a measurement unit.

11. The method of claim 9 further comprising, prior to detecting, converting each of the raw data and the baseline data from a first format into a second format, and wherein the first format comprises a portable document format and the second format comprises a high-resolution image format.

12. The method of claim 9 , wherein the plurality of classes are a nutrition facts box, a thumbnail, an ingredients list, and a net contents portion.

13. The method of claim 9 , wherein the raw data comprises a nutrition label and the baseline data comprises a label information document.

14. The method of claim 13 further comprising, prior to detecting the baseline data object, detecting that the baseline data is tabulated and converting the tabulated baseline data from a Java-based tabulated format into a JavaScript Object Notation (JSON) format.

15. A system for processing a product label, the system comprising:
an object detection module configured to, in response to receiving a raw data input and a baseline data input, detect corresponding objects within data of each of the raw data input and the baseline data input, and classify the corresponding objects using a customized object detection model;
a text recognition module configured to, in response to receiving the corresponding objects, recognize text in each of the corresponding objects and extract the text to classify the text using a customized classification model; and
a content comparison module configured to compare the text of the corresponding objects with one another, using a character-by-character approach, and issue a notification in response to one of identifying a discrepancy or identifying a match,
wherein to detect and classify the corresponding objects comprises to perform contour detection to generate a plurality of candidate bound regions within data of each of the raw data input and the baseline data input and classify as a nutrition facts box one of the plurality of candidate bound regions based on a magnitude of an overlap be

Assignees

• Pepsico Inc

Inventors

• Hui Jingting

Classifications

• G06V30/10

  Character recognition · CPC title

• G06V30/413 Primary

  Classification of content, e.g. text, photographs or tables · CPC title

• G06F40/12

  Use of codes for handling textual entities · CPC title

• G06F40/183

  Tabulation, i.e. one-dimensional [1D] positioning · CPC title

• G06F16/35

  Clustering; Classification · CPC title