Methods, systems, apparatus and articles of manufacture for receipt decoding

Patent metadata

Field	Value
Publication number	US-11768993-B2
Application number	US-202217883309-A
Country	US
Kind code	B2
Filing date	Aug 8, 2022
Priority date	Nov 22, 2019
Publication date	Sep 26, 2023
Grant date	Sep 26, 2023

Abstract

Methods, apparatus, systems and articles of manufacture are disclosed for receipt decoding. An example apparatus includes processor circuitry to execute instructions to extract text from the receipt image, the text including bounding boxes; associate ones of the bounding boxes to link horizontally related fields of a the receipt image by selecting a first bounding box; identifying first horizontally aligned bounding boxes, the first horizontally aligned bounding boxes to include at least one bounding box of the bounding boxes that is horizontally aligned relative to the first bounding box; adding the first horizontally aligned bounding boxes to a word sync list; and connecting ones of the first horizontally aligned bounding boxes and the first bounding box based on at least one of an amount of the first horizontally aligned bounding boxes in the word sync list and a relationship among the first horizontally aligned bounding boxes and the first bounding box.

First claim

What is claimed is:

1. An apparatus comprising:
interface circuitry to receive a receipt image;
at least one memory;
machine readable instructions; and
programmable circuitry to execute the machine readable instructions to at least:
extract machine readable text from the receipt image, the machine readable text to include bounding boxes, each bounding box of the bounding boxes to include a respective set of coordinates relative to the receipt image;
associate ones of the bounding boxes to link horizontally related fields of a receipt represented in the receipt image by:
selecting a first bounding box;
identifying first horizontally aligned bounding boxes, the first horizontally aligned bounding boxes to include at least one bounding box of the bounding boxes that is horizontally aligned relative to the first bounding box, the at least one bounding box of the bounding boxes to have a center coordinate that is positioned between a vertical minimum of the first bounding box and a vertical maximum of the first bounding box;
adding the first horizontally aligned bounding boxes to a word sync list; and
connecting ones of the first horizontally aligned bounding boxes and the first bounding box based on at least one of an amount of the first horizontally aligned bounding boxes in the word sync list and a relationship among the first horizontally aligned bounding boxes and the first bounding box.

2. The apparatus of claim 1 , wherein the related fields of the receipt correspond to a first product from the receipt image and not a second product from the receipt image.

3. The apparatus of claim 1 , wherein the first bounding box is one of a single bounding box or a group of connected bounding boxes.

4. The apparatus of claim 3 , wherein the amount of the first horizontally aligned bounding boxes in the word sync list is two, and wherein the relationship is an orientation, the programmable circuitry is to execute the instructions to:
identify a first orientation corresponding to the first bounding box, the orientation indicative of which one of the two first horizontally aligned bounding boxes to connect to the first bounding box;
select the one of the two first horizontally aligned bounding boxes based on the first orientation; and
connect the first bounding box and the selected one of the two first horizontally aligned bounding boxes.

5. The apparatus of claim 1 , wherein, when the first bounding box does not include an orientation indicative of a trend, and the relationship is an association, and the programmable circuitry is to execute the machine readable instructions to:
determine a first association between the first bounding box and each a first one of the two first horizontally aligned bounding boxes in the word sync list and a second association between the first bounding box and a second one of the two first horizontally aligned bounding boxes in the word sync list;
select one of the two first horizontally aligned bounding boxes based on the first and second associations; and
connect the first bounding box and the selected one of the two first horizontally aligned bounding boxes.

6. The apparatus of claim 5 , wherein the first and second associations between the first bounding box and respective ones of the two first horizontally aligned bounding boxes are based on connection coefficients, the programmable circuitry is to execute the machine readable instructions to:
calculate a first connection coefficient corresponding to the first one of the two first horizontally aligned bounding boxes;
calculate a second connection coefficient corresponding to the second one of the two first horizontally aligned bounding boxes;
compare the first and second connection coefficients; and
select the one of the two first horizontally aligned bounding boxes having a higher connection coefficient.

7. The apparatus of claim 6 , wherein each of the first and second connection coefficients represent a vertical minimum and vertical maximum of a respective one of the two first horizontally aligned bounding boxes compared to a sum of intersections between the first bounding box and the respective one of the two first horizontally aligned bounding boxes.

8. The apparatus of claim 1 , wherein the amount of the first horizontally aligned bounding boxes in the word sync list is more than two, the programmable circuitry is to execute the machine readable instructions to:
determine an absolute distance of each of the first horizontally aligned bounding boxes in the word sync list relative to the first bounding box; and
select two of the first horizontally aligned bounding boxes based on the absolute distances to maintain in the word sync list.

9. The apparatus of claim 1 , wherein the amount of the first horizontally aligned bounding boxes in the word sync list is one, the programmable circuitry is to execute the machine readable instructions to:
select the one first horizontally aligned bounding box as a second bounding box;
determine whether the bounding boxes include second horizontally aligned bounding boxes, the second horizontally aligned bounding boxes to include at least one bounding box that is horizontally aligned relative to the second bounding box;
in response to determining the bounding boxes do not include the second horizontally aligned bounding boxes, connect the first bounding box and the second bounding box; and
in response to determining the bounding boxes include the second horizontally aligned bounding boxes, add the second horizontally aligned bounding boxes to the word sync list.

10. The apparatus of claim 1 , wherein, prior to connecting the ones of the horizontally aligned bounding boxes, the programmable circuitry is to execute the machine readable instructions to:
identify third horizontally aligned bounding boxes, the third horizontally aligned bounding boxes to include at least one bounding box of the bounding boxes that is horizontally aligned relative to ones of the bounding boxes in the word sync list; and
add the third horizontally aligned bounding boxes to the word sync list.

11. The apparatus of claim 1 , wherein the programmable circuitry is to execute the instructions to identify a vendor corresponding to the receipt by:
searching the text against a vendor name database; and
in response to identifying a match, associating the vendor with the receipt.

12. The apparatus of claim 1 , wherein the programmable circuitry is to execute the machine readable instructions to apply at least one of a computer vision technique or a deep learning technique to the receipt and the extracted text to identify a region within the receipt, the region to include at least one of a vendor logo, a product description, a product identifier number, a product price, a total price, or a payment detail.

13. The apparatus of claim 12 , wherein the programmable circuitry is to execute the machine readable instructions to classify the identified region.

14. At least one non-transitory machine readable storage medium comprising instructions to cause programmable circuitry to at least:
extract text from an image of a receipt, the text to include bounding boxes;
link ones of the bounding boxes to associate horizontally related fields of the receipt represented in the image by:
selecting a first bounding box;
identifying first horizontally aligned bounding boxes, the first horizontally aligned bounding boxes to include at least one bounding box of the bounding boxes that is horizontally aligned relative to the first bounding box;
adding the first horizontally aligned bounding boxes to a data structure, wherein an amount o

Assignees

• Nielsen Consumer Llc

Inventors

• Shanmuganathan Kannan
• Masthan Hussain
• Soundararajan Padmanabhan
• Yebes Torres Jose Javier
• Allam Raju Kumar

Classifications

• G06F40/131 Primary

  Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces · CPC title

• G06F40/295

  Named entity recognition · CPC title

• G06V30/1478

  of characters or characters lines · CPC title

• G06V30/153

  using recognition of characters or words · CPC title

• G06V30/1912

  Selecting the most significant subset of features (G06V30/19127 takes precedence) · CPC title