Credit card auto-fill

Patent metadata

Field	Value
Publication number	US-9449239-B2
Application number	US-201414292763-A
Country	US
Kind code	B2
Filing date	May 30, 2014
Priority date	May 30, 2014
Publication date	Sep 20, 2016
Grant date	Sep 20, 2016

Abstract

Differing embodiments of this disclosure may employ one or all of the several techniques described herein to perform credit card recognition using electronic devices with integrated cameras. According to some embodiments, the credit card recognition process may comprise: obtaining a first representation of a first image, wherein the first representation comprises a first plurality of pixels; identifying a first credit card region within the first representation; extracting a first plurality of sub-regions from within the identified first credit card region, wherein a first sub-region comprises a credit card number, wherein a second sub-region comprises an expiration date, and wherein a third sub-region comprises a card holder name; generating a predicted character sequence for the first, second, and third sub-regions; and validating the predicted character sequences for at least the first, second, and third sub-regions using various credit card-related heuristics, e.g., expected character sequence length, expected character sequence format, and checksums.

First claim

The invention claimed is:

1. A non-transitory program storage device, readable by a programmable control device and comprising instructions stored thereon to cause one or more processing units to:
obtain a first representation of a first image, wherein the first representation comprises a first plurality of pixels;
identify a first credit card region within the first representation;
extract a plurality of sub-regions from within the identified first credit card region, wherein a first sub-region of the plurality of sub-regions comprises a credit card number, wherein a second sub-region of the plurality of sub-regions comprises an expiration date, and wherein a third sub-region of the plurality of sub-regions comprises a card holder name;
generate a plurality of cuts at a plurality of locations associated with one of the sub-regions when extracting the plurality of sub-regions, wherein a first cut of the plurality of cuts has a different vertical offset and/or a different horizontal offset than another of the plurality of cuts;
generate a predicted character sequence for the first, second, and third sub-regions; and
determine a sequence length of the predicted character sequence for the first sub-region and evaluate a checksum on the predicted character sequence for the first sub-region;
determine a sequence length of the predicted character sequence for the second sub-region and evaluate a format of the predicted character sequence for the second sub-region; and
compare the predicted character sequence for the third sub-region with a plurality of valid character sequences.

2. The non-transitory program storage device of claim 1 , wherein the instructions to identify a first credit card region within the first representation further comprise instructions to execute a rectangle detector on the first representation of the first image.

3. The non-transitory program storage device of claim 2 , wherein the rectangle detector comprises at least one of: aspect-ratio constraints and size constraints.

4. The non-transitory program storage device of claim 1 , wherein the instructions to determine a sequence length of the predicted character sequence for the first sub-region further comprise instructions to determine a sequence length of fifteen or sixteen characters for the predicted character sequence for the first sub-region.

5. The non-transitory program storage device of claim 1 , wherein at least one valid character sequence of the predicted character sequence for the first sub-region comprises at least one of the following: a name from an address book; and a character sequence from a language model.

6. The non-transitory program storage device of claim 1 , wherein the one of the sub-region is the second sub-region.

7. A system, comprising:
a memory having, stored therein, computer program code;
a digital camera; and
one or more processing units operatively coupled to the digital camera and memory and configured to execute instructions in the computer program code that cause the one or more processing units to:
obtain a first representation of a first image from the digital camera, wherein the first representation comprises a first plurality of pixels;
identify a first credit card region within the first representation;
extract a plurality of sub-regions from within the identified first credit card region, wherein a first sub-region of the plurality of sub-regions comprises a credit card number, wherein a second sub-region of the plurality of sub-regions comprises an expiration date, and wherein a third sub-region of the plurality of sub-regions comprises a card holder name;
generate a plurality of cuts at a set of probable locations associated with one of the sub-regions in order to extract one of the sub-regions, wherein at least some of the plurality of cuts have at least one different vertical offset, horizontal offset, or both than at least some of the plurality of cuts;
generate a predicted character sequence for the first, second, and third sub-regions; and
determine a sequence length of the predicted character sequence for the first sub-region and evaluate a checksum on the predicted character sequence for the first sub-region;
determine a sequence length of the predicted character sequence for the second sub-region and evaluate a format of the predicted character sequence for the second sub-region; and
compare the predicted character sequence for the third sub-region with a plurality of valid character sequences.

8. The system of claim 7 , wherein the instructions to identify a first credit card region within the first representation further comprise instructions to execute a rectangle detector on the first representation of the first image.

9. The system of claim 8 , wherein the rectangle detector comprises at least one of: aspect-ratio constraints and size constraints.

10. The system of claim 7 , wherein the instructions to determine a sequence length of the predicted character sequence for the first sub-region further comprise instructions to determine a sequence length of fifteen or sixteen characters for the predicted character sequence for the first sub-region.

11. The system of claim 7 , wherein least one valid character sequence of the predicted character sequence for the first sub-region comprises at least one of the following: a name from an address book; and a character sequence from a language model.

12. The system of claim 7 , wherein the one of the sub-region is the second sub-region.

13. A computer-implemented method, comprising:
obtaining a first representation of a first image from a first digital camera, wherein the first representation comprises a first plurality of pixels;
identifying, using a computer, a first credit card region within the first representation;
extracting, using the computer, a plurality of sub-regions from within the identified first credit card region, wherein a first sub-region comprises a credit card number, wherein a second sub-region comprises an expiration date, and wherein a third sub-region comprises a card holder name;
generating, using the computer, a plurality of cuts to extract one of the following regions: the first sub-region of the plurality of sub-regions, the second sub-region of the plurality of sub-regions, or the third sub-region of the plurality of sub-regions, wherein a first cut of the plurality of cuts has a different vertical offset, a horizontal offset, or combination thereof than a second cut of the plurality of cuts;
generating, using the computer, a predicted character sequence for the first, second, and third sub-regions; and
determining, using the computer, a sequence length of the predicted character sequence for the first sub-region and evaluating a checksum on the predicted character sequence for the first sub-region;
determining, using the computer, a sequence length of the predicted character sequence for the second sub-region and evaluating a format of the predicted character sequence for the second sub-region; and
comparing, using the computer, the predicted character sequence for the third sub-region with a plurality of valid character sequences.

14. The computer-implemented method of claim 13 , wherein the act of identifying a first credit card region within the first representation further comprises executing a rectangle detector on the first representation of the first image.

15. The computer-implemented method of claim 14 , wherein the rectangle detector comprises at least one of: aspect-ratio constraints and size constraints.

16. The computer-implemented method of claim 13 , wherein at

Assignees

• Apple Inc

Inventors

• Dixon Ryan S
• Meier Ueli
• Groethe Karl M
• Bellegarda Jerome R
• Pagallo Giulia M

Classifications

• G06V30/2253

  Recognition of characters printed with magnetic ink (G06V30/2247 takes precedence) · CPC title

• G06V30/142 Primary

  using hand-held instruments; Constructional details of the instruments · CPC title

• G06V10/768

  using context analysis, e.g. recognition aided by known co-occurring patterns · CPC title

• G06K9/228

  Physics · mapped topic

• G06K9/72

  Physics · mapped topic