Interactive technique for using a user-provided image of a document to collect information

Patent metadata

Field	Value
Publication number	US-10769554-B2
Application number	US-201113195298-A
Country	US
Kind code	B2
Filing date	Aug 1, 2011
Priority date	Aug 1, 2011
Publication date	Sep 8, 2020
Grant date	Sep 8, 2020

Abstract

In a collection technique, a user (such as a taxpayer) provides information (such as income-tax information) by submitting an image of a document, such as an income-tax summary or form. In particular, the user may provide a description of the document. In response, the user is prompted for the information associated with the field in the document. Then, the user provides the image of a region in the document that includes the field. Based on the image, the information is extracted, and the field in the form is populated using the extracted information. The prompting, receiving, extracting and populating operations may be repeated for one or more additional fields in the document.

First claim

What is claimed is:

1. A method for populating a field in a form performed by a computer, comprising:
receiving a fillable form based on a description of a physical document, wherein the physical document comprises a set of fields, and wherein the fillable form comprises a fillable field for each field in the set of fields;
sending a template of the set of fields to be displayed to a user on a portable electronic device that is remotely located from the computer;
sending a unique prompt, for each field in the set of fields, to be displayed to the user at the portable electronic device, wherein each unique prompt displayed to the user requests the user to capture an image, based on the displayed template, of a region of the physical document that includes the field and data that appears in the field corresponding to that unique prompt;
receiving, for each of the one or more fields, the image of the region of the physical document that includes the field and data that appears in the field;
extracting, for each received image, the data that appears in the field from the image of the region; and
populating the extracted data into the fillable field in the fillable form that corresponds to the field from the image of the region of the physical document.

2. The method of claim 1 , wherein the physical document includes an income-tax form.

3. The method of claim 1 , wherein the physical document includes a form having a format specified by a government agency.

4. The method of claim 1 , wherein the image includes a digital photograph.

5. The method of claim 1 , wherein the image includes multiple fields; and
wherein the extracting involves:
capturing a set of information associated with the multiple fields;
identifying the field based on a description proximate to the field; and
selecting a subset of the set of information that includes the data.

6. The method of claim 5 , wherein, if the computer is unable to identify the field, the method further includes prompting the user to identify the field from the multiple fields.

7. The method of claim 1 , wherein the image includes a real-time video stream provided by the portable electronic device.

8. The method of claim 1 , wherein extracting the data involves performing optical character recognition using the image of the region.

9. The method of claim 8 , wherein the optical character recognition includes a radial image analysis technique that identifies a boundary of the field in the region.

10. The method of claim 1 ,
further comprising using the image of the region to determine a next field in the set of fields to prompt the user to take an image of, wherein using the image of the region comprises determining that the image of the region does not include an image of the next field;
wherein prompting the user comprises requesting that the user provide an image of a second region in the document that includes the next field;
wherein the method further comprises repeating the operations until images that include all the fields in the set of fields have been received from the user;
wherein receiving the image comprises receiving a different image for the field than images received for other fields in the set of fields, and wherein the region is smaller than a page of the document.

11. The method of claim 1 , wherein, for each field in the set of fields, the physical document comprises a descriptor for the field and a monetary value for the field, and wherein the method is performed without the user entering any of the descriptors or any of the monetary values for any of the fields.

12. A computer-program product for use in conjunction with an electronic device, the computer-program product comprising a non-transitory computer-readable storage medium and a computer-program mechanism embedded therein to populate a field in a form, the computer-program mechanism including instructions for:
receiving, at a computer, a fillable form based on a description of a physical document, wherein the physical document comprises a set of fields, and wherein the fillable form comprises a fillable field for each field in the set of fields;
sending a template of the set of fields to be displayed to a user on a portable electronic device that is remotely located from the computer;
sending a unique prompt, for each field in the set of fields, to be displayed to the user at the portable electronic device, wherein each unique prompt displayed to the user requests the user to capture an image, based on the displayed template, of a region of the physical document that includes the field and data that appears in the field corresponding to that unique prompt;
receiving, from the portable electronic device, for each of the one or more fields, the image of the region of the physical document that includes the field and data that appears in the field;
extracting, for each received image, the data that appears in the field from the image of the region; and
populating the extracted data into the fillable field in the fillable form that corresponds to the field from the image of the region of the physical document.

13. The computer-program product of claim 12 , wherein the physical document includes a form having a format specified by a government agency.

14. The computer-program product of claim 12 , wherein the image includes multiple fields; and
wherein the instructions for extracting involve:
capturing a set of information associated with the multiple fields;
identifying the field based on a description proximate to the field; and
selecting a subset of the set of information that includes the data.

15. An electronic device, comprising:
a processor;
memory; and
a program module, wherein the program module is stored in the memory and configurable to be executed by the processor to populate a field in a form, the program module including instructions for:
receiving, at a computer, a fillable form based on a description of a physical document, wherein the physical document comprises a set of fields, and wherein the fillable form comprises a fillable field for each field in the set of fields;
sending a template of the set of fields to be displayed to a user on a portable electronic device that is remotely located from the computer;
sending a unique prompt, for each field in the set of fields, to be displayed to the user at the portable electronic device, wherein each unique prompt displayed to the user requests the user to capture an image, based on the displayed template, of a region of the physical document that includes the field and data that appears in the field corresponding to that unique prompt;
receiving, from the portable electronic device, for each of the one or more fields, the image of the region of the physical document that includes the field and data that appears in the field;
extracting, for each received image, the data that appears in the field from the image of the region; and
populating the extracted data into the fillable field in the fillable form that corresponds to the field from the image of the region of the physical document.

16. The method of claim 15 , wherein the computer comprises a desktop or laptop computer, wherein the portable electronic device comprises a mobile phone with a camera, and wherein the user captures the image of the region by using the camera.

Assignees

• Eftekhari Amir
• Tifford Alan
• Intuit Inc

Inventors

• Eftekhari Amir
• Tifford Alan

Classifications

• G06F40/186

  Templates · CPC title

• G06Q10/00 Primary

  Administration; Management · CPC title

• G06Q40/00

  Finance; Insurance; Tax strategies; Processing of corporate or income taxes · CPC title

• G06F40/174

  Form filling; Merging · CPC title