Enhanced interpretation of character arrangements

Patent metadata

Field	Value
Publication number	US-9524429-B2
Application number	US-201414465307-A
Country	US
Kind code	B2
Filing date	Aug 21, 2014
Priority date	Aug 21, 2014
Publication date	Dec 20, 2016
Grant date	Dec 20, 2016

Abstract

Technologies are described herein for interpreting character arrangements. An image including an arrangement of characters may be received or captured by a computing device. Techniques described herein generate data representative of the characters. Characteristics and other information interpreted from the image may be processed to determine a data type. The data representative of the characters may be arranged into a data structure based on the data type, an arrangement type and/or other information interpreted from the image. The data type may indicate one or more attributes of the arranged data such as a format, font, date, language, or currency. The data type may also indicate how data is used in a process, equation or calculation. In addition, the data type may identify an anchor that may be used to merge data generated from the image with other data generated from another image.

First claim

What is claimed is:

1. A computer-implemented method comprising computer-implemented operations for:
obtaining an image comprising an arrangement of a plurality of characters;
generating data representing individual characters of the plurality of characters;
determining an arrangement type based on the arrangement of the plurality of characters;
determining a data type of at least one character of the plurality of characters;
arranging the data representing the individual characters of the plurality of characters in a data structure based on the arrangement type and the data type of the at least one character of the plurality of characters;
displaying the data structure on an interface configured to receive an input gesture to modify the data structure, wherein the displaying of the data structure is performed simultaneously as a camera of a computing device is obtaining the image;
receiving the input gesture to modify the data structure; and
modifying the data representing the individual characters of the plurality of characters based on the input gesture.

2. The computer-implemented method of claim 1 , further comprising computer-implemented operations for:
obtaining a second image comprising a second arrangement of a second plurality of characters;
generating other data representing individual characters of the second plurality of characters;
determining a second data type of at least one character of the second plurality of characters; and
arranging the other data representing the individual characters of the second plurality of characters in the data structure based on the second arrangement of the second plurality of characters and the second data type.

3. The computer-implemented method of claim 2 , wherein the data type and the second data type identify an anchor, and wherein the data representing the individual characters of the plurality of characters and the other data representing the individual characters of the second plurality of characters are merged in the data structure based on the anchor.

4. The computer-implemented method of claim 1 , wherein the determining the data type of the at least one character of the plurality of characters is based on a format of the at least one character of the plurality of characters, a font of the at least one character of the plurality of characters, or at least one text pattern formed from the plurality of characters.

5. The computer-implemented method of claim 1 , wherein the plurality of characters comprises the at least one character and other characters, wherein the data type indicates that data representing the at least one character is a header, and wherein the arranging the data in the data structure comprises distinguishing a display of the data representing the at least one character from a display of data representing the other characters of the plurality of characters.

6. The computer-implemented method of claim 1 , wherein the plurality of characters comprises the at least one character and other characters, and wherein the computer-implemented method further comprises computer-implemented operations for:
identifying a second data type of the other characters, wherein the second data type of the other characters is incompatible with the data type of the at least one character of the plurality of characters; and
providing a graphical distinction between a display of data representing the at least one character from a display of other data representing the other characters.

7. The computer-implemented method of claim 6 , further comprising computer-implemented operations for generating at least one equation utilizing a data set, the data set consisting of the other data representing the other characters of the plurality of characters.

8. A computer, comprising:
a processor; and
a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the computer to:
obtain an image comprising an arrangement of a plurality of characters,
generate data representing individual characters of the plurality of characters,
determine a data type of at least one character of the plurality of characters, wherein the data type is based on the at least one character, the arrangement of the plurality of characters, or contextual data interpreted from the image,
arrange the data representing the individual characters of the plurality of characters in a data structure based on the data type of the at least one character of the plurality of characters,
display the data structure on an interface configured to receive an input gesture to modify the data structure, wherein the display of the data structure is performed simultaneously as a camera of the computer is capturing the image,
receive the input gesture to modify the data structure, and
modify the data representing individual characters of the plurality of characters based on the input gesture.

9. The computer of claim 8 , wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the computer to:
obtain a second image comprising a second arrangement of a second plurality of characters;
generate other data representing individual characters of the second plurality of characters;
determine a second data type of at least one character of the second plurality of characters; and
arrange the other data representing the individual characters of the second plurality of characters in the data structure based on the second arrangement of the second plurality of characters and the second data type.

10. The computer of claim 9 , wherein the data type and the second data type identify an anchor, and wherein the data representing the individual characters of the plurality of characters and the other data representing the individual characters of the second plurality of characters are merged in the data structure based on the anchor.

11. The computer of claim 8 , wherein the determining the data type of the at least one character of the plurality of characters is based on a format of the at least one character of the plurality of characters or at least one text pattern formed from the plurality of characters.

12. The computer of claim 8 , wherein the data type indicates that data representing the at least one character is a header, and wherein the arranging the data representing the individual characters of the plurality of characters in the data structure comprises distinguishing a display of the data representing the at least one character from a display of data representing other characters of the plurality of characters.

13. The computer of claim 8 , wherein the plurality of characters comprises the at least one character and other characters, and wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the computer to:
identify a second data type of the other characters, wherein the second data type is incompatible with the data type; and
provide a graphical distinction between a display of data representing the at least one character and a display of other data representing the other characters.

14. The computer of claim 13 , wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the processor to generate at least one equation utilizing a data set, the data set consisting of the other data represen

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Zhang Yi
• Parish Daniel
• Wilson Aaron Lamar
• Fang Shangwei

Classifications

• G06V30/245

  Font recognition · CPC title

• G06V30/10

  Character recognition · CPC title

• G06K9/18

  Physics · mapped topic

• G06K9/00442 Primary

  Physics · mapped topic

• G06K9/6828

  Physics · mapped topic