System and method for superimposed handwriting recognition technology

Patent metadata

Field	Value
Publication number	US-10007859-B2
Application number	US-201615083199-A
Country	US
Kind code	B2
Filing date	Mar 28, 2016
Priority date	Apr 4, 2014
Publication date	Jun 26, 2018
Grant date	Jun 26, 2018

Abstract

A system and method that is able to recognize a user's natural superimposed handwriting without any explicit separation between characters. The system and method is able to process single-stroke and multi-stroke characters. It can also process cursive handwriting. Further, the method and system can determine the boundaries of input words either by the use of a specific user input gesture or by detecting the word boundaries based on language characteristics and properties. The system and method analyzes the handwriting input through the processes of fragmentation, segmentation, character recognition, and language modeling. At least some of these processes occur concurrently through the use of dynamic programming.

First claim

What is claimed is:

1. A system for providing handwriting recognition for a plurality of at least partially superimposed fragments of input strokes on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium under control of the processor, the at least one non-transitory computer readable medium configured to:
determine the time order of input of at least sequential fragments;
detect the geometry of the input strokes in the at least sequential fragments;
detect the relative positions of the input strokes of the at least sequential fragments;
determine from the determined time order and detected relative positions and geometries whether one or more of the input strokes of the at least sequential fragments combine to form one or more likely characters;
classify the fragments based on the determined likely characters; and
provide the classified fragments to a recognition engine for evaluation of character hypotheses based on the classified fragments,
wherein, within the recognition engine, a method comprises:
creating a segmentation graph based on the strokes of the classified fragments, wherein the segmentation graph includes nodes corresponding to character hypotheses;
assigning a recognition score to each node of the segmentation graph based on a pattern classifier; and
generating linguistic meaning of the input strokes based on the recognition scores and a language model.

2. A system according to claim 1 , wherein the at least one non-transitory computer readable medium is configured to determine the likely characters by determining whether at least a segment of at least one input stroke of a first fragment of the at least sequential fragments combines with at least a segment of at least one input stroke of a second fragment of the at least sequential fragments to likely form at least one character.

3. A system according to claim 1 , wherein each classified fragment is defined to contain complete characters formed by the input strokes of one or more input fragments.

4. A system according to claim 1 , wherein the relative positions of the input strokes of the at least sequential fragments are detected from both spatial and temporal information of the input strokes.

5. A system according to claim 1 , wherein, within the recognition engine, the method comprises:
providing an output based on the simultaneous analysis of the segmentation graph, the recognition score, and the language model.

6. A system according to claim 5 , wherein the segmentation graph further includes nodes corresponding to space hypotheses between the character hypotheses based on the classified fragments.

7. A method for providing handwriting recognition for a plurality of at least partially superimposed fragments of input strokes on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for recognizing the handwriting under control of the processor, the method comprising:
determining the time order of input of at least sequential fragments;
detecting the geometry of the input strokes in the at least sequential fragments;
detecting the relative positions of the input strokes of the at least sequential fragments;
determining from the determined time order and detected relative positions and geometries whether one or more of the input strokes of the at least sequential fragments combine to form one or more likely characters;
classifying the fragments based on the determined likely characters;
providing the classified fragments to a recognition engine for evaluation of character hypotheses based on the classified fragments;
creating a segmentation graph based on the strokes of the classified fragments, wherein the segmentation graph includes nodes corresponding to character hypotheses;
assigning a recognition score to each node of the segmentation graph based on a pattern classifier; and
generating linguistic meaning of the input strokes based on the recognition scores and a language model.

8. A method according to claim 7 , wherein the likely characters are determined by determining whether at least a segment of at least one input stroke of a first fragment of the at least sequential fragments combines with at least a segment of at least one input stroke of a second fragment of the at least sequential fragments to likely form at least one character.

9. A method according to claim 7 , wherein each classified fragment is defined to contain complete characters formed by the input strokes of one or more input fragments.

10. A method according to claim 7 , wherein the relative positions of the input strokes of the at least sequential fragments are detected from both spatial and temporal information of the input strokes.

11. A method according to claim 7 , wherein, within the recognition engine, the method comprises:
providing an output based on the simultaneous analysis of the segmentation graph, the recognition score, and the language model.

12. A method according to claim 11 , wherein the segmentation graph further includes nodes corresponding to space hypotheses between the character hypotheses based on the classified fragments.

13. A non-transitory computer readable medium having computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for a plurality of at least partially superimposed fragments of input strokes on a computing device, the computing device comprising a processor, the non-transitory computer readable medium under control of the processor implementing the method comprising:
determining the time order of input of at least sequential fragments;
detecting the geometry of the input strokes in the at least sequential fragments;
detecting the relative positions of the input strokes of the at least sequential fragments;
determining from the determined time order and detected relative positions and geometries whether one or more of the input strokes of the at least sequential fragments combine to form one or more likely characters;
classifying the fragments based on the determined likely characters;
providing the classified fragments to a recognition engine for evaluation of character hypotheses based on the classified fragments;
creating a segmentation graph based on the strokes of the classified fragments, wherein the segmentation graph includes nodes corresponding to character hypotheses;
assigning a recognition score to each node of the segmentation graph based on a pattern classifier; and
generating linguistic meaning of the input strokes based on the recognition scores and a language model.

14. A non-transitory computer readable medium according to claim 13 , wherein the likely characters are determined by determining whether at least a segment of at least one input stroke of a first fragment of the at least sequential fragments combines with at least a segment of at least one input stroke of a second fragment of the at least sequential fragments to likely form at least one character.

15. A non-transitory computer readable medium according to claim 13 , wherein each classified fragment is defined to contain complete characters formed by the input strokes of one or more input fragments.

16. A non-transitory computer readable medium according to claim 13 , wherein the relative positions of the input strokes of the at least sequential fragments are detected from both spatial and temporal information of the input strokes.

17. A non-transitory computer readable medium acco

Assignees

• Myscript

Inventors

• Wimmer Zsolt
• Perraud Freddy
• Lallican Pierre-Michel
• Aradilla Guillermo

Classifications

• G06V30/32 Primary

  Digital ink · CPC title

• G06V30/1423

  the instrument generating sequences of position coordinates corresponding to handwriting (preprocessing or recognising digital ink G06V30/32) · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V30/268

  Lexical context · CPC title

• G06V30/2272

  with lexical matching · CPC title