System and method for superimposed handwriting recognition technology

Patent metadata

Field	Value
Publication number	US-9911052-B2
Application number	US-201615353214-A
Country	US
Kind code	B2
Filing date	Nov 16, 2016
Priority date	Apr 4, 2014
Publication date	Mar 6, 2018
Grant date	Mar 6, 2018

Abstract

A system and method is able to recognize a user's natural superimposed handwriting without any explicit separation between characters. The system and method is able to process single-stroke and multi-stroke characters. The system and method can also process cursive handwriting. Further, the system and method can determine the boundaries of input words either by the use of a specific user input gesture or by detecting the word boundaries based on language characteristics and properties. The system and method analyzes the handwriting input through the processes of segmentation, character recognition, and language modeling. These three processes occur concurrently through the use of dynamic programming.

First claim

What is claimed is:

1. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for providing handwriting recognition for superimposed stroke input, said method comprising:
creating, with a segmentation expert, a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;
assigning, with a recognition expert, a recognition score to each node of the segmentation graph based on language recognition information;
generating, with a language expert, linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and
providing an output based on the collaborative analysis of the segmentation graph, the recognition score, and the language model by the segmentation, recognition and language experts.

2. A non-transitory computer readable medium according to claim 1 , wherein the input stroke is preprocessed, wherein the preprocessing includes at least the normalization and smoothing of the input stroke.

3. A non-transitory computer readable medium according to claim 2 , wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

4. A non-transitory computer readable medium according to claim 2 , wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

5. A non-transitory computer readable medium according to claim 4 , wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

6. A non-transitory computer readable medium according to claim 4 , wherein the neural networks are multilayer perceptrons.

7. A non-transitory computer readable medium according to claim 1 , wherein the language model includes linguistic information specific to one or more languages.

8. A non-transitory computer readable medium according to claim 1 , wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.

9. A method for providing handwriting recognition for a superimposed input stroke, said method comprising:
creating, with a segmentation expert, a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;
assigning, with a recognition expert, a recognition score to each node of the segmentation graph based on language recognition information;
generating, with a language expert, linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and
providing an output based on the collaborative analysis of the segmentation graph, the recognition score, and the language model by the segmentation, recognition and language experts.

10. A method according to claim 9 , wherein the input stroke is preprocessed, wherein the preprocessing includes at least the normalization and smoothing of the input stroke.

11. A method according to claim 10 , wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

12. A method according to claim 10 , wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

13. A method according to claim 12 , wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

14. A method according to claim 12 , wherein the neural networks are multilayer perceptrons.

15. A method according to claim 9 , wherein the language model includes linguistic information specific to one or more languages.

16. A method according to claim 9 , wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.

17. A system for providing handwriting recognition for a superimposed stroke input to a computing device, the computing device comprising a processor and at least one computer readable program for recognizing the input under control of the processor, said at least one program configured to:
create, with a segmentation expert, a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;
assign, with a recognition expert, a recognition score to each node of the segmentation graph based on language recognition information;
generate, with a language expert, linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and
provide an output based on the collaborative analysis of the segmentation graph, the recognition score, and the language model by the segmentation, recognition and language experts.

18. A system according to claim 17 , wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

19. A system according to claim 17 , wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

20. A system according to claim 19 , wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

21. A system according to claim 19 , wherein the neural networks are multilayer perceptrons.

22. A system according to claim 17 , wherein the language model includes linguistic information specific to one or more languages.

23. A system according to claim 17 , wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.

Assignees

• Myscript

Inventors

• Wimmer Zsolt
• Perraud Freddy
• Lallican Pierre Michel

Classifications

• G06T7/10

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06T7/13

  Edge detection · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V30/2268 Primary

  using stroke segmentation · CPC title