System and method of guiding handwriting diagram input

Patent metadata

Field	Value
Publication number	US-11740783-B2
Application number	US-202117205246-A
Country	US
Kind code	B2
Filing date	Mar 18, 2021
Priority date	Oct 19, 2015
Publication date	Aug 29, 2023
Grant date	Aug 29, 2023

Abstract

A system, method and computer program product for guiding hand-drawing of diagrams including text and non-text elements on a computing device are provided. The computing device has a processor and a non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor. Display is performed, on an interactive display of the computing device, of a guide element associated with at least one diagram element of displayed handwriting diagram input. The guide element is configured with a depiction of the at least one diagram element in recognized form.

First claim

We claim:

1. A system for guiding hand-drawing of diagrams including text and non-text element types on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium storing instructions that, when executed by the processor, cause the processor to:
accept, on an input area of an interactive display of the computing device, input of a hand-drawing diagram element;
cause classification of the hand-drawing diagram element as a first type element;
cause recognition of the first type hand-drawing diagram element based on the classification;
cause display of, on the interactive display, a digital ink of the hand-drawing diagram element;
accept, on the input area of the interactive display, a selection gesture causing selection of the digital ink of the hand-drawing diagram element;
in response to accepting the selection gesture, cause display of, on the interactive display, i) dynamic prompter which dynamically depicts at least one recognized form of the hand-drawing diagram element, and ii) a manual classification action;
cause selection of the manual classification action in response to receiving interaction with the dynamic prompter;
cause re-classification of the hand-drawing diagram element as being a second type element, in response to said selection; and
cause re-recognition of the hand-drawing diagram element in response to said re-classification,
wherein the dynamic prompter is configured to cause depiction of a recognized form of the re-recognized second type hand-drawing diagram element.

2. The system as claimed in claim 1 , wherein the dynamic prompter dynamically depicts the at least one recognized form of the hand-drawing diagram element as handwriting is input.

3. The system as claimed in claim 1 , wherein the first type element is non-text and the dynamic prompter depicts the at least one recognized form of the hand-drawing non-text diagram element in icon form.

4. The system as claimed in claim 3 , wherein the hand-drawing diagram element is a shape element of the diagram, the icon depicting the shape.

5. The system as claimed in claim 1 , wherein the first type element is text and the dynamic prompter depicts the at least one recognized form of the hand-drawing text diagram element in typeset ink.

6. The system as claimed in claim 1 , wherein
the hand-drawing diagram element is classified as non-text;
the selection of the manual classification action causes re-classification of the non-text element as text;
the re-classification causes re-recognition of the re-classified text element; and
the dynamic prompter causes depiction of a recognized form of the re-recognized text element.

7. The system as claimed in claim 1 , wherein
the hand-drawing diagram element is classified as text;
the selection of the manual classification action causes re-classification of the text element as non-text;
the re-classification causes re-recognition of the re-classified non-text element; and
the dynamic prompter causes depiction of a recognized form of the re-recognized non-text element.

8. A method for guiding hand-drawing of diagrams including text and non-text element types on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:
accepting, on an input area of an interactive display of the computing device, input of a hand-drawing diagram element;
classifying the hand-drawing diagram element as a first type element;
recognizing the first type hand-drawing diagram element based on the classification;
displaying, on the interactive display, a digital ink of the hand-drawing diagram element;
accepting, on the input area of the interactive display, a selection gesture causing selection of the digital ink of the hand-drawing diagram element;
in response to accepting the selection gesture, displaying, on the interactive display, i) a dynamic prompter which dynamically depicts at least one recognized form of the hand-drawing diagram element, and ii) a manual classification action;
selecting the manual classification action in response to receiving interaction with the dynamic prompter;
re-classifying the hand-drawing diagram element as being a second type element, in response to said selection; and
re-recognizing of the hand-drawing diagram element in response to re-classification,
wherein the dynamic prompter is configured to cause depiction of a recognized form of the re-recognized hand-drawing diagram element.

9. The method as claimed in claim 8 , wherein the dynamic prompter dynamically depicts the at least one recognized form of the hand-drawing diagram element as the handwriting is input.

10. The method as claimed in claim 8 , comprising displaying a recognized form of the hand-drawing diagram element overlaid on the digital ink of the diagram element wherein the first type element is non-text.

11. The method as claimed in claim 8 , comprising depicting the at least one recognized form of the hand-drawing diagram element in icon form wherein the first type element is non-text.

12. The method as claimed in claim 11 , wherein the hand-drawing diagram element is a shape element of the diagram, the icon depicting the shape.

13. The method as claimed in claim 8 , comprising depicting of the at least one recognized form of the hand-drawing diagram element in typeset ink wherein the first type element is text.

14. The method as claimed in claim 8 , comprising
classifying the hand-drawing diagram element as non-text;
re-classifying the non-text element as text following the selection of the manual classification action;
re-recognizing the re-classified text element; and
depicting a recognized form of the re-recognized text element by the dynamic prompter.

15. The method as claimed in claim 8 , comprising
classifying the hand-drawing diagram element as text;
re-classifying the text element as non-text following the selection of the manual classification action;
re-recognizing the re-classified non-text element; and
depicting a recognized form of the re-recognized non-text element by the dynamic prompter causes.

16. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for guiding hand-drawing of diagrams including text and non-text element types on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:
accepting, on an input area of an interactive display of the computing device, a hand-drawing diagram element;
classifying the hand-drawing diagram element as either text or non-text;
recognizing the hand-drawing diagram element based on the classification;
displaying, on the interactive display, a digital ink of the hand-drawing diagram element;
accepting, on the input area of the interactive display, a selection gesture causing selection of the digital ink of the hand-drawing diagram element;
in response to accepting the selection gesture, displaying, on the interactive display, i) a dynamic prompter which dynamically depicts at least one recognized form of the hand-drawing diagram element, and ii) a manual classification action;
selecting the manual classification action in response to receiving interact

Assignees

• Myscript

Inventors

• Mélinand Robin
• Bednarowicz Romain
• SIDOLI Claire
• RIC Fabien
• Rucine Nicolas
• Jestin Erwan
• LAURENCE Anthony
• Elagouni Khaoula
• CEROVIC Cyril
• Bonnaud Anne

Classifications

• G06F3/04883 Primary

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/04817

  using icons (graphical or visual programming using iconic symbols G06F8/34) · CPC title

• G06F40/166

  Editing, e.g. inserting or deleting · CPC title

• G06F40/171

  by use of digital ink · CPC title