System and method of guiding handwriting diagram input

Patent metadata

Field	Value
Publication number	US-10976918-B2
Application number	US-201514955198-A
Country	US
Kind code	B2
Filing date	Dec 1, 2015
Priority date	Oct 19, 2015
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

A system, method and computer program product for guiding hand-drawing of diagrams including text and non-text elements on a computing device are provided. The computing device has a processor and a non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor. Display is performed, on an interactive display of the computing device, of a guide element associated with at least one diagram element of displayed handwriting diagram input. The guide element is configured with a depiction of the at least one diagram element in recognized form.

First claim

We claim:

1. A system for guiding hand-drawing of diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the at least one non-transitory computer readable medium configured to:
accept input of, on an input area of an interactive display of the computing device:
a non-text hand-drawing diagram element, and
a selection gesture; and

cause display of, on the interactive display:
a digital ink of the non-text hand-drawing diagram element,
a recognized form of the non-text hand-drawing diagram element overlaid on the digital ink of the non-text hand-drawing diagram element, wherein the recognized form of the non-text hand-drawing diagram element includes one or more geometric characteristics of the digital ink of the non-text hand-drawing diagram element and permits modification of one or more of the geometric characteristics of the recognized form of the non-text hand-drawing diagram element, and
a dynamic prompter which dynamically displays at least one depiction of the non-text hand-drawing diagram element, the dynamic prompter configured with the recognized form of the non-text hand-drawing diagram element and geometrically similar variants of the non-text hand-drawing diagram element;

wherein the computer readable medium is configured to cause display of the dynamic prompter in response to the selection gesture selecting the digital ink of the non-text hand-drawing diagram element; and
wherein the recognized form of the non-text hand-drawing diagram element is initially determined based on a probability score calculated from at least one of: a spatial score, a temporal score, and an input characteristic of the non-text hand-drawing diagram element.

2. A system as claimed in claim 1 , wherein the dynamic prompter dynamically displays at least one depiction of a handwriting text diagram element as handwriting is input.

3. A system as claimed in claim 1 , wherein the dynamic prompter includes at least one interactive prompter element, the at least one non-transitory computer readable medium configured to display an action menu in response to receiving interaction with the at least one interactive prompter element.

4. A system as claimed in claim 1 , the at least one non-transitory computer readable medium configured to display the depiction of a non-text element in icon form.

5. A system as claimed in claim 4 , wherein the non-text element is a shape element of the diagram, the icon depicting the shape.

6. A system as claimed in claim 1 , the at least one non-transitory computer readable medium configured to display the depiction of a text element in typeset ink.

7. A method for guiding hand-drawing of diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:
inputting, on an input area of an interactive display of the computing device:
a non-text hand-drawing diagram element, and
a selection gesture; and

displaying, on the interactive display:
a recognized form of the non-text hand-drawing diagram element overlaid on the digital ink of the non-text hand-drawing diagram element, wherein the recognized form of the non-text hand-drawing diagram element includes one or more geometric characteristics of the digital ink of the non-text hand-drawing diagram element and permits modification of one or more of the geometric characteristics of the recognized form of the non-text hand-drawing diagram element, and
a dynamic prompter which dynamically displays at least one depiction of the non-text hand-drawing diagram element, the dynamic prompter configured with the recognized form of the non-text hand-drawing diagram element and geometrically similar variants of the non-text hand-drawing diagram element;

wherein the dynamic prompter is displayed in response to selecting, via the selection gesture, the digital ink of the non-text hand-drawing diagram element; and
wherein the recognized form of the non-text hand-drawing diagram element is initially determined based on a probability score calculated from at least one of: a spatial score, a temporal score, and an input characteristic of the non-text hand-drawing diagram element.

8. A method as claimed in claim 7 , wherein the dynamic prompter dynamically displays at least one depiction of a handwriting text diagram element as the handwriting is input.

9. A method as claimed in claim 7 , wherein the dynamic prompter includes at least one interactive prompter element, the at least one non-transitory computer readable medium configured to display an action menu in response to receiving interaction with the at least one interactive prompter element.

10. A method as claimed in claim 7 , comprising displaying the depiction of a non-text element in icon form.

11. A method as claimed in claim 10 , wherein the non-text element is a shape element of the diagram, the icon depicting the shape.

12. A method as claimed in claim 7 , comprising displaying the depiction of a text element in typeset ink.

13. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for guiding hand-drawing of diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:
inputting, on an input area of an interactive display of the computing device:
a non-text hand-drawing diagram element, and
a selection gesture; and

displaying, on the interactive display:
a recognized form of the non-text hand-drawing diagram element overlaid on the digital ink of the non-text hand-drawing diagram element, wherein the recognized form of the non-text hand-drawing diagram element includes one or more geometric characteristics of the digital ink of the non-text hand-drawing diagram element and permits modification of one or more of the geometric characteristics of the recognized form of the non-text hand-drawing diagram element, and
a dynamic prompter which dynamically displays at least one depiction of the non-text hand-drawing diagram element, the dynamic prompter configured with the recognized form of the non-text hand-drawing diagram element and geometrically similar variants of the non-text hand-drawing diagram element;

wherein the dynamic prompter is displayed in response to selecting, via the selection gesture, the digital ink of the non-text hand-drawing diagram element; and
wherein the recognized form of the non-text hand-drawing diagram element is initially determined based on a probability score calculated from at least one of: a spatial score, a temporal score, and an input characteristic of the non-text hand-drawing diagram element.

14. A non-transitory computer readable medium as claimed in claim 13 , wherein the dynamic prompter dynamically displays at least one diagram element as the handwriting is input.

15. A non-transitory computer readable medium as claimed in claim 13 , wherein the dynamic prompter includes at least one interactive prompter element, the at least one non-transitory computer readable medium config

Assignees

• Myscript

Inventors

• Mélinand Robin
• Bednarowicz Romain
• SIDOLI Claire
• RIC Fabien
• Rucine Nicolas
• Jestin Erwan
• LAURENCE Anthony
• Elagouni Khaoula
• CEROVIC Cyril
• Bonnaud Anne

Classifications

• G06V30/387

  using human interaction, e.g. selection of the best displayed recognition candidate · CPC title

• G06V30/32

  Digital ink · CPC title

• G06V30/10

  Character recognition · CPC title

• G06F40/166

  Editing, e.g. inserting or deleting · CPC title

• G06F3/04817

  using icons (graphical or visual programming using iconic symbols G06F8/34) · CPC title