System and method for recognition of handwritten diagram connectors

Patent metadata

Field	Value
Publication number	US-10417491-B2
Application number	US-201514955174-A
Country	US
Kind code	B2
Filing date	Dec 1, 2015
Priority date	Oct 19, 2015
Publication date	Sep 17, 2019
Grant date	Sep 17, 2019

Abstract

A system, method and computer program product for hand-drawing diagrams including text and non-text elements on a computing device are provided. The computing device has a processor and a non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor. Display of a plurality of input diagram elements in interactive ink is performed on a display device associated with the computing device. At least one diagram element is identified as a connector which connects a plurality of diagram elements. Geometrical characteristics of the at least identified connector are determined and the diagram elements are re-displayed based on one or more interactions received with the interactive ink of the at least identified connector or one or more of the plurality of diagram elements connected thereby and in accordance with the determined geometrical characteristics.

First claim

We claim:

1. A system for hand-drawing connections of diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the at least one non-transitory computer readable medium configured to:
cause display of a plurality of input diagram elements in interactive ink on a display device associated with the computing device;
identify at least one diagram element as a non-centered or a crossed connector which connects a plurality of diagram elements, each diagram element having a center of geometry, the non-centered or crossed connector defining a connection path which is offset from the centers of geometry of the connected diagram elements with a non-zero offset;
determine at least one of a projection line, point, and direction associated with the at least one identified connector; and
cause re-display of the diagram elements based on one or more interactions received with the interactive ink of the at least one identified non-centered or crossed connector or one or more of the plurality of diagram elements connected thereby and in accordance with the non-zero offset and the position of the determined at least one projection line, point, and direction, the position of the determined at least one projection line, point, and direction being updated when the diagram elements are re-displayed;
wherein an orientation of the at least one of a projection line, point, and direction associated with the at least one identified connector is determined based on an angle formed by a reference connector, and wherein the reference connector is connected nearest to a center of geometry of the at least one diagram element of the plurality of diagram elements.

2. A system as claimed in claim 1 , wherein the at least one connector is identified based on characteristics of the connector and positional relationships between the diagram elements.

3. A system as claimed in claim 1 , wherein the at least one projection line, point, and direction associated with the at least one identified connector is related to a connection path between geometrical features of the connected diagram elements.

4. A system as claimed in claim 3 , wherein the center of geometry is one of the geometrical features of the connected diagram elements.

5. A system as claimed in claim 4 , the at least one non-transitory computer readable medium being configured to determine the at least one projection line, point, and direction associated with the at least one identified connector to include a relationship between points of connection of the at least one connector with the connected diagram elements.

6. A system as claimed in claim 5 , wherein the connection path is offset from the centers of geometry of the connected diagram elements based on the points of connection.

7. A system as claimed claim 1 , wherein the interactive ink is digital ink.

8. A method for hand-drawing diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:
displaying a plurality of input diagram elements in interactive ink on a display device associated with the computing device;
identifying at least one diagram element as a non-centered or a crossed connector which connects a plurality of diagram elements, each diagram element having a center of geometry, the non-centered or crossed connector defining a connection path which is offset from the centers of geometry of the connected diagram elements with a non-zero offset;
determining at least one of a projection line, point, and direction associated with the at least one identified; and
re-displaying the diagram elements based on one or more interactions received with the interactive ink of the at least one identified non-centered or crossed connector or one or more of the plurality of diagram elements connected thereby and in accordance with the non-zero offset and the position of the determined at least one projection line, point, and direction, the position of the determined at least one projection line, point, and direction being updated when the diagram elements are re-displayed;
wherein an orientation of the at least one of a projection line, point, and direction associated with the at least one identified connector is determined based on an angle formed by a reference connector, and wherein the reference connector is connected nearest to a center of geometry of the at least one diagram element of the plurality of diagram elements.

9. A method as claimed in claim 8 , comprising identifying the at least one connector based on characteristics of the connector and positional relationships between the diagram elements.

10. A method as claimed in claim 8 , wherein the at least one projection line, point, and direction associated with the at least one identified connector is related to a connection path between geometrical features of the connected diagram elements.

11. A method as claimed in claim 10 , wherein the center of geometry is one of the geometrical features of the connected diagram elements.

12. A method as claimed in claim 11 , wherein the at least one projection line, point, and direction associated with the at least one identified connector are determined to include a relationship between points of connection of the at least one connector with the connected diagram elements.

13. A method as claimed in claim 12 , wherein the connection path is offset from the centers of geometry of the connected diagram elements based on the points of connection.

14. A method as claimed claim 8 , wherein the interactive ink is digital ink.

15. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for hand-drawing diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:
displaying a plurality of input diagram elements in interactive ink on a display device associated with the computing device;
identifying at least one diagram element as a non-centered or a crossed connector which connects a plurality of diagram elements, each diagram element having a center of geometry, the non-centered or crossed connector defining a connection path which is offset from the centers of geometry of the connected diagram elements with a non-zero offset;
determining at least one of a projection line, point, and direction associated with the at least one identified connector; and
re-displaying the diagram elements based on one or more interactions received with the interactive ink of the at least one identified non-centered or crossed connector or one or more of the plurality of diagram elements connected thereby and in accordance with the non-zero offset and the position of the determined at least one projection line, point, and direction, the position of the determined at least one projection line, point, and direction being updated when the diagram elements are re-displayed;
wherein an orientation of the at least one of a projection line, point, and direction associated with the at least one i

Assignees

• Myscript

Inventors

• Mélinand Robin
• Bednarowicz Romain
• SIDOLI Claire
• RIC Fabien
• Bonnaud Anne

Classifications

• G06K9/00476 Primary

  Physics · mapped topic

• G06K9/00422

  Physics · mapped topic

• G06K9/4638

  Physics · mapped topic

• G06K9/222

  Physics · mapped topic

• G06K9/2081

  Physics · mapped topic