System and method for mixed content digital ink interactivity

Patent metadata

Field	Value
Publication number	US-10852938-B2
Application number	US-201816139928-A
Country	US
Kind code	B2
Filing date	Sep 24, 2018
Priority date	Jan 7, 2016
Publication date	Dec 1, 2020
Grant date	Dec 1, 2020

Abstract

A system, method and computer program product for use in providing interactive ink from handwriting input to a computing device are provided. The computing device is connected to an input device in the form of an input surface. A user is able to provide input by applying pressure to or gesturing above the input surface using either his or her finger or an instrument such as a stylus or pen. The present system and method monitors the input strokes. The computing device further has a processor and an ink management system for recognizing the handwriting input under control of the processor. The ink management system is configured to cause display of, on a display interface of a computing device, first digital ink in accordance with first handwriting input, allocate references to ink elements of the first digital ink, define boundaries of the first digital ink associated with the allocated references, cause placement of an editing position at a defined boundary in response to an editing gesture, cause display of, on the display interface of the computing device, first typeset ink in accordance with first typeset input at the editing position.

First claim

We claim:

1. A system for providing interactive ink on a computing device, the computing device comprising a processor, a memory and at least one non-transitory computer readable medium for recognizing handwriting input under control of the processor, the at least one non-transitory computer readable medium configured to:
cause display of, on a display interface of the computing device, first digital ink in accordance with first handwriting input;
allocate references to ink elements of the first digital ink;
define boundaries of the first digital ink associated with the allocated references;
cause placement of an editing position at a defined boundary in response to an editing gesture;
launch a typeset input subsystem in response to the editing gesture, the typeset input subsystem configured to receive typeset instructions from any type of typeset form of input;
cause display of, on the display interface of the computing device, first typeset ink in accordance with first typeset input at the editing position.

2. A system according to claim 1 , wherein the ink elements are at least part of strokes of the first digital ink.

3. A system according to claim 1 , the at least one non-transitory computer readable medium further configured to determine and store, in the memory of the computing system, ink objects including the references and mapped recognized element, wherein the recognized elements include at least part of strokes of the first handwriting input.

4. A system according to claim 3 , wherein the recognized elements are recognition candidates of the first handwriting input.

5. A system according to claim 1 , wherein the placement of the editing position comprises placement of a cursor at the defined boundary.

6. A system according to claim 1 , wherein the typeset input subsystem comprises a software keyboard on part of the display interface.

7. A system according to claim 6 , wherein the displayed software keyboard comprises a portion with editing suggestions associated with the ink elements at the boundary of the editing position.

8. A method for providing interactive ink on a computing device, the computing device comprising a processor, a memory and at least one non-transitory computer readable medium for recognizing handwriting input under control of the processor, the method comprising:
displaying on a display interface of the computing device, first digital ink in accordance with first handwriting input;
allocating references to ink elements of the first digital ink; defining boundaries of the first digital ink associated with the allocated references;
placing an editing position at a defined boundary in response to an editing gesture;
launching a typeset input subsystem in response to the editing gesture, the typeset input subsystem configured to receive typeset instructions from any type of typeset form of input;
displaying on the display interface of the computing device, first typeset ink in accordance with first typeset input at the editing position.

9. A method according to claim 8 , wherein the ink elements are at least part of strokes of the first digital ink.

10. A method according to claim 8 , further comprising determining and storing, in the memory of the computing system, ink objects including the references and mapped recognized element, wherein the recognized elements include at least part of strokes of the first handwriting input.

11. A method according to claim 10 , wherein the recognized elements are recognition candidates of the first handwriting input.

12. A method according to claim 8 , wherein placing the editing position comprises placing a cursor at the defined boundary.

13. A method according to claim 8 , further comprising launching a typeset input subsystem software keyboard on part of the display interface in response to the editing gesture.

14. A method according to claim 13 , wherein launching the typeset input subsystem comprises launching a software keyboard on part of the display interface.

15. A method according to claim 14 , further comprising providing on a portion of the displayed launched keyboard editing suggestions associated with the ink elements at the boundary of the editing position.

16. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for providing interactive ink on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for recognizing handwriting input under control of the processor, the method comprising:
displaying on a display interface of the computing device, first digital ink in accordance with first handwriting input;
allocating references to ink elements of the first digital ink;
defining boundaries of the first digital ink associated with the allocated references;
placing an editing position at a defined boundary in response to an editing gesture;
launching a typeset input subsystem in response to the editing gesture, the typeset input subsystem configured to receive typeset instructions from any type of typeset form of input;
displaying on the display interface of the computing device, first typeset ink in accordance with first typeset input at the editing position.

17. A non-transitory computer readable medium according to claim 16 , wherein the ink elements are at least part of strokes of the first digital ink.

18. A non-transitory computer readable medium according to claim 16 , further comprising determining and storing, in the memory of the computing system, ink objects including the references and mapped recognized element, wherein the recognized elements include at least part of strokes of the first handwriting input.

19. A non-transitory computer readable medium according to claim 18 , wherein the recognized elements are recognition candidates of the first handwriting input.

20. A non-transitory computer readable medium according to claim 16 , wherein placing the editing position comprises placing a cursor at the defined boundary.

21. A non-transitory computer readable medium according to claim 16 , wherein launching the typeset input subsystem comprises launching a software keyboard on part of the display interface.

22. A non-transitory computer readable medium according to claim 21 , further comprising providing on a portion of the displayed launched keyboard editing suggestions associated with the ink elements at the boundary of the editing position.

Assignees

• Myscript

Inventors

• Guiavarc'H Jean-Nicolas
• Chateigner Alain
• Bednarowicz Romain

Classifications

• G06F40/279

  Recognition of textual entities · CPC title

• G06F40/166

  Editing, e.g. inserting or deleting · CPC title

• G06F3/04883 Primary

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F3/0487

  using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser · CPC title

• G06F3/04812

  Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects · CPC title