Handwriting feedback

Patent metadata

Field	Value
Publication number	US-11675495-B2
Application number	US-202217676014-A
Country	US
Kind code	B2
Filing date	Feb 18, 2022
Priority date	Mar 9, 2021
Publication date	Jun 13, 2023
Grant date	Jun 13, 2023

Abstract

A computer-implemented method for generating feedback based on a handwritten text, comprises the steps of initializing a writing instrument to be used in a writing operation comprising a handwritten text and capturing and processing the handwritten text to generate digital text data. The method further comprises the steps of identifying at least one handwritten text attribute associated with the digital text data, comparing the at least one handwritten text attribute with predefined textual feature attributes, and generating a textual feature based on the compared at least one handwritten text attribute and predefined textual feature attributes. In addition, the method comprises the steps of modifying the digital text data using the textual feature and generating feedback to a user based on the modified digital text data.

First claim

The invention claimed is:

1. A computer-implemented method for generating feedback based on a handwritten text, comprising steps of:
initializing a writing instrument to be used in a writing operation comprising the handwritten text;
capturing and processing the handwritten text to generate digital text data by applying a writing digitization algorithm, wherein applying the writing digitization algorithm includes converting tracking data to a set of handwriting feature vectors, and converting the set of handwriting feature vectors to the digital text data via vector-to-character mapping;
identifying at least one handwritten text attribute associated with the digital text data;
comparing the at least one handwritten text attribute with predefined textual feature attributes;
generating a textual feature based on the compared at least one handwritten text attribute and the predefined textual feature attributes, wherein generating the textual feature comprises applying a textual feature generator network trained to perform a style transfer of the predefined textual feature attributes;
modifying the digital text data using the textual feature; and
generating the feedback to a user (U) based on the modified digital text data.

2. The method according to claim 1 , wherein identifying the at least one handwritten text attribute associated with the digital text data comprises applying a handwriting attribute identification algorithm that determines at least one relevant handwriting feature vector of the set of handwriting feature vectors based on a dominance of respective handwriting feature vectors.

3. The method according to claim 1 , wherein initializing the writing instrument comprises capturing input data.

4. The method according to claim 3 , wherein capturing the input data comprises applying an input algorithm.

5. The method according to claim 1 , wherein capturing and processing the handwritten text comprises capturing the tracking data from a capturing subsystem, wherein the capturing subsystem is configured to generate the tracking data of the writing instrument during the writing operation.

6. The method according to claim 5 , wherein the capturing subsystem comprises one or more sensors including a capacity sensitive surface, an accelerometer, a gyroscope, a magnetometer, a force sensor, and/or an optical sensor.

7. The method according to claim 5 , wherein the tracking data comprises motion tracking data of the writing instrument, and wherein the motion tracking data comprises one or more of a position of the writing instrument, a direction of motion of the writing instrument, a speed of the motion of the writing instrument, an acceleration of the motion of the writing instrument, a force acting on the writing instrument, a mode of handling of the writing instrument, the mode of use of the writing instrument.

8. The method according to claim 1 , wherein comparing the at least one handwritten text attribute with the predefined textual feature attributes comprises applying a textual feature selection algorithm.

9. The method according to claim 8 , wherein the predefined textual feature attributes are stored in a textual feature attribute database, and wherein the textual feature attribute database comprises predefined attribute weightings for each of the predefined textual feature attributes.

10. The method according to claim 9 , wherein the textual feature selection algorithm is configured to alter one or more of the predefined attribute weightings of relevant predefined textual feature attributes based on the at least one handwritten text attribute and/or based on an identified preference.

11. A system for generating feedback based on a handwritten text, comprising:
a writing instrument configured to be operated by a user (U) to perform the handwritten text;
a capturing subsystem configured to capture the handwritten text;
a processing subsystem configured to:
process the handwritten text to digital text data by applying a writing digitization algorithm, wherein the writing digitization algorithm is configured to convert tracking data to a set of handwriting feature vectors, and convert the set of handwriting feature vectors to the digital text data via vector-to-character mapping;
identify at least one handwritten text attribute associated with the digital text data;
compare the at least one handwritten text attribute with predefined textual feature attributes;
generate a textual feature based on the compared at least one handwritten text attribute and the predefined textual feature attributes, wherein generating the textual feature comprises applying a textual feature generator network trained to perform a style transfer of the predefined textual feature attributes;
modify the digital text data using the textual feature; and
a user interface subsystem configured to generate the feedback based on the modified digital text data and to reproduce the generated feedback to the user (U).

12. The system according to claim 11 , further comprising a textual feature attribute database comprising predefined textual feature attributes each having a predefined attribute weighting required to generate the textual feature, or, a link to the textual feature attribute database comprising the predefined textual feature attributes each having the predefined attribute weighting.

13. The system according to claim 11 , wherein the system is configured to run a computer-implemented method for generating the feedback based on the handwritten text.

14. The system according to claim 11 , wherein the capturing subsystem is configured to generate the tracking data of the writing instrument during a writing operation.

15. The system according to claim 14 , wherein the tracking data comprises one or more of motion tracking data of the writing instrument, visual data of a writing medium and/or the writing instrument, and capacity sensitive surface data of a capacity sensitive surface.

16. The system according to claim 11 , wherein the processing subsystem is configured to run a textual feature selection algorithm configured to select relevant predefined textual features, each having associated predefined textual feature attributes and predefined attribute weightings.

17. The system according to claim 16 , wherein the textual feature selection algorithm is configured to determine the relevant predefined textual features by comparing and/or associating the predefined textual feature attributes with the at least one handwritten text attribute, more specifically wherein the predefined attribute weightings are altered based on the compared and associated predefined textual feature attributes and the at least one handwritten text attribute.

18. A non-transitory computer-readable storage medium carrying one or more sequences of one or more instructions for generating feedback based on a handwritten text which, when executed by one or more processors, cause an apparatus to at least perform the following steps:
initializing a writing instrument to be used in a writing operation comprising the handwritten text;
capturing and processing the handwritten text to generate digital text data by applying a writing digitization algorithm, wherein applying the writing digitization algorithm includes converting tracking data to a set of handwriting feature vectors, and converting the set of handwriting feature vectors to the digital text data via vector-to-character mapping;
identifying at least one handwritten text attribute associated with the digital text data;
comparing the at least one

Assignees

• SOCIéTé BIC

Inventors

• Duffy David
• Wright Christopher-John
• Beard Timothy Giles
• Lau Wai Keith
• Schnabel William Andrew

Classifications

• G06F3/04883 Primary

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F18/24133

  Distances to prototypes · CPC title

• G06V30/245

  Font recognition · CPC title

• G06F40/166

  Editing, e.g. inserting or deleting · CPC title

• G06V30/226

  of cursive writing · CPC title