System and method of providing accessibility in use of digital ink

Patent metadata

Field	Value
Publication number	US-12374143-B2
Application number	US-202318167201-A
Country	US
Kind code	B2
Filing date	Feb 10, 2023
Priority date	Feb 10, 2023
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

A method of and system for providing accessibility for digital ink is carried out by detecting digital ink input on a user interface (UI) screen, the digital ink input being received via a writing instrument used by a user within a given time period; capturing an image of the digital ink input; examining the image of the digital ink input to detect tremors in the user; assigning a level to the tremors based on discontinuities or irregularities in the digital ink input within the given time period; determining based on the level that the digital ink input requires correction; and applying a correction to the digital ink input to correct the digital ink input.

First claim

What is claimed is:

1. A device comprising:
a processor; and
a memory in communication with the processor, the memory comprising executable instructions that, when executed by the processor, cause the device to perform functions of:
detecting receipt of a digital ink input on a user interface (UI) screen, the digital ink input being provided by a user;
determining a type of ink from the digital ink input, the type of ink including one of writing, drawing, highlighting, or circling;
examining the digital ink input to detect a physical disability of the user;
assigning a level for the physical disability based on at least one of a number of discontinuities in the digital ink input, a size of the discontinuities in the digital ink input and irregularities in the digital ink input;
determining based on the level and the type of ink that the digital ink input requires correction; and
automatically applying the correction responsive to the type of ink to the digital ink input to correct the digital ink input.

2. The device of claim 1 , wherein determining based on the level that the digital ink input requires the correction includes determining that the level exceeds a threshold.

3. The device of claim 1 , wherein automatically applying the correction to the digital ink input includes applying a correction type that includes at least one of joining one or more discontinued strokes of the digital ink input to correct discontinuities, smoothing the one or more strokes of the digital ink input, trimming the one or more strokes of the digital ink input, and straightening the one or more strokes of the digital ink input.

4. The device of claim 1 , wherein automatically applying the correction to the digital ink input includes providing the digital ink input and the type of ink as an input to a machine-learning model and receiving the corrected digital ink input as an output of the machine-learning model.

5. The device of claim 1 , wherein the memory comprises executable instructions that, when executed by the processor, further cause the device to perform functions of:
upon determining that the digital ink input requires correction, displaying a user interface element that enables the user to transmit a request for correcting the digital ink input; and
applying the correction, upon receiving the request for correcting the digital ink input.

6. The device of claim 1 , wherein the physical disability includes presence of tremors.

7. The device of claim 6 , wherein assigning the level for the physical disability includes assigning a level for the tremors.

8. The device of claim 1 , wherein the digital ink input includes at least one of a writing or a drawing.

9. A method for providing accessibility for digital ink comprising:
detecting digital ink input on a user interface (UI) screen, the digital ink input being received via a writing instrument used by a user within a given time period;
capturing an image of the digital ink input;
determining a type of ink from the digital ink input, the type of ink including one of writing, drawing, highlighting, or circling;
examining the image of the digital ink input to detect tremors in the user;
assigning a level to the tremors based on at least one of discontinuities or irregularities in the digital ink input within the given time period;
determining based on the level and the type of ink that the digital ink input requires correction; and
applying a correction responsive to the type of ink to the digital ink input to correct the digital ink input.

10. The method of claim 9 , wherein examining the image of the digital ink input and assigning the level to the tremors occurs in real time.

11. The method of claim 10 , further comprising providing the corrected digital ink input for display on the UI screen.

12. The method of claim 11 , further comprising displaying at least one of a first UI element that enables the user to accept the corrected digital ink input and a second UI element that enables the user to reject the corrected digital ink input.

13. The method of claim 12 , further comprising collecting data about acceptance or rejection of the corrected digital ink input and using the collected data to train a machine-learning model used to correct the digital ink input.

14. The method of claim 9 , wherein applying the correction to the digital ink input includes providing the digital ink input to a trained machine-learning model for providing the correction to the digital ink input and receiving a corrected digital ink input as an output from the machine-learning model.

15. The method of claim 9 , assigning the level to the tremors includes assigning the level based on a predetermined scale by examining at least one of a number of the discontinuities or the irregularities in the digital ink input within the given time period, and a size of the discontinuities or the irregularities.

16. The method of claim 9 , wherein applying the correction to the digital ink input includes applying a correction type that includes at least one of joining one or more discontinued strokes of the digital ink input to correct discontinuities, smoothing the one or more strokes of the digital ink input, trimming one or more strokes of the digital ink input, and straightening the one or more strokes of the digital ink input.

17. A non-transitory computer readable medium on which are stored instructions that when executed cause a programmable device to perform functions of: detecting receipt of digital ink input on a user interface (UI) screen, the digital ink input being provided by a user;
capturing the digital ink input;
determining a type of ink from the digital ink input, the type of ink including one of writing, drawing, highlighting, or circling;
examining the digital ink input to detect tremors in the user;
assigning a level to the tremors based on the digital ink input;
determining based on the level and the type of ink that the digital ink input requires correction;
providing the digital ink input and the type of ink to a trained machine-learning model for providing correction to the digital ink input;
receiving a corrected digital ink input as an output from the machine-learning model; and
providing the corrected digital ink input for display.

18. The non-transitory computer readable medium of claim 17 , wherein determining based on the level that the digital ink input requires the correction includes determining that the level exceeds a threshold.

19. The non-transitory computer readable medium of claim 17 , wherein the instructions further cause the programmable device to perform functions of:
upon determining that the digital ink input requires the correction, displaying a user interface element that enables the user to transmit a request for correcting the digital ink input; and
applying the correction, upon receiving the request for correcting the digital ink input.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Sharma Mrinal Kumar

Classifications

• G06V30/347

  Sampling; Contour coding; Stroke extraction · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V30/1423

  the instrument generating sequences of position coordinates corresponding to handwriting (preprocessing or recognising digital ink G06V30/32) · CPC title

• G06F40/171

  by use of digital ink · CPC title

• G06F3/04186

  Touch location disambiguation · CPC title