Physical cursor control in microfluidic display devices

Patent metadata

Field	Value
Publication number	US-11481069-B2
Application number	US-202017021191-A
Country	US
Kind code	B2
Filing date	Sep 15, 2020
Priority date	Sep 15, 2020
Publication date	Oct 25, 2022
Grant date	Oct 25, 2022

Abstract

Physical assistance for identification of an input location on a touchscreen may include detecting that a user has contacted a touchscreen at an initial touch point with a pointing device, predicting an input location from one or more input locations for receiving input on the touchscreen, determining a path from the initial touch point to the predicted input location, and generating a physical cursor on the touchscreen at a location proximate to the pointing device, wherein the physical cursor is a raised portion of the touchscreen.

First claim

What is claimed is:

1. A computer-implemented method for physically assisting in identification of an input location on a touchscreen, the method comprising:
identifying a plurality of input locations on the touchscreen;
detecting that a user has contacted the touchscreen at an initial touch point with a pointing device;
predicting the input location from the plurality of input locations on the touchscreen;
determining a path from the initial touch point to a predicted input location;
generating a raised portion of the touchscreen at a location proximate to the initial touch point;
determining that a current pointing device location is not on the path; and
moving the raised portion of the touchscreen such that the pointing device is guided to the path.

2. The computer-implemented method of claim 1 , wherein the pointing device is a finger of a user.

3. The computer-implemented method of claim 1 , further comprising predicting a level of dexterity of the user.

4. The computer-implemented method of claim 1 , further comprising:
detecting a level of dexterity of the user; and
predicting an assistance requirement for the user based on a detected level of dexterity.

5. The computer-implemented method of claim 4 , wherein a machine learning model that predicts user attention based on user patterns while controlling a cursor is used to detect the level of dexterity of the user.

6. The computer-implemented method of claim 1 , wherein the generating the raised portion of the touchscreen further comprises generating the raised portion on a front side of the pointing device between a touchpoint and the input location.

7. The computer-implemented method of claim 1 , wherein the generating the raised portion of the touchscreen further comprises generating the raised portion on a back side of the pointing device between a touchpoint and the input location.

8. The computer-implemented method of claim 1 , wherein the generating the raised portion of the touchscreen further comprises generating the raised portion on two or more sides of the pointing device.

9. A computer system that physically assists in identification of an input location on a touchscreen, the computer system comprising:
a processor and a memory;
a touchscreen display that includes microfluidics, wherein in response to an instruction from the processor, fluid is selectively applied to a portion of the touchscreen to raise the portion of the touchscreen;
one or more computer readable storage media having program instructions executable by the processor to perform operations comprising:
identifying a plurality of input locations on the touchscreen;
detecting that a user has contacted the touchscreen at an initial touch point with a pointing device;
predicting the input location from the plurality of input locations on the touchscreen;
determining a path from the initial touch point to a predicted input location;
generating a raised portion of the touchscreen at a location proximate to the initial touch point;
determining that a current pointing device location is not on the path; and
moving the raised portion of the touchscreen such that the pointing device is guided to the path.

10. The computer system of claim 9 , wherein the pointing device is a finger of a user.

11. The computer system of claim 9 , further comprising predicting a level of dexterity of the user.

12. The computer system of claim 9 , further comprising:
detecting a level of dexterity of the user; and
predicting an assistance requirement for the user based on a detected level of dexterity.

13. The computer system of claim 12 , wherein a machine learning model that predicts user attention based on user patterns while controlling a cursor is used to detect the level of dexterity of the user.

14. The computer system of claim 9 , wherein the generating the raised portion of the touchscreen further comprises generating the raised portion on a front side of the pointing device between a touchpoint and the input location.

15. The computer system of claim 9 , wherein the generating the raised portion of the touchscreen further comprises generating the raised portion on a back side of the pointing device between a touchpoint and the input location.

16. The computer system of claim 9 , wherein the generating the raised portion of the touchscreen further comprises generating the raised portion on two or more sides of the pointing device.

17. A computer program product for physically assisting in identification of an input location on a touchscreen, the computer program product comprising:
a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:
instructions for identifying a plurality of input locations on the touchscreen;
instructions for detecting that a user has contacted the touchscreen at an initial touch point with a pointing device;
instructions for predicting the input location from the plurality of input locations on the touchscreen;
instructions for determining a path from the initial touch point to a predicted input location;
instructions for generating a raised portion of the touchscreen at a location proximate to the initial touch point;
instructions for determining that a current pointing device location is not on the path; and
instructions for moving the raised portion of the touchscreen such that the pointing device is guided to the path.

18. The computer program product of claim 17 , further comprising instructions for predicting a level of dexterity of the user.

19. The computer program product of claim 17 , further comprising:
instructions for detecting a level of dexterity of the user; and
instructions for predicting an assistance requirement for the user based on a detected level of dexterity.

20. The computer program product of claim 19 , wherein a machine learning model that predicts user attention based on user patterns while controlling a cursor is used to detect the level of dexterity of the user.

Assignees

• IBM

Inventors

• Walcott Aisha
• Rakshit Sarbajit K
• Crawford Catherine H
• Gunnels John A

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• A61F4/00 Primary

  Methods or devices enabling patients or disabled persons to operate an apparatus or a device not forming part of the body · CPC title

• G16H50/20

  for computer-aided diagnosis, e.g. based on medical expert systems · CPC title

• G16H40/63

  for local operation · CPC title

• G06F2203/04801

  Cursor retrieval aid, i.e. visual aspect modification, blinking, colour changes, enlargement or other visual cues, for helping user do find the cursor in graphical user interfaces · CPC title