Guided object targeting based on physiological feedback

Patent metadata

Field	Value
Publication number	US-12118141-B2
Application number	US-202318123212-A
Country	US
Kind code	B2
Filing date	Mar 17, 2023
Priority date	Mar 17, 2023
Publication date	Oct 15, 2024
Grant date	Oct 15, 2024

Abstract

This document relates to employing physiological feedback to aid a user in targeting objects using a computing device. One example relates to a method or technique that can include receiving a user attention signal conveying where a user directs attention. The method or technique can also include, based on the user attention signal, identifying a predicted object that the user intends to target with a targeting mechanism and outputting a visual identification of the predicted object. The method or technique can also include receiving a user reaction signal conveying a physiological reaction of the user to the visual identification of the predicted object. The method or technique can also include, in an instance when the physiological reaction of the user indicates an error, identifying another predicted object that the user intends to target with the targeting mechanism.

First claim

The invention claimed is:

1. A method comprising:
receiving a user attention signal conveying where a user directs attention;
based at least on the user attention signal, identifying a predicted object that the user intends to target with a targeting mechanism;
outputting a visual identification of the predicted object;
receiving a user reaction signal conveying an implicit physiological reaction of the user to the visual identification of the predicted object; and
in an instance when the implicit physiological reaction of the user indicates an error, identifying another predicted object that the user intends to target with the targeting mechanism.

2. The method of claim 1 , further comprising:
responsive to receiving a selection input, performing a selection action on the another predicted object.

3. The method of claim 2 , wherein the predicted object is a first character, the another predicted object is a second character, and the selection action involves entering the second character into a text entry region on a display.

4. The method of claim 3 , wherein the targeting mechanism comprises a cursor.

5. The method of claim 4 , wherein the visual identification of the predicted object involves changing at least one of color, size, or font of the first character.

6. The method of claim 1 , further comprising:
outputting haptic feedback concurrently with the visual identification of the predicted object.

7. The method of claim 6 , wherein the haptic feedback is provided via a controller.

8. The method of claim 7 , further comprising:
receiving navigation inputs from the controller, the navigation inputs moving the targeting mechanism on a trajectory toward the predicted object; and
when the targeting mechanism reaches a threshold distance from the predicted object, performing a jump action that moves the targeting mechanism to the predicted object.

9. The method of claim 1 , further comprising determining respective targeting probabilities for a plurality of objects based at least on the user attention signal.

10. The method of claim 9 , wherein the predicted object has a highest targeting probability and the another predicted object has a next-highest targeting probability.

11. The method of claim 10 , wherein the respective targeting probabilities are based at least on attention-based coordinates derived from the user attention signal.

12. The method of claim 11 , wherein the respective targeting probabilities are based at least on respective distances from the attention-based coordinates to respective centers of each of the objects.

13. The method of claim 1 , wherein the predicted object and the another predicted object are displayed in a virtual or augmented reality experience.

14. A system comprising:
a processor; and
a computer-readable storage medium storing instructions which, when executed by the processor, cause the system to:
based at least on a user attention signal, identify a predicted object that a user intends to target;
output an identification of the predicted object;
receiving a user reaction signal conveying an implicit physiological reaction of the user to the identification of the predicted object;
in an instance when the user reaction signal indicates that the predicted object is erroneous;
identify another predicted object that the user intends to target; and
perform a selection action on the another predicted object responsive to a selection input.

15. The system of claim 14 , further comprising an electroencephalogram sensor configured to provide an electroencephalogram signal as the user reaction signal.

16. The system of claim 15 , wherein the instructions, when executed by the processor, cause the system to:
determine that the electroencephalogram signal indicates that the user perceives an error when the electroencephalogram signal includes an error-related potential within a specified time window after the outputting the identification of the predicted object.

17. The system of claim 14 , further comprising an eye tracking sensor configured to provide a gaze tracking signal as the user reaction signal.

18. The system of claim 17 , wherein the instructions, when executed by the processor, cause the system to:
calibrate the eye tracking sensor based at least on the user reaction signal.

19. A computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform acts comprising:
receiving a user attention signal conveying where a user directs attention;
based at least on the user attention signal, identifying a predicted object that the user intends to target with a targeting mechanism;
outputting a visual identification of the predicted object;
receiving a user reaction signal conveying an implicit physiological reaction of the user to the visual identification of the predicted object; and
in an instance when the implicit physiological reaction of the user indicates an error, identifying another predicted object that the user intends to target with the targeting mechanism.

20. The computer-readable storage medium of claim 19 , wherein the user reaction signal comprises a pupil diameter measurement.

Assignees

• Microsoft Technology Licensing Llc
• Micrsoft Tech Licensing Llc

Inventors

• Wang Yu-Te
• Dong Jen-Tse

Classifications

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/02

  Input arrangements using manually operated switches, e.g. using keyboards or dials · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G02B27/017

  Head mounted · CPC title

• G06F3/04892

  Arrangements for controlling cursor position based on codes indicative of cursor displacements from one discrete location to another, e.g. using cursor control keys associated to different directions or using the tab key (arrangements for controlling cursor position based on coordinate signals G06F3/038) · CPC title