Physiological feedback for predictive models

Patent metadata

Field	Value
Publication number	US-12533064-B2
Application number	US-202318118849-A
Country	US
Kind code	B2
Filing date	Mar 8, 2023
Priority date	Mar 8, 2023
Publication date	Jan 27, 2026
Grant date	Jan 27, 2026

Abstract

This document relates to employing biosignals to evaluate predictions made by predictive models. For example, user attention can be inferred from a user attention signal such as gaze. When the user directs attention to a prediction output by a given predictive model, a user reaction signal such as an electroencephalogram or pupillary diameter measurement can be processed to determine whether the user perceives an error. If the user perceives an error, an error indication can be output. Error indications can be used to evaluate the predictive model, replace predictions generated by the predictive model, train the predictive model, etc.

First claim

The invention claimed is:

1 . A method comprising:
receiving a user attention signal conveying where a user directs attention over a period of time; based on the user attention signal, identifying a particular time when the user directs attention to a prediction output by a predictive model; receiving a user reaction signal conveying a physiological reaction of the user to the prediction; determining whether the physiological reaction indicates that the user perceives an error responsive to directing attention to the prediction; and in an instance when the physiological reaction indicates that the user perceives an error, outputting an error indication.

2 . The method of claim 1 , wherein the user attention signal indicates where the user is gazing during the period of time.

3 . The method of claim 2 , wherein the user reaction signal comprises an electroencephalogram signal.

4 . The method of claim 3 , further comprising determining that the electroencephalogram signal indicates that the user perceives an error when the electroencephalogram signal includes an error-related potential occurring after the particular time when the user directs attention to the prediction.

5 . The method of claim 4 , wherein the error-related potential occurs within one second after the particular time when the user directs attention to the prediction.

6 . The method of claim 2 , wherein the user reaction signal comprises a pupil diameter measurement.

7 . The method of claim 1 , further comprising evaluating and ranking multiple predictive models based on user attention signals and user reaction signals relating to multiple predictions output by the multiple predictive models.

8 . The method of claim 1 , wherein determining whether the physiological reaction indicates that the user perceives an error comprises applying a rule to the user reaction signal.

9 . The method of claim 1 , wherein determining whether the physiological reaction indicates that the user perceives an error comprises applying a machine-trained classifier to the user reaction signal.

10 . The method of claim 9 , further comprising tuning the machine-trained classifier to the user.

11 . The method of claim 10 , the machine-trained classifier comprising a deep neural network.

12 . A system comprising:
a processor; and a computer-readable storage medium storing instructions which, when executed by the processor, cause the system to: output a prediction; receive an error indication indicating that a user perceives an error in the prediction, the error indication being based on a user attention signal indicating that the user directs attention to the prediction at a particular time and a user reaction signal indicating that the user perceives an error responsive to directing attention to the prediction; and based on the error indication, replace the prediction with another prediction.

13 . The system of claim 12 , wherein the prediction comprises a predicted word and the another prediction comprises another predicted word.

14 . The system of claim 13 , wherein the predicted word has a highest score assigned by a generative text model, and the another predicted word has a next-highest score assigned by the generative text model.

15 . The system of claim 14 , wherein the user attention signal indicates that the user is gazing at the predicted word.

16 . The system of claim 15 , further comprising an eye tracking sensor configured to generate the user attention signal.

17 . The system of claim 12 , further comprising an electroencephalogram sensor configured to generate the user reaction signal.

18 . The system of claim 12 , further comprising an eye tracking sensor configured to generate the user reaction signal based on pupil diameter.

19 . A computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform acts comprising:
receiving a user attention signal conveying where a user directs attention over a period of time; based on the user attention signal, identifying a particular time when the user directs attention to a prediction output by a predictive model; receiving a user reaction signal conveying a physiological reaction of the user to the prediction; determining whether the physiological reaction indicates that the user perceives an error responsive to directing attention to the prediction; and in an instance when the physiological reaction indicates that the user perceives an error, outputting an error indication.

20 . The computer-readable storage medium of claim 19 , the user attention signal indicating that the user gazes at the prediction at the particular time, the user reaction signal indicating an error-related potential occurring within a specified time window after the particular time.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wang Yu-Te
• Djuric Nemanja
• Tashev Ivan J
• Winters Raymond Michael
• Gamper Hannes
• Emmanouilidou Dimitra

Classifications

• A61B5/291

  for electroencephalography [EEG] · CPC title

• A61B5/163

  by tracking eye movement, gaze, or pupil change · CPC title

• A61B5/168 Primary

  Evaluating attention deficit, hyperactivity · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08

  Learning methods · CPC title