Building a model based on responses from sensors

Patent metadata

Field	Value
Publication number	US-12045694-B2
Application number	US-201916448176-A
Country	US
Kind code	B2
Filing date	Jun 21, 2019
Priority date	Jun 21, 2019
Publication date	Jul 23, 2024
Grant date	Jul 23, 2024

Abstract

A system includes a virtual reality system configured to enable a user to interact with a virtual environment, a plurality of biofeedback sensors configured to monitor a user, and a computer system including a virtual reality module configured to generate at least a view of the virtual environment, a biofeedback module configured to fuse output of the plurality of biofeedback sensors with a plurality of events within the virtual environment, a training module configured to generate a model of user behavior, wherein the training module executed by the computer system enables the computer system to make a prediction of a user response of the user based on a corpus of biofeedback data, and an alert module configured to generate at least one alert to the user via the virtual reality system based on the user response predicted by the computer system.

First claim

What is claimed is:

1. A method of constructing a predictive model for user behavior in a virtual environment comprising:
determining a baseline response for each of a plurality of sensors and a user;
building an individualized model comprising a plurality of priors generated from the baseline responses, the individualized model configured to predict a given response of the user prior to the user starting a performance of the given response based on a time-stamped stimulus presentation in the virtual environment occurring after the building of the individualized model, wherein each of the priors is associated with a respective prediction and a plurality of baseline features in a feature space that is specific to the user and the prediction for an interval of time;
sampling, iteratively, new responses of the user to respective events in the virtual environment during the time-stamped stimulus presentation, wherein respective intervals of times are determined for each of the new responses;
linking each of the new responses and the respective events to respective ones of the intervals of time; and
updating a first prior of the priors of the individualized model based on linked ones of the new responses and events, wherein updating the first prior comprises updating at least one value of the plurality of baseline features in the feature space of the first prior.

2. The method of claim 1 , further comprising:
predicting a future response of the user by determining a threshold of predictability of the given response of the user under a pre-determined context within the virtual environment;
receiving, during the sampling, output data from the sensors;
detecting in the output data that the threshold is satisfied; and
generating an alert to the virtual environment upon detecting that the threshold is satisfied.

3. The method of claim 2 , further comprising receiving a confirmation from the user based on the alert to the virtual environment.

4. The method of claim 2 , further comprising pausing access of the user to the virtual environment upon detecting that the threshold is satisfied.

5. The method of claim 2 , further comprising generating an alert, which is transmitted to a remote device.

6. The method of claim 5 , further comprising:
pausing access of the user to the virtual environment upon detecting that the threshold is satisfied; and
resuming the access of the user to the virtual environment upon receipt of a confirmation from the remote device.

7. The method of claim 1 , wherein updating the first prior further comprises updating a weight of the at least one value of the plurality of baseline features in the feature space of the first prior.

8. The method of claim 1 , further comprising:
generating a prediction of a future response of the user within the virtual environment, using the individualized model and the new responses of the user; and
generating an alert to the virtual environment based on the prediction of the future response.

9. A non-transitory computer readable storage medium comprising computer executable instructions which when executed by a computer cause the computer to perform a method of constructing a predictive model for user behavior in a virtual environment, the method comprising:
determining a baseline response for each of a plurality of sensors and a user;
building an individualized model comprising a plurality of priors generated from the baseline responses, the individualized model configured to predict a given response of the user prior to the user starting a performance of the given response based on a time-stamped stimulus presentation in the virtual environment occurring after the building of the individualized model, wherein each of the priors is associated with a respective prediction and a plurality of baseline features in a feature space that is specific to the user and the prediction for an interval of time;
sampling, iteratively, new responses of the user to respective events in the virtual environment during the time-stamped stimulus presentation, wherein respective intervals of times are determined for each of the new responses;
linking each of the new responses and the respective events to respective ones of the intervals of time; and
updating a first prior of the priors of the individualized model based on linked ones of the new responses and events, wherein updating the first prior comprises updating at least one value of the plurality of baseline features in the feature space of the first prior.

10. The computer readable storage medium of claim 9 , further comprising:
predicting a future response of the user by determining a threshold of predictability of the given response of the user under a pre-determined context within the virtual environment;
receiving, during the sampling, output data from the sensors;
detecting in the output data that the threshold is satisfied; and
generating an alert to the virtual environment upon detecting that the threshold is satisfied.

11. The computer readable storage medium of claim 10 , further comprising receiving a confirmation from the user based on the alert to the virtual environment.

12. The computer readable storage medium of claim 10 , further comprising pausing access of the user to the virtual environment upon detecting that the threshold is satisfied.

13. The computer readable storage medium of claim 10 , further comprising generating an alert, which is transmitted to a remote device.

14. The computer readable storage medium of claim 13 , further comprising:
pausing access of the user to the virtual environment upon detecting that the threshold is satisfied; and
resuming the access of the user to the virtual environment upon receipt of a confirmation from the remote device.

15. The computer readable storage medium of claim 9 , wherein updating the first prior further comprises updating a weight of the at least one value of the plurality of baseline features in the feature space of the first prior.

16. The computer readable storage medium of claim 9 , further comprising:
generating a prediction of a future response of the user within the virtual environment, using the individualized model and the new responses of the user; and
generating an alert to the virtual environment based on the prediction of the future response.

Assignees

• IBM

Inventors

• Reinen Jenna
• Sipolins Aldis
• Watson Patrick
• Tejwani Ravi
• Cavallo Marco
• Wu Hui

Classifications

• H04L67/535

  Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34) · CPC title

• G06F3/015

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title

• G16C60/00

  Computational materials science, i.e. ICT specially adapted for investigating the physical or chemical properties of materials or phenomena associated with their design, synthesis, processing, characterisation or utilisation · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F2203/011

  Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns · CPC title