Brain-mobile interface optimization using internet-of-things

Patent metadata

Field	Value
Publication number	US-10796246-B2
Application number	US-201715857794-A
Country	US
Kind code	B2
Filing date	Dec 29, 2017
Priority date	Dec 29, 2016
Publication date	Oct 6, 2020
Grant date	Oct 6, 2020

Abstract

A Brain-Mobile Interface (BMoI) system is provided. A control circuit is configured to execute a predictive model to generate a defined number of predicted signal features in future time based on a number of signal features extracted from a first type sensory data (e.g., electroencephalogram (EEG) data). A predicted future mental state(s) can thus be generated based on the number of predicted signal features and used to trigger a corresponding action(s) in a BMoI application(s). In a non-limiting example, a second type sensory data (e.g., electrocardiogram (ECG) data) can be used to improve accuracy of the predictive model. By using the predicted signal features to generate the predicted future mental state(s) to control the BMoI application(s), it is possible to duty-cycle the BMoI system to help reduce power consumption and processing latency, thus allowing the BMoI application(s) to operate in real-time with improved accuracy and power consumption.

First claim

What is claimed is:

1. A Brain-Mobile Interface (BMoI) system comprising:
an input interface coupled to a selected communication medium;
an output interface coupled to the selected communication medium; and
a control circuit coupled to the input interface and the output interface and configured to:
receive a first type sensory data via the input interface;
receive a second type sensory data via the input interface within a time window from receiving the first type sensory data;
extract a plurality of signal features from the received first type sensory data;
execute a predictive model to generate a defined number of predicted signal features in at least one prediction window based on the plurality of extracted signal features;
validate the defined number of predicted signal features based on the second type sensory data;
generate at least one predicted future mental state based on the defined number of predicted signal features; and
provide the at least one predicted future mental state to the output interface.

2. The BMoI system of claim 1 wherein the control circuit is further configured to execute the predictive model to generate six predicted signal features in the at least one prediction window based on the plurality of extracted signal features.

3. The BMoI system of claim 1 wherein:
the at least one prediction window corresponds to a temporal duration; and
the control circuit is further configured to execute the predictive model to generate the defined number of predicted signal features in the at least one prediction window in less than one-half of the temporal duration.

4. The BMoI system of claim 1 wherein the control circuit comprises:
feature extraction circuitry configured to receive the first type sensory data and extract the plurality of signal features from the received first type sensory data;
prediction circuitry configured to execute the predictive model based on the plurality of extracted signal features to generate the defined number of predicted signal features in the at least one prediction window; and
translation circuitry configured to generate the at least one predicted future mental state based on the defined number of predicted signal features and provide the at least one predicted future mental state to the output interface.

5. The BMoI system of claim 1 further comprising:
at least one primary sensor configured to collect the first type sensory data;
at least one secondary sensor configured to collect the second type sensory data; and
data sensing circuitry configured to:
receive the first type sensory data from the at least one primary sensor;
receive the second type sensory data from the at least one secondary sensor; and
provide the first type sensory data and the second type sensory data to the input interface over the selected communication medium.

6. The BMoI system of claim 5 wherein:
the at least one primary sensor comprises at least one electroencephalogram (EEG) sensor configured to collect low accuracy and low latency EEG data representing human physiological response to a stimulus; and
the at least one secondary sensor comprises at least one electrocardiogram (ECG) sensor configured to collect high accuracy and high latency ECG data representing human psychological response to the stimulus.

7. The BMoI system of claim 5 wherein the control circuit is further configured to receive the second type sensory data via the input interface and calibrate the predictive model based on the received second type sensory data.

8. The BMoI system of claim 5 wherein the control circuit is further configured to stop receiving the second type sensory data in response to a pre-defined number of predicted future mental states being validated based on the second type sensory data.

9. The BMoI system of claim 5 wherein the control circuit is further configured to stop receiving the second type sensory data after calibrating the predictive model for a defined number of calibration iterations.

10. The BMoI system of claim 5 further comprising:
an Internet-of-Things (IoT) fog server comprising the control circuit, the input interface, and the output interface; and
a mobile communication device comprising the data sensing circuitry.

11. The BMoI system of claim 10 wherein the mobile communication device further comprises actuator circuitry communicatively coupled to the output interface via the selected communication medium and configured to trigger an application-specific action in the mobile communication device in response to receiving the at least one predicted future mental state.

12. A method for optimizing a Brain-Mobile Interface (BMoI) system using Internet-of-Things (IoT) comprising:
receiving a first type sensory data;
receiving a second type sensory data within a time window from receiving the first type sensory data;
extracting a plurality of signal features from the received first type sensory data;
executing a predictive model to generate a defined number of predicted signal features based on the plurality of extracted signal features in at least one prediction window;
validating the defined number of predicted signal features based on the second type sensory data; and
generating at least one predicted future mental state based on the defined number of predicted signal features.

13. The method of claim 12 further comprising generating six predicted signal features in the at least one prediction window based on the plurality of extracted signal features.

14. The method of claim 12 further comprising executing the predictive model to generate the defined number of predicted signal features in less than one-half of a temporal duration of the at least one prediction window.

15. The method of claim 12 further comprising:
receiving the first type sensory data from at least one primary sensor; and
receiving the second type sensory data from at least one secondary sensor.

16. The method of claim 15 further comprising calibrating the predictive model based on the received second type sensory data.

17. The method of claim 15 further comprising stopping receiving the second type sensory data in response to a pre-defined number of predicted future mental states being validated based on the second type sensory data.

18. The method of claim 15 further comprising stopping receiving the second type sensory data after calibrating the predictive model for a defined number of calibration iterations.

19. The method of claim 15 further comprising:
communicatively coupling a mobile communication device to an IoT fog server via a selected communication medium;
coupling the mobile communication device to the at least one primary sensor and the at least one secondary sensor;
configuring the mobile communication device to:
receive the first type sensory data from the at least one primary sensor;
receive the second type sensory data from the at least one secondary sensor; and
provide the first type sensory data and the second type sensory data to the IoT fog server via the selected communication medium; and

configuring the IoT fog server to:
extract the plurality of signal features from the received first type sensory data;
execute the predictive model to generate the defined number of predicted signal features based on the plurality of extracted signal features in the at least one prediction window;
generate the at least one predicted future mental state based on the defined number of predicted signal features; and
provide the at least one predicte

Assignees

• Gupta Sandeep
• Banerjee Ayan
• Esfahani Mohammad Javad Sohankar
• Oskooyee Seyed Koosha Sadeghi
• Univ Arizona State

Inventors

• Gupta Sandeep
• Banerjee Ayan
• Esfahani Mohammad Javad Sohankar
• Oskooyee Seyed Koosha Sadeghi

Classifications

• H04L67/12 Primary

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• G06N5/02 Primary

  Knowledge representation; Symbolic representation · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• H04W40/24

  Connectivity information management, e.g. connectivity discovery or connectivity update · CPC title

• Y02D30/00

  Reducing energy consumption in communication networks · CPC title