Method and system for classification of cognitive load using data obtained from wearable sensors

Patent metadata

Field	Value
Publication number	US-12412671-B2
Application number	US-202217808468-A
Country	US
Kind code	B2
Filing date	Jun 23, 2022
Priority date	Jun 23, 2021
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

This disclosure relates generally to a method and system for classification of cognitive load (CL) using data obtained from wearable sensors. The disclosed method uses a multi-modal based approach using wrist-worn sensors for real time monitoring of CL in real world scenarios and improves the accuracy of detection of CL. A set of distinguishing features are selected from physiological signals received from the wrist-worn sensors. These features are used for training a classification model for classifying the CL of a patient into a no load or a high load. The set of distinguishing features are selected from domain specific features and signal property based generic features of the physiological signals. The disclosed method is used for classification of CL in scenarios such as to check how the cognitive load of a candidate varies during interviews, to assess the participants workload during online meetings and so on.

First claim

What is claimed is:

1. A processor implemented method comprising the steps of:
providing, via one or more hardware processors, a plurality of physiological signals collected from a plurality of subjects through a plurality of wearable sensors as training data for training a classification model for classification of cognitive load, wherein each of the plurality of physiological signals are prelabelled as one of (i) a first class or (ii) a second class, wherein the training data is a balanced dataset, and wherein the plurality of physiological signals comprises (i) Galvanic skin response, (ii) Heart rate, (iii) RR interval and (iv) skin temperature;
extracting, via the one or more hardware processors, (i) a plurality of domain specific features and (ii) a plurality of signal property based generic features as a plurality of features from the plurality of physiological signals using a multi-level approach, wherein the multi-level approach comprises:
extracting time domain features, short term Fourier transform (STFF) based features, and Discrete wavelet transform (DWT) based features in a first level,
extracting spectral, statistical, peak-trough features in a second level based on feature extracted in the first level, and
computing ratios and derivatives in a third level, from the features extracted in the second level;

selecting, via the one or more hardware processors, a set of optimal features from the plurality of features using a maximal information coefficient algorithm and a minimum redundancy maximum relevance algorithm;
augmenting, via the one or more hardware processors, the training data by artificially inducing class imbalance between the first class and the second class, wherein the class imbalance is induced by taking all instances of the first class with half of instances of the second class;
applying, via the one or more hardware processors, a synthetic minority over-sampling technique on the augmented training data to generate a set of synthetic data;
training, via the one or more hardware processors, the classification model using (i) the training data and (ii) the set of synthetic data to classify the cognitive load as one of (i) a low load or (ii) a high load;
obtaining, via a wearable device, a set of physiological signals from a subject for classifying in real time the cognitive load of the subject;
obtaining, via the one or more hardware processors, the set of optimal features from the set of physiological signals of the subject;
classifying, via the one or more hardware processors, the cognitive load using the trained classification model as one of (i) the low load or (ii) the high load; and
monitoring, in real time via the one or more hardware processors, the cognitive load of the subject based on the classification of the cognitive load using the trained classification model to analyze the cognitive load of the subject in real world scenarios including analyzing the cognitive load of the subject during one of an interview, an online meeting, a workshop, and a tutorial.

2. The method of claim 1 , wherein generating the synthetic data comprises,
obtaining, via the one or more hardware processors, a first set of physiological signals amongst the plurality of physiological signals prelabelled as the first class;
obtaining, via the one or more hardware processors, a half of a second set of physiological signals amongst the plurality of physiological signals prelabelled as the second class; and
applying, via the one or more hardware processors, the synthetic minority over-sampling technique over the first set of physiological signals and the second set of physiological signals to generate the synthetic data.

3. The method of claim 1 , wherein the set of optimal features are one or more of (i) a set of discrete wavelet transform (DWT) features and (ii) a set of short term Fourier transform features.

4. The method of claim 1 , wherein the set of discrete wavelet transform (DWT) features comprises variance, kurtosis and skewness, mean, difference of mean, box-pierce statistics, hurst exponent, difference of skewness, kurtosis of windowed DWT, zero crossing, and standard deviation of windowed DWT.

5. The method of claim 1 , wherein the set of short term Fourier transform features comprises kurtosis of fast Fourier transform (FFT) coefficient, difference of skewness of windowed FFT, difference of kurtosis of windowed FFT, mean, spectral centroid, linear prediction, total harmonic distortion, hurst exponent, and linear prediction.

6. A system, comprising:
a memory storing instructions;
one or more communication interfaces; and
one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
provide a plurality of physiological signals collected from a plurality of subjects through a plurality of wearable sensors as training data for training a classification model for classification of cognitive load, wherein each of the plurality of physiological signals are prelabelled as one of (i) a first class or (ii) a second class, wherein the training data is a balanced dataset, and wherein the plurality of physiological signals comprises (i) Galvanic skin response, (ii) Heart rate, (iii) RR interval and (iv) skin temperature;
extract (i) a plurality of domain specific features and (ii) a plurality of signal property based generic features as a plurality of features from the plurality of physiological signals using a multi-level approach, wherein to use the multi-level approach, the one or more hardware processors are configured by the instructions to:
extract time domain features, short term Fourier transform (STFF) based features, and Discrete wavelet transform (DWT) based features in a first level,
extract spectral, statistical, peak-trough features in a second level based on feature extracted in the first level, and
compute ratios and derivatives in a third level, from the features extracted in the second level;

select a set of optimal features from the plurality of features using a maximal information coefficient algorithm and a minimum redundancy maximum relevance algorithm;
augment the training data by artificially inducing class imbalance between the first class and the second class, wherein the class imbalance is induced by taking all instances of the first class with half of instances of the second class;
apply a synthetic minority over-sampling technique on the augmented training data to generate a set of synthetic data;
train the classification model using (i) the training data and (ii) the set of synthetic data to classify the cognitive load as one of (i) a low load or (ii) a high load;
obtain a set of physiological signals from a subject for classifying in real time the cognitive load of the subject;
obtain the set of optimal features from the set of physiological signals of the subject;
classify the cognitive load using the trained classification model as one of (i) the low load or (ii) the high load; and
monitor, in real time, the cognitive load of the subject based on the classification of the cognitive load using the trained classification model to analyze the cognitive load of the subject in real world scenarios including analyzing the cognitive load of the subject during one of an interview, an online meeting, a workshop, and a tutorial.

7. The system of claim 6 , wherein generating the synthetic data comprises,
obtaining, via the one or more hardware processors, a first set of physiological signals amongst the plurality of physiological signals prelabelled as the first class;
obtaining, via the one or more hardware processors, a half of a second set of physiological signals amongst the plurality of ph

Assignees

• Tata Consultancy Services Ltd

Inventors

• Chatterjee Debatri
• Jaiswal Dibyanshu
• Ramakrishnan Ramesh Kumar
• Gavas Rahul Dasharath
• Pal Arpan

Classifications

• G06N20/00

  Machine learning · CPC title

• G16H40/67

  for remote operation · CPC title

• A61B5/7267

  involving training the classification device · CPC title

• A61B5/7264

  Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems · CPC title

• A61B5/165

  Evaluating the state of mind, e.g. depression, anxiety · CPC title