Personalized intelligent wake-up system and method based on multimodal deep neural network

Patent metadata

Field	Value
Publication number	US-10083397-B2
Application number	US-201615247665-A
Country	US
Kind code	B2
Filing date	Aug 25, 2016
Priority date	Aug 25, 2016
Publication date	Sep 25, 2018
Grant date	Sep 25, 2018

Abstract

A method for personalized intelligent wake-up system based on multimodal deep neural network comprises monitoring a sleeping status of a user; obtaining a current sleeping-stage of the user within a current time frame and a prediction of a next sleeping-stage of the user for a next time frame; correcting the current sleeping-stage of the user through combining the current sleeping-stage and the prediction of the next sleeping-stage; determining a wake up strategy for the current time frame; determining a relationship between each of a plurality of alarm impulses adopted to wake up the user and a corresponding reaction of the user; identifying a change in the current sleeping-stage for the current time frame; determining an alarm impulse to be triggered for waking up the user; and triggering the determined alarm impulse.

First claim

What is claimed is:

1. A method for a personalized intelligent wake-up system based on multimodal deep neural network, comprising:
monitoring a sleeping status of a user;
obtaining a current sleeping-stage of the user within a current time frame and a prediction of a next sleeping-stage of the user for a next time frame;
correcting the current sleeping-stage of the user through combining the current sleeping-stage and the prediction of the next sleeping-stage;
based on the current sleeping-stage of the user, prior knowledge learnt from sleep-related research studies, and at least one user preference of waking up, determining a wake up strategy for the current time frame;
determining a relationship between each of a plurality of alarm impulses adopted to wake up the user and a corresponding reaction of the user;
identifying a change in the current sleeping-stage for the current time frame;
based on the wake-up strategy established for the current time frame and the relationship between each of the plurality of alarm impulses and the reaction of the user, determining an alarm impulse to be triggered for waking up the user; and
triggering the determined alarm impulse.

2. The method for a personalized intelligent wake-up system based on multimodal deep neural network according to claim 1 , wherein obtaining a current sleeping-stage of a user within a current time frame further includes:
receiving sensor date provided by a sensor capable of sensing body movements of the user during a sleep; and
obtaining the current sleeping-stage of the user according to the sensor date.

3. The method for a personalized intelligent wake-up system based on multimodal deep neural network according to claim 2 , wherein obtaining a prediction of a next sleeping-stage of the user for a next time frame further includes:
receiving historical data of a plurality of users' sleeping status and personal data of the user sleeping status; and
predicting the next sleeping-stage of the user based on a pre-trained sleeping-stage prediction according to the historical data and a personalized sleeping-stage prediction according to the personal data.

4. The method for a personalized intelligent wake-up system based on multimodal deep neural network according to claim 3 , wherein:
predicting the next sleeping-stage of the user based on a recursive neural network (RNN) model.

5. The method for a personalized intelligent wake-up system based on multimodal deep neural network according to claim 4 , wherein correcting the current sleeping-stage of the user through combining the current sleeping-stage and the prediction of the next sleeping-stage further includes:
correcting the current sleeping-stage of the user to obtain a corrected current sleeping-stage of the user through combing the sensor data, the pre-trained sleeping-stage prediction, and the personalized sleeping-stage prediction.

6. A method for a personalized intelligent wake-up system based on multimodal deep neural network, comprising:
monitoring a sleeping status of a user;
obtaining a current sleeping-stage of the user within a current time frame, comprising:
receiving sensor date provided by a sensor capable of sensing body movements of the user during a sleep; and
obtaining the current sleeping-stage of the user according to the sensor date;

obtaining a prediction of a next sleeping-stage of the user for a next time frame, comprising:
receiving historical data of a plurality of users' sleeping status and personal data of the user sleeping status; and
predicting the next sleeping-stage of the user based on a pre-trained sleeping-stage prediction according to the historical data and a personalized sleeping-stage prediction according to the personal data;

correcting the current sleeping-stage of the user through combining the current sleeping-stage and the prediction of the next sleeping-stage, comprising:
correcting the current sleeping-stage of the user to obtain a corrected current sleeping-stage of the user through combing the sensor data, the pre-trained sleeping-stage prediction, and the personalized sleeping-stage prediction;

based on the current sleeping-stage of the user, prior knowledge learnt from sleep-related research studies, and at least one user preference of waking up, determining a wake up strategy for the current time frame;
determining a relationship between each of a plurality of alarm impulses adopted to wake up the user and a corresponding reaction of the user;
identifying a change in the current sleeping-stage for the current time frame;
based on the wake-up strategy established for the current time frame and the relationship between each of the plurality of alarm impulses and the reaction of the user, determining an alarm impulse to be triggered for waking up the user; and
triggering the determined alarm impulse;
wherein, the corrected current sleeping-stage of the user is calculated as:
S=αS sensor +(1−α) S scaled

where S denotes the corrected current sleeping-stage of the user, S sensor denotes the sensor data, S scaled denotes a scaled prediction of the user's sleep-stage, α denotes a weight.

7. The method for a personalized intelligent wake-up system based on multimodal deep neural network according to claim 6 , wherein:
the weight α is calculated as

α
=

S
wake

-

S

p
′

2
⁢

S
wake

-

S

p
′

-

S

s
′

;

and
the S scaled is calculated as

S
scaled

=

S

w
′

S
wake

⁢

S
predict

,

where S p′ denotes a last prediction of the user's sleeping-stage from the RNN model, S s′ denotes a last inference of the user's sleeping-stage from the sensor data, S wake denotes a wake-up point stage of the user, S w′ denotes a stage prediction of the wake-up point stage from a last available time frame, and S predict denotes a current prediction of the user's sleep-stage outputted from the RNN model.

8. The method for a personalized intelligent wake-up system based on multimodal dee

Assignees

• Tcl Res America Inc

Inventors

• Yuan Jianbo
• Wang Haohong
• Ren Xiaobo

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N5/02 Primary

  Knowledge representation; Symbolic representation · CPC title

• G08B3/10

  using electric transmission; using electromagnetic transmission · CPC title

• G06N3/08

  Learning methods · CPC title