System and method for health assessment on smartphones

Patent metadata

Field	Value
Publication number	US-11317870-B1
Application number	US-201916267376-A
Country	US
Kind code	B1
Filing date	Feb 4, 2019
Priority date	Sep 13, 2017
Publication date	May 3, 2022
Grant date	May 3, 2022

Abstract

Described is a system for health assessment. The system is implemented on a mobile device having at least one of an accelerometer, a geographic location sensor, and a camera. In operation, the system obtains sensor data related to an operator of the mobile device from one of the sensors. A network of networks (NoN) is generated based on the sensor data, the NoN having a plurality of layers with linked nodes. Tuples are thereafter generated. Each tuple contains a node from each layer that optimizes importance, diversity, and coherence. Storylines are created based on the tuples that solves a longest path problem for each tuple. The storylines track multiple symptom progressions of the operator. Finally, a disease prediction of the operator is provided based on the storylines.

First claim

What is claimed is:

1. A system for health assessment, the system comprising:
a mobile device having one or more sensors, including at least one of an accelerometer, a geographic location sensor, and a camera;
wherein the mobile device includes at least one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of:
obtaining sensor data related to an operator of the mobile device from the one or more sensors;
generating a network of networks (NoN) based on the sensor data, the NoN having a plurality of layers with linked nodes;
generating tuples, where each tuple contains a node from each layer that optimizes importance, diversity, and coherence;
generating storylines based on the tuples that solves a longest path problem for each tuple, the storylines tracking multiple symptom progressions of the operator; and
generating a disease prediction of the operator based on the storylines.

2. The system as set forth in claim 1 , wherein the plurality of layers includes a context layer, a predictor layer, and an activity layer, the context layer representing features within the sensor data, the activity layer representing detected activities of the operator based on the sensor data, and the predictor layer representing domain knowledge regarding at least one disease.

3. The system as set forth in claim 2 , wherein each node within the context layer is a feature value of the operator, and where pairs of nodes are linked according to their similarity such that a link between feature nodes indicates that feature measurements corresponds to the operator taken at a common time stamp.

4. The system as set forth in claim 2 , wherein each node within the activity layer is an activity classification of the operator, and where pairs of nodes are linked according to their similarity such that a link between activity nodes indicates that activity classification corresponds to the operator taken at a common time stamp.

5. The system as set forth in claim 2 , wherein each node within the predictor layer is a disease classification based on domain knowledge.

6. The system as set forth in claim 1 , wherein the storylines are temporal motifs, where each temporal motif is a subgraph of the NoN that comprises nodes that are linked across different tuples and nodes that are linked across a temporal dimension.

7. A computer program product for health assessment, the computer program product comprising:
a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of:
obtaining sensor data related to an operator of a mobile device from one or more sensors embedded in the mobile device;
generating a network of networks (NoN) based on the sensor data, the NoN having a plurality of layers with linked nodes;
generating tuples, where each tuple contains a node from each layer that optimizes importance, diversity, and coherence;
generating storylines based on the tuples that solves a longest path problem for each tuple, the storylines tracking multiple symptom progressions of the operator; and
generating a disease prediction of the operator of the mobile device based on the storylines.

8. The computer program product as set forth in claim 7 , wherein the plurality of layers includes a context layer, a predictor layer, and an activity layer, the context layer representing features within the sensor data, the activity layer representing detected activities of the operator based on the sensor data, and the predictor layer representing domain knowledge regarding at least one disease.

9. The computer program product as set forth in claim 8 , wherein each node within the context layer is a feature value of the operator, and where pairs of nodes are linked according to their similarity such that a link between feature nodes indicates that feature measurements corresponds to the operator taken at a common time stamp.

10. The computer program product as set forth in claim 8 , wherein each node within the activity layer is an activity classification of the operator, and where pairs of nodes are linked according to their similarity such that a link between activity nodes indicates that activity classification corresponds to the operator taken at a common time stamp.

11. The computer program product as set forth in claim 8 , wherein each node within the predictor layer is a disease classification based on domain knowledge.

12. The computer program product as set forth in claim 7 , wherein the storylines are temporal motifs, where each temporal motif is a subgraph of the NoN that comprises nodes that are linked across different tuples and nodes that are linked across a temporal dimension.

13. A computer implemented method for health assessment, the method comprising an act of:
causing one or more processers to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
obtaining sensor data related to an operator of a mobile device from one or more sensors embedded in the mobile device;
generating a network of networks (NoN) based on the sensor data, the NoN having a plurality of layers with linked nodes;
generating tuples, where each tuple contains a node from each layer that optimizes importance, diversity, and coherence;
generating storylines based on the tuples that solves a longest path problem for each tuple, the storylines tracking multiple symptom progressions of the operator; and
generating a disease prediction of the operator of the mobile device based on the storylines.

14. The method as set forth in claim 13 , wherein the plurality of layers includes a context layer, a predictor layer, and an activity layer, the context layer representing features within the sensor data, the activity layer representing detected activities of the operator based on the sensor data, and the predictor layer representing domain knowledge regarding at least one disease.

15. The method as set forth in claim 14 , wherein each node within the context layer is a feature value of the operator, and where pairs of nodes are linked according to their similarity such that a link between feature nodes indicates that feature measurements corresponds to the operator taken at a common time stamp.

16. The method as set forth in claim 14 , wherein each node within the activity layer is an activity classification of the operator, and where pairs of nodes are linked according to their similarity such that a link between activity nodes indicates that activity classification corresponds to the operator taken at a common time stamp.

17. The method as set forth in claim 14 , wherein each node within the predictor layer is a disease classification based on domain knowledge.

18. The method as set forth in claim 13 , wherein the storylines are temporal motifs, where each temporal motif is a subgraph of the NoN that comprises nodes that are linked across different tuples and nodes that are linked across a temporal dimension.

Assignees

• Hrl Lab Llc

Inventors

• De Sapio Vincent
• Choe Jaehoon
• Mohammadrezazadeh Iman
• Ni Kang-Yu
• Hoffmann Heiko
• Martin Charles E
• Owechko Yuri

Classifications

• G16H70/60

  relating to pathologies · CPC title

• G16H50/20

  for computer-aided diagnosis, e.g. based on medical expert systems · CPC title

• A61B2562/0219

  Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches · CPC title

• A61B5/7275

  Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor · CPC title

• A61B5/7267

  involving training the classification device · CPC title