Determining cardiovascular features using camera-based sensing

Patent metadata

Field	Value
Publication number	US-10939834-B2
Application number	US-201815951105-A
Country	US
Kind code	B2
Filing date	Apr 11, 2018
Priority date	May 1, 2017
Publication date	Mar 9, 2021
Grant date	Mar 9, 2021

Abstract

In one embodiment, a computer-readable non-transitory storage medium embodies software that is operable when executed to, in real time, capture a number of images of a user; and determine a time-series signal for the user based on the plurality of images. The signal includes one or more segments that are physiologically plausible and one or more segments that are physiologically implausible. The software is further operable to identify one or more of the physiologically plausible sub-segments based on one or more pre-defined signal characteristics; and calculate one or more heartrate measurements based on the physiologically plausible sub-segments.

First claim

The invention claimed is:

1. One or more computer-readable non-transitory storage media embodying software that is operable when executed by a client system to, in real-time:
capture, by one or more cameras of the client system, a plurality of images of a user;
determine, by the client system, a time-series signal for the user based on the plurality of images, wherein the signal comprises one or more physiologically plausible sub-segments and one or more physiologically implausible sub-segments;
identify, by the client system, one or more of the physiologically plausible sub-segments and one or more of the physiologically implausible sub-segments based on one or more pre-defined, physiological signal characteristics; and
calculate, by the client system, an average heartrate or heartrate range for the user based on the one or more physiologically plausible sub-segments.

2. The media of claim 1 , wherein the plurality of images is captured by a single camera.

3. The media of claim 1 , wherein the software is further operable to:
determine a baseline signal for the user;
calculate a signal-to-noise ratio (SNR) of the time-series signal based on the baseline signal;
compare the SNR to a threshold SNR; and
identify sub-segments of the signal with a calculated SNR that is higher than the threshold SNR.

4. The media of claim 1 , wherein the software is further operable to:
compare the time-series signal to a pre-determined respiratory profile, wherein the pre-determined respiratory profile comprises characteristics of a pulse volume measurement relative to R wave-to-R wave (RR) intervals during exhalation or during inhalation; and
identify sub-segments of the signal that are consistent with the respiratory profile.

5. The media of claim 1 , wherein the software is further operable to:
perform a Fourier transform of the time-series signal; and
identify sub-segments of the time-series signal based on a measured RR interval being consistent with a dominant frequency of the Fourier transform.

6. The media of claim 1 , wherein the software is further operable to identify sub-segments for which a corresponding systolic portion of the signal is less than a corresponding diastolic portion.

7. The media of claim 1 , wherein the software is further operable to:
extract red, green, and blue channel components of the time-series signal;
compare the red, green, and blue channel components with respective components of a pre-determined RGB profile, wherein the pre-determined RGB profile comprises a pulsatile relationship, total power relationship, or co-variation relationship between the red, green, and blue channel components; and
identify sub-segments of the signal that are consistent with the pre-determined RGB profile.

8. A method executed by a client system comprising, in real-time:
capturing, by one or more cameras of the client system, a plurality of images of a user;
determining, by the client system, a time-series signal for the user based on the plurality of images, wherein the signal comprises one or more physiologically plausible sub-segments and one or more physiologically implausible sub-segments;
identifying, by the client system, one or more of the physiologically plausible sub-segments and one or more of the physiologically implausible sub-segments based on one or more pre-defined, physiological signal characteristics; and
calculating, by the client system, an average heartrate or heartrate range for the user based on the one or more physiologically plausible sub-segments.

9. The method of claim 8 , wherein the plurality of images is captured by a single camera.

10. The method of claim 8 , further comprising:
determining a baseline signal for the user;
calculating a signal-to-noise ratio (SNR) of the time-series signal based on the baseline signal;
comparing the SNR to a threshold SNR; and
identifying sub-segments of the signal with a calculated SNR that is higher than the threshold SNR.

11. The method of claim 8 , further comprising:
comparing the time-series signal to a pre-determined respiratory profile, wherein the pre-determined respiratory profile comprises characteristics of a pulse volume measurement relative to R wave-to-R wave (RR) intervals during exhalation or during inhalation; and
identifying sub-segments of the signal that are consistent with the respiratory profile.

12. The method of claim 8 , further comprising:
performing a Fourier transform of the time-series signal; and
identifying sub-segments of the time-series signal based on a measured RR interval being consistent with a dominant frequency of the Fourier transform.

13. The method of claim 8 , further comprising identifying sub-segments for which a corresponding systolic portion of the signal is less than a corresponding diastolic portion.

14. The method of claim 8 , further comprising:
extracting red, green, and blue channel components of the time-series signal;
comparing the red, green, and blue channel components with respective components of a pre-determined RGB profile, wherein the pre-determined RGB profile comprises a pulsatile relationship, total power relationship, or co-variation relationship between the red, green, and blue channel components; and
identifying sub-segments of the signal that are consistent with the pre-determined RGB profile.

15. A system comprising:
one or more processors; and
a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions by a client system to, in real-time:
capture, by one or more cameras of the client system, a plurality of images of a user;
determine, by the client system, a time-series signal for the user based on the plurality of images, wherein the signal comprises one or more physiologically plausible sub-segments and one or more physiologically implausible sub-segments;
identify, by the client system, one or more of the physiologically plausible sub-segments and one or more of the physiologically implausible sub-segments based on one or more pre-defined, physiological signal characteristics; and
calculate, by the client system, an average heartrate or heartrate range for the user based on the one or more physiologically plausible sub-segments.

16. The system of claim 15 , wherein the processors are further operable to:
determine a baseline signal for the user;
calculate a signal-to-noise ratio (SNR) of the time-series signal based on the baseline signal;
compare the SNR to a threshold SNR; and
identify sub-segments of the signal with a calculated SNR that is higher than the threshold SNR.

17. The system of claim 15 , wherein the processors are further operable to:
compare the time-series signal to a pre-determined respiratory profile, wherein the pre-determined respiratory profile comprises characteristics of a pulse volume measurement relative to R wave-to-R wave (RR) intervals during exhalation or during inhalation; and
identify sub-segments of the signal that are consistent with the respiratory profile.

18. The system of claim 15 , wherein the processors are further operable to:
perform a Fourier transform of the time-series signal; and
identify sub-segments of the time-series signal based on a measured RR interval being consistent with a dominant frequency of the Fourier transform.

19. The system of claim 15 , wherein the processors are further operable to identify sub-segments for which a corresponding systolic portion of the signal is less tha

Assignees

• Samsung Electronics Co Ltd

Inventors

• Khwaja Ayesha
• Young James
• Wortham Cody
• Sadi Sajid
• Jain Jawahar

Classifications

• A61B5/349

  Detecting specific parameters of the electrocardiograph cycle · CPC title

• G06V40/174

  Facial expression recognition · CPC title

• G06V40/15

  Biometric patterns based on physiological signals, e.g. heartbeat, blood flow · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V40/176

  Dynamic expression · CPC title