Systems and methods for computer monitoring of remote photoplethysmography based on chromaticity in a converted color space

Patent metadata

Field	Value
Publication number	US-11229372-B2
Application number	US-201716335480-A
Country	US
Kind code	B2
Filing date	Sep 21, 2017
Priority date	Sep 21, 2016
Publication date	Jan 25, 2022
Grant date	Jan 25, 2022

Abstract

Provided herein are systems and methods for computer monitoring of remote photoplethysmography (rPPG) from camera images based on chromaticity in a converted color space, which reduces motion-induced artifacts in camera images for improved rPPG computer monitoring of physiological parameters. In particular, a rPPG system for monitoring at least one physiological parameter from image data is disclosed herein. A processor subsystem electronically receives a first image data set representative of a series of consecutive images of at least a portion of a living body. The processor subsystem converts the first image data set from a first color space to a second color space to generate a second image data set including first channel data comprising a luminance component and second channel data comprising a chromatic component. The processor subsystem processes the second channel data to monitor the at least one physiological parameter of the living body.

First claim

What is claimed is:

1. A remote photoplethysmography (rPPG) system for monitoring by a computer system of at least one physiological parameter of a living body from image data, the rPPG system comprising a processor subsystem to:
electronically receive, at the computer system, a first image data set from an optical imaging element, the first image data set being representative of a series of consecutive images of at least a portion of the living body;
convert, by the computer system, the first image data set from a first color space to a second color space to generate a second image data set including a* channel data comprising a chromatic component corresponding to a chromaticity value defined between a green chromatic value and a red chromatic value, wherein the second color space is a perceptually uniform color space such that the chromatic component of the a* channel data is independent of an illumination of the series of consecutive images;
process, by the computer system, the a* channel data to monitor at least one physiological parameter of the living body;
automatically determine, by the computer system, a region of interest (ROI) of the living body in the first image data set and tracking, by the computer system, the ROI through at least some consecutive images of the series of consecutive images;
initialize, by the computer system, the ROI based on a first image of the series of consecutive images; and
reinitialize, by the computer system, the ROI after a predetermined number of images of the series of consecutive images.

2. The rPPG system of claim 1 , wherein the first color space comprises an RGB color space.

3. The rPPG system of claim 1 , wherein the second color space comprises:
a CIELab color space, wherein the a* channel data is descriptive of a variation in color of the living body over a period of time between a green chromatic value and a red chromatic value.

4. The rPPG system of claim 1 , wherein the at least one physiological parameter comprises at least one of breathing pattern, respiration rate, or heart rate.

5. The rPPG system of claim 1 , wherein the computer system removes one or more consecutive images of the series of consecutive images if a number of identified feature points in the ROI in the one or more consecutive images is below a predetermined threshold.

6. The rPPG system of claim 1 , wherein the computer system converts the first image data set from the first color space to the second color space by:
converting the first image data set from the first color space to an intermediate color space to generate an intermediate image data set; and
converting the intermediate image data set from the intermediate color space to the second color space to generate the second image data set;
wherein the intermediate color space is an XYZ color space.

7. A method for remote photoplethysmography (rPPG) monitoring by a computer system, the method comprising:
electronically receiving, at the computer system, a first image data set from an optical imaging element, the first image data set being representative of a series of consecutive images of at least a portion of a living body;
converting, by the computer system, the first image data set from a first color space to a second color space to generate a second image data set including a* channel data comprising a chromatic component corresponding to a chromaticity value defined between a green chromatic value and a red chromatic value, wherein the second color space is a perceptually uniform color space such that the chromatic component of the a* channel data is independent of an illumination of the series of consecutive images;
processing, by the computer system, the a* channel data to monitor at least one physiological parameter of the living body;
automatically determining, by the computer system, a region of interest (ROI) of the living body in the first image data set, and tracking, by the computer system, the ROI through at least some consecutive images of the series of consecutive images; and
removing, by the computer system, one or more consecutive images of the series of consecutive images if a number of identified feature points in the ROI in the one or more consecutive images is below a predetermined threshold.

8. The method of claim 7 , wherein the first color space comprises an RGB color space.

9. The method of claim 7 , wherein the second color space comprises:
a CIELab color space, wherein the a* channel data is descriptive of a variation in color of the living body over a period of time between a green chromatic value and a red chromatic value.

10. The method of claim 7 , wherein the at least one physiological parameter comprises at least one of breathing pattern, respiration rate, or heart rate.

11. The method of claim 7 , further comprising automatically determining, by the computer system, a region of interest (ROI) of the living body in the first image data set, and tracking, by the computer system, the ROI through at least some consecutive images of the series of consecutive images.

12. The method of claim 11 , further comprising:
initializing, by the computer system, the ROI based on a first image of the series of consecutive images; and
reinitializing, by the computer system, the ROI after a predetermined number of images of the series of consecutive images.

13. The method of claim 7 , wherein the step of converting the first image data set from the first color space to the second color space by:
converting the first image data set from the first color space to an intermediate color space to generate an intermediate image data set; and
converting the intermediate image data set from the intermediate color space to the second color space to generate the second image data set;
wherein the intermediate color space is an XYZ color space.

14. A non-transitory computer readable medium comprising program instructions for execution by a processor of a computer system to cause the computer system to perform the following steps:
electronically receive, at the computer system, a first image data set from an optical imaging element, the first image data set being representative of a series of consecutive images of at least a portion of a living body;
convert, by the computer system, the first image data set from a first color space to a second color space to generate a second image data set including a* channel data comprising a chromatic component corresponding to a chromaticity value defined between a green chromatic value and a red chromatic value, wherein the second color space is a perceptually uniform color space such that the chromatic component of the a* channel data is independent of an illumination of the series of consecutive images;
process, by the computer system, the a* channel data to monitor at least one physiological parameter of the living body;
automatically determine, by the computer system, a region of interest (ROI) of the living body in the first image data set and tracking, by the computer system, the ROI through at least some consecutive images of the series of consecutive images;
initialize, by the computer system, the ROI based on a first image of the series of consecutive images; and
reinitialize, by the computer system, the ROI after a predetermined number of images of the series of consecutive images.

15. The non-transitory computer readable medium of claim 14 , wherein the first color space comprises an RGB color space.

16. The non-transitory computer readable medium of claim 14 , wherein the second color space comprises:
a CIELab color space, wherein t

Assignees

• Univ Arizona State
• Arizona Board Of Regents On Behalf Of Arizona State Of Univ

Inventors

• Tao Nongjian
• Yang Yuting

Classifications

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• G06T5/40

  using histogram techniques · CPC title

• G06T2207/30076

  Plethysmography · CPC title

• G06T7/0016

  involving temporal comparison · CPC title

• G06T7/00

  Image analysis · CPC title