Processing a video for respiration rate estimation

Patent metadata

Field	Value
Publication number	US-9301710-B2
Application number	US-201213529648-A
Country	US
Kind code	B2
Filing date	Jun 21, 2012
Priority date	Jun 1, 2012
Publication date	Apr 5, 2016
Grant date	Apr 5, 2016

Abstract

What is disclosed is a system and method for estimating a respiration rate by analyzing distortions in reflections of structured illumination patterns captured in a video containing a view of a subject's thoracic region. In one embodiment, a video of a target region of a body of a subject of interest is received. Video image frames are processed to estimate 3D time-series data for the target region. As more fully disclosed herein, the subject's respiration rate is estimated from the 3D time-series data. Measurements can be acquired under a diverse set of lighting conditions. The teachings hereof provide a non-contact approach to patient respiratory function monitoring that is useful for intensive care units and for monitoring at homes, and which aid in the detection of sudden deterioration of physiological conditions due to changes in respiration rates. The teachings hereof provide an effective tool for non-contact respiratory function study and analysis.

First claim

What is claimed is:

1. A method for estimating respiration rate of a subject of interest being monitored for respiratory function in a non-contact, remote sensing environment, the method comprising:
receiving a video of a target region of a body of a subject of interest being monitored for respiratory function, wherein said video is captured using a video camera and an illuminator configured to project a pattern of structured illumination, said video camera being sensitive to electromagnetic radiation in a wavelength of said structured illumination, each image of said captured video comprising a sampling of radiation emitted by a reflection of said structured illumination off a surface of said target region, a spatial distortion being introduced by a reflection of said projected pattern off said surface;
processing said video images to reconstruct 3D depth information of the scene and estimate 3D time-series data from said 3D depth information for said target region; and
estimating said subject's respiration rate from said 3D time-series data comprising:
for each image of said video:
comparing spatial attributes of said spatial distortion to known spatial attributes of undistorted projected patterns such that said distortion can be characterized in said image;
calculating a depth map from said characterized distortion at different locations on said surface of said target region;
estimating a 3D volume from said depth map; and
concatenating said resulting estimated 3D volumes to obtain said estimated 3D time-series data.

2. The method of claim 1 , wherein said video camera comprises one of: a video camera that is sensitive in a visible wavelength range, and an IR video camera that is sensitive in an infrared wavelength range.

3. The method of claim 2 , wherein said infrared wavelength range comprises any of: near infrared (NIR), short range infrared red (SWIR), mid-range infrared (MWIR), and long wave infrared (LWIR).

4. The method of claim 1 , wherein said target region comprises one of: said subject's anterior thoracic region, a region of said subject's dorsal body, and a side view containing said subject's thoracic region.

5. The method of claim 4 , further comprising:
de-trending said obtained 3D time-series data to remove low frequency variations from said data;
performing a Fast Fourier Transform on said de-trended time-series data; and
performing automatic peak detection to extract said respiration rate.

6. The method of claim 1 , further comprising monitoring incremental changes in said respiration rate for an occurrence of any of: PUHD Type I and PUHD Type II.

7. The method of claim 1 , wherein said video is captured using multiple video cameras with non-zero disparity among them with respect to the scene to form stereo vision of the scene, said video cameras being sensitive to the same electromagnetic radiation in a wavelength of interest.

8. The method of claim 1 , further comprising monitoring said respiration signal for detecting sudden infant death.

9. The method of claim 1 , further comprising monitoring said respiration signal for detecting respiratory function in sleep apnea patients.

10. A system for estimating respiration rate of a subject of interest being monitored for respiratory function in a non-contact, remote sensing environment, the system comprising:
an illuminator configured to project a pattern of structured illumination;
a video camera for capturing a video of a target region of a subject of interest being monitored for respiratory function, said video camera being sensitive to at least the electromagnetic radiation in a wavelength of said structured illumination, wherein said video is captured using a video camera and an illuminator configured to project a pattern of structured illumination, said video camera being sensitive to electromagnetic radiation in a wavelength of said structured illumination; and
a processor in communication with said video camera and a display device, said processor executing machine readable program instructions for performing:
receiving video captured by said video camera, each image of said video comprising a sampling of radiation emitted by a reflection of said structured illumination off a surface of said target region, a spatial distortion being introduced by a reflection of said projected pattern off said surface;
processing said video to estimate 3D time-series data for said target region;

estimating a respiration rate from said 3D time-series data comprising:
for each image of said video:
comparing spatial attributes of said spatial distortion to known spatial attributes of undistorted projected patterns such that said distortion can be characterized in said image;
calculating a depth map from said characterized distortion at different locations on said surface of said target region;
estimating a 3D volume from said depth map; and
concatenating said resulting estimated 3D volumes to obtain said estimated 3D time-series data; and
communicating said estimated respiration rate to said display device.

11. The system of claim 10 , wherein said video camera comprises one of: a video camera that is sensitive in a visible wavelength range, and an IR video camera that is sensitive in an infrared wavelength range.

12. The system of claim 11 , wherein said infrared wavelength range comprises any of: near infrared (NIR), short range infrared red (SWIR), mid-range infrared (MWIR), and long wave infrared (LWIR).

13. The system of claim 10 , wherein said target region comprises one of: said subject's anterior thoracic region, a region of said subject's dorsal body, and a side view containing said subject's thoracic region.

14. The system of claim 13 , further comprising:
de-trending said obtained 3D time-series data to remove low frequency variations from said data;
performing a Fast Fourier Transform on said de-trended time-series data; and
performing automatic peak detection to extract said respiration rate.

15. The system of claim 10 , further comprising monitoring incremental changes in said respiration rate for an occurrence of any of: PUHD Type I and PUHD Type II.

16. The system of claim 10 , wherein said video is captured using multiple video cameras with non-zero disparity among them with respect to the scene to form stereo vision of the scene, said video cameras being sensitive to the same electromagnetic radiation in a wavelength of interest.

17. The system of claim 10 , further comprising monitoring said respiration signal for detecting sudden infant death.

18. The system of claim 10 , further comprising monitoring said respiration signal for detecting respiratory function in sleep apnea patients.

19. A computer implemented method for estimating respiration rate of a subject of interest being monitored for respiratory function in a non-contact, remote sensing environment, the method comprising:
receiving a video of a target region of a body of a subject of interest being monitored for respiratory function, said video being captured using a video camera and an illuminator configured to project a pattern of structured illumination, said video camera being sensitive to electromagnetic radiation in a wavelength of said structured illumination, each image of said captured video comprising a sampling of radiation emitted by a reflection of said structured illumination off a surface of said target region, a spatial distortion being introduced by a reflection of said projected pattern off said surface;
processing said video images to estimate 3D time

Assignees

• Mestha Lalit Keshav
• Bernal Edgar A
• Xu Beilei
• Xerox Corp

Inventors

• Mestha Lalit Keshav
• Bernal Edgar A
• Xu Beilei

Classifications

• G06F2218/10

  by analysing the shape of a waveform, e.g. extracting parameters relating to peaks · CPC title

• A61B5/0064

  Body surface scanning · CPC title

• G06T2207/10048

  Infrared image · CPC title

• G09G2380/08

  Biomedical applications · CPC title

• G06T2207/10024

  Color image · CPC title