Activity analysis, fall detection and risk assessment systems and methods

Patent metadata

Field	Value
Publication number	US-9408561-B2
Application number	US-201313871816-A
Country	US
Kind code	B2
Filing date	Apr 26, 2013
Priority date	Apr 27, 2012
Publication date	Aug 9, 2016
Grant date	Aug 9, 2016

Abstract

Aspects of the present disclosure include methods and corresponding systems for performing health risk assessments for a patient in the home environment. In various aspects, depth image data for a person may be obtained and subsequently processed to generate one or more parameters, such as temporal and spatial gait parameters. Subsequently, the generated parameters may be processed with other medical information related to the patient, such as electronic health records, to perform various health risk assessments.

First claim

What is claimed is:

1. A method comprising:
receiving, by at least one processor, depth image data from at least one depth camera, wherein the depth image data comprises a plurality of frames that depict a person walking through a home environment over time, the frames comprising a plurality of pixels;
performing, by the at least one processor, segmentation on the pixels of the frames;
in response to the segmentation, (1) generating, by the at least one processor, a three-dimensional (3D) data object based on the depth image data, and (2) tracking, by the at least one processor, the 3D data object over a plurality of frames of the depth image data, wherein the tracked 3D data object comprises time-indexed spatial data that represents the person walking through the home environment over time;
identifying, by the at least one processor, a walking sequence from the tracked 3D data object, wherein the identifying step comprises:
the at least one processor determining a speed for the tracked 3D data object over a time frame;
the at least one processor comparing the determined speed with a speed threshold;
in response to the comparison indicating that the determined speed is greater than the speed threshold, the at least one processor assigning a state indicative of walking to the tracked 3D data object;
while the tracked 3D data object is in the assigned walking state:
the at least one processor determining a walk straightness for the tracked 3D data object;
the at least one processor determining a walk length for the tracked 3D data object;
the at least one processor determining a walk duration for the tracked 3D data object;
the at least one processor saving the tracked 3D data object in memory as the identified walking sequence when (i) the determined walk straightness exceeds a straightness threshold, (ii) the determined walk length exceeds a walk length threshold, and (iii) the determined walk duration exceed a walk duration threshold;
the at least one processor excluding from the identified walking sequence in the memory the time-indexed spatial data from the tracked 3D data object corresponding to a time period where the determined walk straightness is less than the walk straightness threshold;
the at least one processor repeating the speed determining step and the comparing step for the tracked 3D data object while the tracked 3D data object is in the assigned walking state; and
the at least one processor assigning a state indicative of not walking to the tracked 3D data object in response to a determination that the speed of the tracked 3D data object in the walking state has fallen below the speed threshold;

analyzing, by the at least one processor, the time-indexed spatial data from the identified walking sequence to generate one or more gait parameters; and
performing, by the at least one processor, at least one health risk assessment based on the one or more gait parameters to determine a health risk assessment score for the person.

2. The method of claim 1 , wherein generating the 3D data object comprises:
for each frame within the plurality of frames, the at least one processor (1) segmenting pixels within that frame into background pixels and foreground pixels according to at least one background model, (2) converting the
foreground pixels into a set of three-dimensional 3D points, (3) segmenting the set of 3D points to generate
at least one 3D object for tracking, the generated 3D data object being representative of an object within the space and comprising a plurality of 3D points that define a spatial position of the represented object, (4) maintaining a list that identifies each 3D data object for tracking, (5) comparing each generated 3D data object with each tracked 3D data object on the list, (6) for each generated 3D data object that matches a tracked 3D data object on the list, updating the matching tracked 3D data object based on the matching generated 3D data object, wherein the updated tracked 3D data object comprises a plurality of 3) points that define the spatial position of the represented object over time, and (7) for each generated 3D data object that does not match any tracked 3D data object on the list, adding the non-matching 3D data object to the list as a new 3D data object for tracking.

3. The method of claim 1 , wherein the depth image data comprises three-dimensional motion tracking data captured at thirty frames per second.

4. The method of claim 1 , wherein performing the health risk assessment comprises mapping the one or more parameters to a standard clinical measure to generate the health risk assessment score.

5. The method of claim 1 , wherein the at least one depth image camera comprises a Microsoft Kinect.

6. The method of claim 1 , wherein the one or more gait parameters comprises at least one of walking speed, stride time, and stride length.

7. A system comprising:
a memory configured to (1) store a model of walk characteristics data for a person of interest and (2) store a plurality of walk sequence data sets in association with the person of interest; and
at least one processor for cooperation with the memory, the at least one processor configured to receive and process the depth image data to populate the memory with the walk sequence data sets associated with the person of interest;
wherein the at least one processor is further configured to:
receive depth image data from at least one depth camera, wherein the depth image data comprises a plurality of frames that depict a space over time, the frames comprising a plurality of pixels;
process the pixels within the frames to generate and track a plurality of three-dimensional (3D) data objects that represent a plurality of objects that are moving within the space over time, each tracked 3D data object comprising a plurality of 3D points that define a spatial position of its represented object over time;
process the 3D points of each tracked 3D data object to make a plurality of determinations as whether any tracked 3D data object is indicative of a person walking;
identify a plurality of walking sequences in response to the walking determinations, each identified walking sequence corresponding to a tracked 3D data object;
analyze the 3D points of the tracked 3D data objects corresponding to the identified walking sequences to generate data indicative of a plurality of walk characteristics for the identified walking sequences;
save a walking sequence data set for each identified walking sequence, each walking sequence data set comprising the walk characteristics data for its corresponding walking sequence;
cluster each saved walk sequence data set and compare the clustered walk sequence data sets with the stored model;
based on the comparison, determine whether any of the clustered walk sequence data sets are attributable to the person of interest;
in response to a determination that a clustered walk sequence data set is attributable to the person of interest, store that walk sequence data set in the memory in association with the person of interest;
perform at least one health risk assessment based on at least one of the stored walk sequence data sets associated with the person of interest; and
communicate a result of the health risk assessment for display.

8. The system of claim of claim 7 , wherein, as part of the pixel processing operation, the at least one processor is further configured to:
for each frame within the plurality of frames, (1) segment pixels within that frame into background pixels and foreground pixels according to at least one background model, (2) convert the
foreground pixels into a set of three-dimensional (3D) points, (3) segment the set of 3D points to generate
at least one 3D data object f

Assignees

• Univ Missouri

Inventors

• Stone Erik Edward
• Skubic Marjorie

Classifications

• A61B5/112 Primary

  Gait analysis · CPC title

• A61B5/004

  adapted for image acquisition of a particular organ or body part (A61B5/0082 takes precedence; arrangements for optical scanning A61B5/0062) · CPC title

• A61B5/0013

  Medical image data (A61B1/00011, A61B6/56, A61B8/56 take precedence) · CPC title