Activity analysis, fall detection and risk assessment systems and methods

Patent metadata

Field	Value
Publication number	US-9597016-B2
Application number	US-201414169508-A
Country	US
Kind code	B2
Filing date	Jan 31, 2014
Priority date	Apr 27, 2012
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

A method for performing health risk assessments for a patient in a home or medical facility is provided. In various embodiments, the method comprises compiling depth image data from at least one depth camera associated with a particular patient, and generating at least one three-dimensional object based on the depth image data. The method additionally includes identifying a walking sequence from the at least one three-dimensional object, analyzing the walking sequence to generate one or more parameters, and performing at least one health risk assessment based on the one or more parameters to determine a health risk assessment score. The method further comprises sending an alert message to at least one caregiver when the at least one health risk assessment indicates the occurrence of an incident (i.e., the recorded depth image data) denoting that the patient has fallen or that there exists a high risk of the patient falling.

First claim

What is claimed is:

1. A method for determining whether a person has fallen or is at a high risk of falling, and notifying a caregiver when the person has fallen or is at high risk of falling, the method comprising:
acquiring depth image data of a person, as the person walks within a particular area, utilizing at least one depth camera of a risk assessment system, the at least one depth camera located in the particular area;
receiving the depth image data from the at least one depth camera by at least one processor of a computer-based remote device of the risk assessment system, wherein the depth image data comprises a plurality of frames that depict the person walking through a home environment over time, the frames comprising a plurality of pixels, the remote device located remotely from the at least one depth camera, the remote device comprising electronic memory on which an image analysis application is electronically stored, and the at least one processor structured and operable to execute the image analysis application;
segmenting, by the at least one processor, the pixels of the frames of the depth image data; in response to the segmenting, (1) generating, by the at least one processor, a three-dimensional data object based on the analyzed depth image data, and (2) tracking by the at least one processor, the three-dimensional data object over a plurality of frames of the depth images data, wherein the tracked three-dimensional object comprises time-indexed spatial data that represents the person walking through the home environment over time;
identifying, by the at least one processor, a walking sequence from the tracked three-dimensional object, wherein the identifying step comprises:
the at least one processor determining a speed for the tracked three-dimensional data object over a time frame;
the at least one processor comparing the determined speed with a speed threshold;
in response to the comparison indicating that the determined speed is greater than the speed threshold, the at least one processor assigning a state indicative of walking to the tracked three-dimensional data object;
while the tracked three-dimensional data object is in the assigned walking state:
the at least one processor determining a walk straightness for the tracked three-dimensional data object;
the at least one processor determining a walk length for the tracked three-dimensional data object;
the at least one processor determining a walk duration for the tracked three-dimensional data object;
the at least one processor saving the tracked three-dimensional data object in memory as the identified walking sequence when (i) the determined walk straightness exceeds a straightness threshold, (ii) the determined walk length exceeds a walk length threshold, and (iii) the determined walk duration exceeds a walk duration threshold;
the at least one processor excluding from the identified walking sequence in the memory the time-indexed spatial data from the tracked three-dimensional data object corresponding to a time period where the determined walk straightness is less than the walk straightness threshold;
the at least one processor repeating the speed determining step and the comparing step for the tracked three-dimensional data object while the tracked three-dimensional data object is in the assigned walking state; and
the at least one processor assigning a state indicative of not walking to the tracked three-dimensional data object in response to a determination that the speed of the tracked three-dimensional data object in the walking state has fallen below the speed threshold;

analyzing by the at least one processor, the time-indexed spatial data from the identified walking sequence to generate one or more gait parameters;
performing, by the at least one processor, at least one health risk assessment based on the one or more gait parameters to determine a health risk assessment score for the person, the health risk assessment score indicative of a level of risk at which the person is of falling;
analyzing, by the at least one processor, the health risk assessment score to determine whether an alert incident has occurred, the alert incident indicating at least one of:
the person has fallen; and
a high risk of the person falling is present; and
generating and sending, by the at least one processor, an alert message to at least one caregiver electronic device when it is determined that the health risk assessment score indicates an alert incident has occurred.

2. The method of claim 1 , wherein the caregiver electronic device comprises at least one of a smart phone, a laptop computer, a desktop computer, a computer tablet, and a television, and wherein sending the alert message comprises sending the alert message to at least one of:
the smart phone of the caregiver,
the laptop computer of the caregiver,
the desktop computer of the caregiver,
the computer tablet of the caregiver, and
the television of the caregiver.

3. The method of claim 2 , wherein sending the alert message comprises sending the alert message in the form of a visual alert message.

4. The method of claim 2 , wherein sending the alert message comprises sending the alert message in the form of an audible alert message.

5. The method of claim 2 , wherein sending the alert message comprising sending the alert message in the form of a tactile alert message.

6. The method of claim 1 , wherein sending the alert message comprises sending the alert message that includes at least one of: confidence of the risk assessment score information, time of the alert incident occurrence information, location of the alert incident occurrence information, and presence of another person at the time of the alert incident occurrence information.

7. The method of claim 1 , wherein sending the alert message comprises sending the alert message including a hyperlink to video data containing the depth image data of the determined alert incident.

8. The method of claim 7 , wherein the video data can be viewed using media player software and can be rewound, replayed and fast forwarded via the media player software.

9. A risk assessment system for determining whether a subject has fallen or is at a high risk of falling, and notifying a caregiver when the subject has fallen or is at high risk of falling, the risk assessment system comprising:
at least one depth camera located in the particular area in which a subject is present; and
a computer-based remote device communicatively connected with, and located remotely from, the at least one depth camera, the remote device comprising:
at least one electronic storage device having stored thereon an image analysis application and structured and operable to (1) store a model of walk characteristics data for the subject and (2) store a plurality of walk sequence data sets in association with the subject;
a display structured and operable to provide various graphical images; and
at least one processor structured and operable to:
receive and process depth image data received from at least one depth camera to populate the electronic storage device with the walk sequence data sets associated with the subject as the subject walks within the particular area, wherein the depth image data comprises a plurality of frames that depict a space over time, the frames comprising a plurality of pixels;
process the pixels within the frames to generate and track a plurality of three dimensional objects that represent a plurality of objects that are moving within the space over time, each tracked three-dimensional data object comprising a plurality of three-dimensional points that define a spatial position of the respective tracked three-dimensional data object over tim

Assignees

• Univ Missouri

Inventors

• Stone Erik Edward
• Skubic Marjorie
• Rantz Marilyn
• Popescu Mihail

Classifications

• A61B5/1113

  Local tracking of patients, e.g. in a hospital or private home · CPC title

• A61B5/1117

  Fall detection · CPC title

• A61B5/0013

  Medical image data (A61B1/00011, A61B6/56, A61B8/56 take precedence) · CPC title

• A61B5/0022

  Monitoring a patient using a global network, e.g. telephone networks, internet · CPC title

• A61B5/1128 Primary

  using image analysis (A61B5/1127 takes precedence) · CPC title