Medication adherence monitoring system and method

Patent metadata

Field	Value
Publication number	US-9977870-B2
Application number	US-201715618944-A
Country	US
Kind code	B2
Filing date	Jun 9, 2017
Priority date	Jun 11, 2014
Publication date	May 22, 2018
Grant date	May 22, 2018

Abstract

A medication management system is described that is operable to determine whether a user is actually following a protocol, provide additional assistance to a user, starting with instructions, video instructions, and the like, and moving up to contact from a medication administrator if it is determined that the user would need such assistance in any medical adherence situation, including clinical trial settings, home care settings, healthcare administration locations, such as nursing homes, clinics, hospitals and the like. Suspicious activity on the part of a patient or other user of the system is identified and can be noted to a healthcare provider or other service provider where appropriate.

First claim

What is claimed is:

1. A display and image capture apparatus for monitoring medication adherence of a user, comprising:
an image capture device for capturing one or more video sequences associated with at least intake of medication into the user's body;
a computer processor associated with the image capture device arranged to receive the captured one or more video sequences, and to determine, based at least in part on the captured one or more video sequences, whether the user has properly performed a predetermined portion of a predetermined process representative of at least intake of the medication into the user's body;
a presentation device, associated with the image capture device, arranged to present visually to the user one or more instructions encouraging proper performance of the predetermined portion of the predetermined process representative of at least intake of the medication into the user's body if the computer processor has determined that the user has not properly performed the predetermined portion of the predetermined process, and arranged to present visually an acknowledgment of proper performance of the predetermined portion of the predetermined process representative of intake of the medication into the user's body if the computer processor has determined that the user has properly performed the predetermined portion of the predetermined process.

2. The apparatus of claim 1 , wherein the computer processor is operable to determine whether the user has performed the predetermined portions of the predetermined process by determining at least one of: (i) that the user has properly place the pill in the user's mouth, or (ii) that the user has properly displayed the user's empty mouth in the field of view of the image capture device.

3. The apparatus of claim 2 , wherein the computer processor is operable to determine that the user has properly place the pill in the user's mouth by extracting one or more features associated with the pill, determining a confidence score for each of the one or more features representative of the likelihood that the pill captured in the one or more video sequences is a predetermined type of pill, and performing a decision fusion process on all of the determined confidence scores to determine whether the pill is the proper pill.

4. The apparatus of claim 2 , wherein the computer processor is operable to determine, in accordance with a deep learning analysis system for extracting one or more features associated with the pill, that the user has properly place the pill in the user's mouth and to determine whether the pill is the proper pill.

5. The apparatus of claim 2 , wherein the computer processor is operable to determine that the user has properly place the pill in the user's mouth by extracting one or more features associated with the pill, performing a feature fusion process on all of the extracted features associate with the pill, and determining a score in accordance with the one or more extracted features to determine whether the pill is the proper pill.

6. The apparatus of claim 1 , further comprising a communication system for transmitting one or more indications of proper or improper performance of each of the predetermined portions of the predetermined process to a remote location.

7. The apparatus of claim 6 , wherein a computer processor at the remote location is operable to perform additional processing on the transmitted one or more indications of proper or improper performance of each of the predetermined portions of the predetermined process, and to transmit to the display and image capture apparatus one or more results of the additional processing.

8. The apparatus of claim 1 , further comprising a communication system for transmitting the captured one or more video sequences to a remote location.

9. The apparatus of claim 8 arranged to transmit the captured one or more video sequences to the remote location over a series of administrations.

10. The apparatus of claim 9 , wherein the computer processor at the remote location is operable to determine whether the user has properly ingested the medication into the user's body in accordance with the received series of administrations.

11. The apparatus of claim 10 , wherein the computer processor is operable to extract one or more suspicious activity indicators in accordance with the received series of administrations.

12. The apparatus of claim 11 , wherein the apparatus is arranged to report the extracted one or more suspicious activity indicators to a third party.

13. The apparatus of claim 11 , wherein the one or more suspicious activity indicators are selected from a group consisting of: hand movement, mouth movement, cheek shape, swallowing, audio sounds, face movement, and time on task.

14. The apparatus of claim 1 , wherein the intake of the medication into the user's body includes ingestion of the medication orally by the user.

15. The apparatus of claim 1 , wherein the presentation device is further arranged to present audibly to the user one or more instruction encouraging proper performance of the predetermined portion of the predetermined process representative of at least intake of the medication into the user's body in response to the computer processor's determining that the user has not properly performed the predetermined portion of the predetermined process, and arranged to present audibly an acknowledgment of proper performance of the predetermined portion of the predetermined process representative of at least intake of the medication into the user's body in response to the computer processor's determining that the user has properly performed the predetermined portion of the predetermined process.

16. A machine-implemented method for monitoring medication adherence of a user, the method comprising:
receiving, by one or more processors, one or more video sequences captured by an image capture device, the one or more sequences being associated with at least intake of the medication into the user's body;
determining, by the one or more processors, based at least in part on the received one or more video sequences, whether the user has properly performed a predetermined portion of a predetermined process representative of at least the intake of the medication into the user's body;
in response to a determination that the user has not properly performed the predetermined portion of the predetermined process, presenting, to the user one or more instructions on a user interface, the one or more instructions encouraging proper performance of the predetermined portion of the predetermined process representative of at least intake of the medication into the user's body; and
in response to a determination that the user has properly performed the predetermined portion of the predetermined process, presenting, on the user interface, an acknowledgment of proper performance of the predetermined portion of the predetermined process representative of at least intake of the medication into the user's body.

17. The method of claim 16 , wherein determining whether the user has performed the predetermined portions of the predetermined process representative of at least intake of the medication into the user's body comprises determining at least one of: (i) that the user has properly place the pill in the user's mouth, or (ii) that the user has properly displayed the user's empty mouth in the field of view of the image capture device.

18. The method of claim 17 , wherein determining that the user has properly placed the pill in the user's mouth includes extractin

Assignees

• Aic Innovations Group Inc

Inventors

• Hanina Adam
• Kessler Gordon
• Guan Lei

Classifications

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• H04N7/183

  for receiving images from a single remote source · CPC title

• G16H20/10 Primary

  relating to drugs or medications, e.g. for ensuring correct administration to patients · CPC title

• A61B5/1128

  using image analysis (A61B5/1127 takes precedence) · CPC title

• A61B5/0022

  Monitoring a patient using a global network, e.g. telephone networks, internet · CPC title