Identification and de-identification within a video sequence

Patent metadata

Field	Value
Publication number	US-9652665-B2
Application number	US-201614990389-A
Country	US
Kind code	B2
Filing date	Jan 7, 2016
Priority date	Nov 18, 2009
Publication date	May 16, 2017
Grant date	May 16, 2017

Abstract

A method and system for de-identifying a video sequence are provided. The method may include the steps of capturing a video sequence, comprising a number of individual frames, including one or more users performing one or more actions, and using activity recognition to recognize one of the one or more actions. One or more of the plurality of frames may be defined as comprising the recognized one or more actions, and a portion of the one or more of the plurality of frames may be identified to remain visible. The non-identified portions of the one or more of the plurality of frames and the non-defined frames may be de-identified. This method may be applied to the determine of whether a user has ingested a medication pill.

First claim

What is claimed is:

1. A method for de-identifying a video sequence, the method comprising:
capturing, in a video capture device, a video sequence comprising a plurality of individual frames, the plurality of individual frames including images of one or more users administering medication;
designating, by at least one processor, one or more frames of the plurality of individual frames as comprising a recognized administration of medication;
identifying a first portion of the one or more designated frames to remain visible, wherein the first portion comprises the recognized administration of medication;
de-identifying a second portion of the one or more designated frames so that the first portion comprising the administration of medication remains visible;
determining that the first portion of the one or more designated frames comprises a face other than faces of the one or more users administering medication; and
de-identifying, from the first portion of the one or more designated frames, the face other than the face of the one or more users administering medication.

2. The method of claim 1 , wherein de-identifying the second portion and/or de-identifying the face other than the face of the one or more users comprises blurring the second portion and/or the face, respectively.

3. The method of claim 1 , further comprising highlighting the visible first portion.

4. The method of claim 1 , wherein the administration of medication comprises ingestion of the medication.

5. The method of claim 1 , further comprising transmitting the video sequence to a remote storage and computing location.

6. The method of claim 1 , further comprising performing, by the at least one processor, activity recognition on the captured video sequence to recognize the administration of medication within the plurality of individual frames, wherein performing activity recognition to recognize the administration of medication further comprises:
determining, through object recognition, the identity of an object for a first plurality of frames of the video sequence; and
determining, through object recognition, the identity of the object and the object's surroundings in a second plurality of frames of the video sequence.

7. The method of claim 1 , wherein the administration of medication comprises micro movements of one or more body parts.

8. The method of claim 1 , further comprising performing, by the at least one processor, activity recognition on the captured video sequence to recognize the administration of medication within the plurality of individual frames,
wherein performing the activity recognition to recognize the administration of medication comprises:
defining a plurality of detection methods for recognizing the administration of medication; and
applying each detection method of the plurality of detection methods to a subset of the plurality of individual frames of the captured video sequence.

9. The method of claim 8 , wherein each detection method of the plurality of detection methods comprises searching for an object positioned at a different orientation.

10. A method for de-identifying a video sequence of a user ingesting a medication pill, comprising:
capturing a first video sequence in a video capture device, the first video sequence comprising a plurality of individual frames, the plurality of individual frames including images of one or more users performing the action of putting the medication pill in their mouth;
designating, by at least one processor, one or more frames of the plurality of individual frames as comprising a recognized action of placing the medication pill in the mouth of the user;
identifying a first portion of the one or more designated frames to remain visible, wherein the first portion comprises the recognized action of placing the medication pill in the mouth of the user;
de-identifying a second portion of the one or more designated frames so that the first portion comprising the recognized action of placing the medication pill in the mouth of the user remains visible;
determining that the first portion of the one or more designated frames comprises a face other than faces of the one or more users performing the action of putting the medication pill in their mouth; and
de-identifying, from the first portion of the one or more designated frames, the face other than faces of the one or more users performing the action of putting the medication pill in their mouth.

11. The method of claim 10 , wherein de-identifying the second portion and/or de-identifying the face comprises blurring the second portion and/or the face, respectively.

12. The method of claim 10 , further comprising highlighting the visible first portion.

13. The method of claim 10 , further comprising transmitting the video sequence to a remote storage and computing location.

14. The method of claim 10 , further comprising performing, by the at least one processor, activity recognition on the captured first video sequence to recognize the placement of the medication pill in the mouth of the user,
wherein performing activity recognition further comprises:
determining, through object recognition, the identity of an object for a first plurality of frames of the video sequence; and
determining, through object recognition, the identity of the object and the object's surroundings in a second plurality of frames of the video sequence.

15. The method of claim 10 , further comprising:
capturing a second video sequence, the second video sequence comprising a plurality of individual frames, the plurality of individual frames of the second video sequence including images of one or more users performing the action of swallowing the medication pill;
performing, by the at least one processor, activity recognition on the second video sequence to recognize one or more micro movements of one or more body parts associated with the swallowing of the medication pill by the one or more users;
designating one or more frames of the second video sequence as comprising the recognized one or more micro movements of the one or more body parts of the user;
identifying a first portion of the one or more frames of the second video sequence to remain visible, wherein first portion of the one or more frames of the second video sequence comprises the recognized one or more micro movements of the one or more body parts of the user;
de-identifying a second portion of the one or more frames of the second video sequence so that the first portion of the one or more frames of the second video sequence remains visible;
determining that the first portion of the one or more frames of the second video sequence comprises a face other than faces of the one or more users performing the action of swallowing the medication pill; and
de-identifying, from the first portion of the one or more frames of the second video sequence, the face other than faces of the one or more users performing the action of swallowing the medication pill.

16. The method of claim 10 , further comprising performing, by the at least one processor, activity recognition on the captured first video sequence to recognize the placement of the medication pill in the mouth of the one or more users performing the action of putting the medication pill in their mouth,
wherein performing activity recognition to recognize the placement of the medication pill in the mouth of the one or more users further comprises:
defining a plurality of detection methods for recognizing the placement of the medication pill in the mouth of the one or more users; and
applying each detection met

Assignees

• Aic Innovations Group Inc

Inventors

• Hanina Adam
• Guan Lei

Classifications

• G16H10/60

  for patient-specific data, e.g. for electronic patient records · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F19/3456

  Physics · mapped topic

• G06K9/00711

  Physics · mapped topic

• G06K9/00335 Primary

  Physics · mapped topic