Utilizing artificial intelligence to detect objects or patient safety events in a patient room

Patent metadata

Field	Value
Publication number	US-12008880-B2
Application number	US-202318314502-A
Country	US
Kind code	B2
Filing date	May 9, 2023
Priority date	Dec 28, 2017
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

Methods and systems are provided for detecting objects or patient safety events in a patient room. Artificial intelligence is utilized to enhance safety issue recognition capabilities by the methods and systems. Sensors collect a series of images and depth data in a room of a patient. Data (corresponding to images and depth data of an object or patent safety event) is received from the sensors and compared to stored data to identify the object or patient safety event. The images are communicated to a central video monitoring system and a user may be prompted to confirm if the object requires learning or a patient safety event occurred (or identify the object or patient safety event) or to provide additional parameters or actions. A patient safety learning system analyzes the data and incorporates the user response to enhance safety issue recognition capabilities of the system and reduce false alerts.

First claim

The invention claimed is:

1. A method comprising:
collecting images of a scene including at least one object;
detecting the at least one object within at least one image of the images, wherein the at least one object is detected by comparing one or more features of the at least one object to one or more reference features;
in response to detecting that the at least one object is within the at least one image, presenting the at least one object in a graphical user interface;
receiving an indication that the at least one object has been selected in the graphical user interface;
in response to receiving the indication that the at least one object has been selected, updating a machine learning model using the one or more features of the at least one object, the updated machine learning model configured to predict that an event has occurred in the scene;
detecting that an event has occurred within the scene;
in response to detecting that the event has occurred, presenting information associated with the event in the graphical user interface;
receiving an indication that the event has been confirmed in the graphical user interface; and
in response to receiving the indication that the event has been confirmed, further updating the updated machine learning model.

2. The method of claim 1 , wherein at least one image of the images is collected by an image capturing device that is configured to capture images of an environment.

3. The method of claim 2 , wherein the environment is a patient room.

4. The method of claim 1 , wherein comparing the one or more features of the at least one object to one or more reference features comprises assigning one or more reference points to the one or more features of the at least one object.

5. The method of claim 1 , further comprising:
receiving, from one or more image capturing devices, a set of images of an environment; and
predicting, using the updated machine learning model, that at least one event has occurred in the environment from the set of images.

6. The method of claim 5 , further comprising:
in response to predicting that the at least one event has occurred in the environment, generating an alert indicating that the at least one event occurred in the environment.

7. A system comprising:
a processing system; and
one or more computer readable storage media storing instructions which, when executed by the processing system, cause the processing system to perform operations comprising:
collecting images of a scene including at least one object;
detecting the at least one object within at least one image of the images, wherein the at least one object is detected by comparing one or more features of the at least one object to one or more reference features;
in response to detecting that the at least one object is within the at least one image, presenting the at least one object in a graphical user interface;
receiving an indication that the at least one object has been selected in the graphical user interface;
in response to receiving the indication that the at least one object has been selected, updating a machine learning model using the one or more features of the at least one object, the updated machine learning model configured to predict that an event has occurred in the scene;
detecting that an event has occurred within the scene;
in response to detecting that the event has occurred, presenting information associated with the event in the graphical user interface;
receiving an indication that the event has been confirmed in the graphical user interface; and
in response to receiving the indication that the event has been confirmed, further updating the updated machine learning model.

8. The system of claim 7 ,
wherein at least one image of the images is collected by an image capturing device that is configured to capture images of an environment.

9. The system of claim 8 , wherein the environment is a patient room.

10. The system of claim 7 , wherein comparing the one or more features of the at least one object to one or more reference features comprises assigning one or more reference points to the one or more features of the at least one object.

11. The system of claim 7 , the operations further comprising:
receiving, from one or more image capturing devices, a set of images of an environment; and
predicting, using the updated machine learning model, that at least one event has occurred in the environment from the set of images.

12. The system of claim 11 , the operations further comprising:
in response to predicting that the at least one event has occurred in the environment, generating an alert indicating that the at least one event occurred in the environment.

13. One or more non-transitory computer-readable media storing computer-readable instructions that, when executed by a processing system, cause a system to perform operations comprising:
collecting images of a scene including at least one object;
detecting that at least one object within at least one image of the images, wherein the at least one object is detected by comparing one or more features of the at least one object to one or more reference features;
in response to detecting that the at least one object is within the at least one image, presenting the at least one object in a graphical user interface;
receiving an indication that the at least one object has been selected in the graphical user interface;
in response to receiving the indication that the at least one object has been selected, updating a machine learning model using the one or more features of the at least one object, the updated machine learning model configured to predict that an event has occurred in the scene;
detecting that an event has occurred within the scene;
in response to detecting that the event has occurred, presenting information associated with the event in the graphical user interface;
receiving an indication that the event has been confirmed in the graphical user interface; and
in response to receiving the indication that the event has been confirmed, further updating the updated machine learning model.

14. The one or more non-transitory computer-readable media of claim 13 , the operations further comprising:
wherein at least one image of the images is collected by an image capturing device that is configured to capture images of an environment.

15. The one or more non-transitory computer-readable media of claim 13 , wherein comparing the one or more features of the at least one object to one or more reference features comprises assigning one or more reference points to the one or more features of the at least one object.

16. The one or more non-transitory computer-readable media of claim 13 , the operations further comprising:
receiving, from one or more image capturing devices, a set of images of an environment; and
predicting, using the updated machine learning model, that at least one event has occurred in the environment from the set of images.

17. The one or more non-transitory computer-readable media of claim 16 , the operations further comprising:
in response to predicting that the at least one event has occurred in the environment, generating an alert indicating that the at least one event occurred in the environment.

Assignees

• Cerner Innovation Inc

Inventors

• Kusens Bruce Howard
• Kusens Michael

Classifications

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title

• G08B29/185

  Signal analysis techniques for reducing or preventing false alarms or for enhancing the reliability of the system · CPC title

• G08B21/0476 Primary

  Cameras to detect unsafe condition, e.g. video cameras · CPC title

• G08B25/08 Primary

  using communication transmission lines {(G08B13/19658, G08B21/0286, G08B25/016 take precedence)} · CPC title