Speculative actions based on predicting negative circumstances

Patent metadata

Field	Value
Publication number	US-11854264-B2
Application number	US-202117304321-A
Country	US
Kind code	B2
Filing date	Jun 18, 2021
Priority date	Jun 18, 2021
Publication date	Dec 26, 2023
Grant date	Dec 26, 2023

Abstract

A computer that identifies the video. The computer annotates the video using a deep learning tool. The computer analyzes the annotated video to highlight a dangerous condition. The computer identifies a video from a repository with the dangerous condition. The computer analyzes the video and the video from the repository using a similarity analysis. The computer determines a score based on the annotated video and based on comparing the video to the video from the repository and based on determining the score is above a threshold value, the computer generates an action.

First claim

What is claimed is:

1. A processor-implemented method for negative circumstance prediction of a video, the method comprising:
identifying the video;
annotating the video, using a deep learning tool, with annotation data identifying one or more individual objects in a frame of the video, and providing annotated video;
analyzing the annotated video, including the annotation data of the annotated video, to identify a dangerous condition;
identifying a stored video from a repository with the identified dangerous condition;
analyzing the video and the stored video from the repository using a similarity analysis;
determining a final score based on a weighted average of a first score determined from the annotated video, a second score determined from the video analysis, and based on a comparison of the video to the stored video from the repository; and
based on determining the score is above a threshold value, performing an action to prevent the dangerous condition.

2. The method of claim 1 , wherein the video is a time-lapse photography of a controlled space.

3. The method of claim 1 , wherein the annotating is performed by an image processing algorithm that identifies the one or more objects from the video and one or more relations of the one or more objects towards each other.

4. The method of claim 1 , wherein analyzing the annotated video is performed using a neural network that was trained using samples of the dangerous condition and predicts a potential outcome of the annotated video.

5. The method of claim 1 , wherein the final score represents a chance of the dangerous condition occurring.

6. The method of claim 1 , wherein the action comprises sending an alert to a user.

7. A computer system for negative circumstance prediction of a video, the computer system comprising:
one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
identifying the video;
annotating the video, using a deep learning tool, with annotation data identifying one or more individual objects in a frame of the video, and providing annotated video;
analyzing the annotated video, including the annotation data of the annotated video, to identify a dangerous condition;
identifying a stored video from a repository with the identified dangerous condition;
analyzing the video and the stored video from the repository using a similarity analysis;
determining a score based on a weighted average of a first score determined from the annotated video, a second score determined from the video analysis, and based on a comparison of the video to the stored video from the repository; and
based on determining the score is above a threshold value, performing an action to prevent the dangerous condition.

8. The computer system of claim 7 , wherein the video is a time-lapse photography of a controlled space.

9. The computer system of claim 7 , wherein the annotating is performed by an image processing algorithm that identifies the one or more objects from the video and one or more relations of the one or more objects towards each other.

10. The computer system of claim 7 , wherein analyzing the annotated video is performed using a neural network that was trained using samples of the dangerous condition and predicts a potential outcome of the annotated video.

11. The computer system of claim 7 , wherein the final score represents a chance of the dangerous condition occurring.

12. The computer system of claim 7 , wherein the action comprises sending an alert to a user.

13. A computer program product for negative circumstance prediction of a video, the computer program product comprising:
one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor, the program instructions comprising:
program instructions to identify the video;
program instructions to annotate the video, using a deep learning tool, with annotation data identifying one or more individual objects in a frame of the video, and providing annotated video;
program instructions to analyze the annotated video, including the annotation data of the annotated video, to identify a dangerous condition;
program instructions to identify a stored video from a repository with the identified dangerous condition;
program instructions to analyze the video and the stored video from the repository using a similarity analysis;
program instructions to determine a score based on a weighted average of a first score determined from the annotated video, a second score determined from the video analysis, and based on a comparison of the video to the stored video from the repository; and
based on determining the score is above a threshold value, program instructions to perform an action to prevent the dangerous condition.

14. The computer program product of claim 13 , wherein the video is a time-lapse photography of a controlled space.

15. The computer program product of claim 13 , wherein the annotating is performed by an image processing algorithm that identifies the one or more objects from the video and one or more relations of the one or more objects towards each other.

16. The computer program product of claim 13 , wherein the program instructions to analyze the annotated video is performed using a neural network that was trained using samples of the dangerous condition and predicts a potential outcome of the annotated video.

17. The computer program product of claim 13 , wherein the final score represents a chance of the dangerous condition occurring.

18. The computer program product of claim 13 , wherein the action comprises sending an alert to a user.

Assignees

• Kyndryl Inc

Inventors

• Marzorati Mauro
• Llamas Virgen Paul
• Jaiswal Priyansh
• Jaiswal Peeyush

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464 Primary

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V20/48 Primary

  Matching video sequences · CPC title

• G06F18/2113

  by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation · CPC title

• G06N3/08

  Learning methods · CPC title