Enhanced collection of training data for machine learning to improve worksite safety and operations

Patent metadata

Field	Value
Publication number	US-2021390280-A1
Application number	US-202016902089-A
Country	US
Kind code	A1
Filing date	Jun 15, 2020
Priority date	Jun 15, 2020
Publication date	Dec 16, 2021
Grant date	—

Abstract

Systems and methods for enhanced collection of training data for machine learning to improve worksite safety and operations. One embodiment is a system with an interface to receive first evaluations of a first scene from a group of trainees, the first scene belonging to safety curriculum content and depicting a worksite with a known hazard that is associated in memory with a hazard profile. The system includes a controller to determine a trusted subgroup of the trainees that correctly identified the known hazard in the first scene. The interface receives second evaluations of a second scene from the trusted subgroup of the trainees that depicts the worksite with an unknown hazard. The controller trains a machine learning function based on the second evaluations from the trusted subgroup of the trainees for automatic identification of hazard indications in the second scene depicting the worksite with the unknown hazard.

First claim

What is claimed is:

1 . A worksite safety system comprising:
an interface configured to receive first evaluations of a first scene from a group of trainees, the first scene belonging to safety curriculum content and depicting a worksite with a known hazard that is associated in memory with a hazard profile; and a safety training controller configured to determine a trusted subgroup of the trainees that correctly identified the known hazard in the first scene based on a match between the first evaluations and the hazard profile, the interface configured to receive second evaluations of a second scene from the trusted subgroup of the trainees, the second scene depicting the worksite with an unknown hazard, and the safety training controller configured to train a machine learning function based on the second evaluations from the trusted subgroup of the trainees for automatic identification of hazard indications in the second scene depicting the worksite with the unknown hazard.

2 . The worksite safety system of claim 1 further comprising:
one or more sensors configured to record sensor data of the worksite,
the safety training controller configured to apply the sensor data to the machine learning function trained with the second evaluations to automatically detect the hazard indications of the unknown hazard in the sensor data.

3 . The worksite safety system of claim 2 wherein:
the safety training controller configured, in response to detecting the hazard indications of the unknown hazard in the sensor data, to generate an automatic action to perform for responding to the unknown hazard.

4 . The worksite safety system of claim 1 wherein:
the safety training controller configured to modify the safety curriculum content based on the first evaluations and the second evaluations.

5 . The worksite safety system of claim 1 wherein:
the safety training controller configured to determine a subset of trainees of the trusted subgroup that provided an evaluation that includes additional hazard data, and to create the second scene based on the additional hazard data.

6 . The worksite safety system of claim 1 wherein:
the safety training controller configured to generate the second scene by retrieving scene data that preceded the first scene and the known hazard.

7 . A method of training a machine learning function for worksite safety, the method comprising:
receiving first evaluations of a first scene from a group of the trainees, the first scene belonging to safety curriculum content and depicting a worksite with a known hazard that is associated in memory with a hazard profile; determining a trusted subgroup of the trainees that correctly identified the known hazard in the first scene based on a match between the first evaluations and the hazard profile; receiving second evaluations of a second scene from the trusted subgroup of the trainees, the second scene depicting the worksite with an unknown hazard; and training the machine learning function based on the second evaluations from the trusted subgroup of the trainees for automatic identification of hazard indications in the second scene depicting the worksite with the unknown hazard.

8 . The method of claim 7 further comprising:
monitoring the worksite with one or more sensors to record sensor data of the worksite; and
applying the sensor data to the machine learning function trained with the second evaluations to automatically detect the hazard indications of the unknown hazard in the sensor data.

9 . The method of claim 8 further comprising:
in response to detecting the hazard indications of the unknown hazard in the sensor data, generating an automatic action to perform for responding to the unknown hazard.

10 . The method of claim 9 wherein:
the automatic action includes one or more of: generating a warning of the hazard indications present in the worksite, and automatically powering down a machine in the worksite.

11 . The method of claim 7 further comprising:
modifying the safety curriculum content based on the first evaluations and the second evaluations.

12 . The method of claim 7 further comprising:
determining a subset of trainees of the trusted subgroup that provided an evaluation that includes additional hazard data; and
creating the second scene based on the additional hazard data.

13 . The method of claim 12 further comprising:
identifying the additional hazard data by:
analyzing the evaluation of a trainee to determine at least one alternative characteristic of the first scene that does not match the hazard profile.

14 . The method of claim 12 further comprising:
generating the additional hazard data by:
requesting the trusted subgroup of the trainees to provide at least one characteristic of another scene in the safety curriculum content;
receiving the at least one characteristic of the another scene; and
identifying the at least one characteristic as the additional hazard data.

15 . The method of claim 7 further comprising:
generating the second scene by retrieving scene data that preceded the first scene and the known hazard.

16 . A non-transitory computer readable medium embodying programmed instructions executed by a processor, wherein the instructions direct the processor to perform a method of training a machine learning function for worksite safety, the method comprising:
receiving first evaluations of a first scene from a group of the trainees, the first scene belonging to safety curriculum content and depicting a worksite with a known hazard that is associated in memory with a hazard profile; determining a trusted subgroup of the trainees that correctly identified the known hazard in the first scene based on a match between the first evaluations and the hazard profile; receiving second evaluations of a second scene from the trusted subgroup of the trainees, the second scene depicting the worksite with an unknown hazard; and training the machine learning function based on the second evaluations from the trusted subgroup of the trainees for automatic identification of hazard indications in the second scene depicting the worksite with the unknown hazard.

17 . The computer readable medium of claim 16 wherein the method further comprises:
monitoring the worksite with one or more sensors to record sensor data of the worksite; and
applying the sensor data to the machine learning function trained with the second evaluations to automatically detect the hazard indications of the unknown hazard in the sensor data.

18 . The computer readable medium of claim 17 wherein the method further comprises:
in response to detecting the hazard indications of the unknown hazard in the sensor data, generating an automatic action to perform for responding to the unknown hazard.

19 . The computer readable medium of claim 16 wherein the method further comprises:
modifying the safety curriculum content based on the first evaluations and the second evaluations.

20 . The computer readable medium of claim 16 wherein the method further comprises:
determining a subset of trainees of the trusted subgroup that provided an evaluation that includes additional hazard data; and
creating the second scene based on the additional hazard data.

Assignees

• Boeing Co

Inventors

• Laughlin Brian D
• Kelsey William D

Classifications

• G06V10/7788 Primary

  the supervisor being a human, e.g. interactive learning with a human teacher · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06N5/046

  Forward inferencing; Production systems · CPC title