Automated collection of machine learning training data

Patent metadata

Field	Value
Publication number	US-11429807-B2
Application number	US-201815870783-A
Country	US
Kind code	B2
Filing date	Jan 12, 2018
Priority date	Jan 12, 2018
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

Methods and systems for automatically generating training data for use in machine learning are disclosed. The methods can involve the use of environmental data derived from first and second environmental sensors for a single event. The environmental data types derived from each environmental sensor are different. The event is detected based on first environmental data derived from the first environmental sensor, and a portion of second environmental data derived from the second environmental sensor is selected to generate training data for the detected event. The resulting training data can be employed to train machine learning models.

First claim

What is claimed is:

1. A training data collection device for automatically generating machine learning training data, the training data collection device comprising:
a first environmental sensor; and
a first machine readable medium including first instructions which, when executed by the training data collection device, cause the training data collection device to:
receive first training event detection data for identifying events of a first event type based on at least environmental data of a first environmental data type;
based on the received first training event detection data, configure a training event detector included in the device to identify events of the first event type based on at least environmental data of the first environmental data type;
obtain first environmental data of the first environmental data type based at least on a first measurement performed by the first environmental sensor at a first time;
obtain second environmental data of a second environmental data type based on at least one or more measurements performed by a second environmental sensor, wherein the second environmental data type is different than the first environmental data type;
automatically determine, based on at least applying the configured training event detector to the first environmental data, that a first event occurred at about the first time;
automatically select a first subportion of the second environmental data based on at least the first subportion of the second environmental data corresponding to the first measurement performed by the first environmental sensor; and
generate training data based on at least the selected first subportion of the second environmental data.

2. A training system for automatically collecting machine learning training data, the training system comprising:
the training data collection device of claim 1 , wherein the first instructions further cause the training data collection device to transmit the training data to a remote computer server; and
a second machine readable medium including second instructions which, when executed by the training system, cause the training system to:
receive the transmitted training data, and
train, using at least the received training data, a first machine learning model configured to be applied to environmental data of the second environmental data type to determine whether an event of the first event type has occurred.

3. The training system of claim 2 , further comprising a trained system that includes a third environmental sensor and a third machine readable medium including instructions which, when executed by the trained system, cause the trained system to:
obtain third environmental data of the second environmental data type based at least on measurements performed by the third environmental sensor; and
automatically determine, by applying the first machine learning model to the third environmental data, that an event of the first event type has occurred.

4. The training data collection device of claim 1 , further comprising the second environmental sensor, wherein the second environmental sensor is different than the first environmental sensor.

5. The training data collection device of claim 1 , wherein the training event detection data includes values for a machine learning model configured to be applied to data of the first environmental data type to detect whether an event of the first event type has occurred.

6. The training data collection device of claim 1 , wherein:
the first instructions further cause the training data collection device to automatically determine, by applying the training event detector to the first environmental data, a region of interest for the first event; and
further base the automatic selection of the first subportion of the second environmental data on an identification of the first subportion of the second environmental data as corresponding to measurements performed by the second environmental sensor for the determined region of interest.

7. The training data collection device of claim 6 , wherein:
the first environmental sensor includes a first optical imaging sensor;
the first environmental data includes a first two-dimensional image corresponding to measurements performed by the first optical imaging sensor;
the training data collection device includes the second environmental sensor;
the second environmental sensor includes a second optical imaging sensor;
the second environmental data includes a second two-dimensional image corresponding to measurements performed by the second optical imaging sensor;
the determined region of interest is a subportion of the first image; and
the first instructions further cause the training data collection device to translate positions for the determined region of interest to corresponding positions in the second image to identify a subportion of the second image that corresponds to the determined region of interest.

8. The training data collection device of claim 2 , wherein the first instructions further cause the training data collection device to:
automatically generate a label for the first event, by applying the training event detector to the first environmental data; and
include the label in the training data transmitted to the remote computer server.

9. The training data collection device of claim 1 , wherein the first instructions further cause the training data collection device to:
further configure the training event detector to identify events of a second event type based at least on environmental data of a third environmental data type;
obtaining third environmental data of the third environmental data type based on at least a second measurement performed by a third environmental sensor; and
further base the determination that the first event occurred on applying the configured training event detector to the third environmental data.

10. A method of automated generation of machine learning training data, the method comprising:
receiving, at a first computing device including a first environmental sensor, first training event detection data for identifying events of a first event type based on at least environmental data of a first environmental data type;
based on the received first training event detection data, configuring a training event detector included in the first computing device to identify events of the first event type based on at least environmental data of the first environmental data type;
obtaining first environmental data of the first environmental data type based at least on a first measurement performed by the first environmental sensor at a first location at a first time;
obtaining second environmental data of a second environmental data type based on at least one or more measurements performed by a second environmental sensor at the first location, wherein the second environmental data type is different than the first environmental data type;
automatically determining, based on at least applying the configured training event detector to the first environmental data, that a first event occurred at about the first time;
automatically selecting a first subportion of the second environmental data based on at least the first subportion of the second environmental data corresponding to the first measurement performed by the first environmental sensor;
generating training data based on at least the selected first subportion of the second environmental data.

11. The method of claim 10 , further comprising training, using at least the received training data, a first machine learning model configured to be applied to environmental data of the second environmental data type to determine wh

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Pradeep Vivek

Classifications

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/811

  the classifiers operating on different input data, e.g. multi-modal recognition · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V40/70 Primary

  Multimodal biometrics, e.g. combining information from different biometric modalities · CPC title

• G06F18/256

  of results relating to different input data, e.g. multimodal recognition · CPC title