Embedding human labeler influences in machine learning interfaces in computing environments

Patent metadata

Field	Value
Publication number	US-11526713-B2
Application number	US-201816145601-A
Country	US
Kind code	B2
Filing date	Sep 28, 2018
Priority date	Sep 28, 2018
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

A mechanism is described for facilitating embedding of human labeler influences in machine learning interfaces in computing environments, according to one embodiment. A method of embodiments, as described herein, includes detecting sensor data via one or more sensors of a computing device, and accessing human labeler data at one or more databases coupled to the computing device. The method may further include evaluating relevance between the sensor data and the human labeler data, where the relevance identifies meaning of the sensor data based on human behavior corresponding to the human labeler data, and associating, based on the relevance, human labeler data with the sensor data to classify the sensor data as labeled data. The method may further include training, based on the labeled data, a machine learning model to extract human influences from the labeled data, and embed one or more of the human influences in one or more environments representing one or more physical scenarios involving one or more humans.

First claim

What is claimed is:

1. At least one machine-readable medium comprising instructions which, when executed by a computing device, cause the computing device to perform operations comprising:
classifying, by one or more processors of the computing device, sensor data with human labeler data, where the sensor data is obtained through one or more sensors communicably coupled to the one or more processors; and
creating and training, by the one or more processors, a unified machine learning model based on features associated with the classified sensor data based on the human labeler data, wherein the features comprise human labeler influences as obtained from the human labeler data associated with the sensor data;
wherein training is further to facilitate the unified machine learning model to interpret, based on the human labeler data, the human labeler influences according to one or more environments prior to embedding the one or more human labeler influences in the one or more environments, wherein the interpretation of the human influences is based on acceptances of human behavior and exceptions to the human behavior as derived from the human labeler data and based on a relevance between the sensor data and the human labeler data, wherein the relevance identifies meaning of the sensor data based on human behavior corresponding to the human labeler data, and wherein the acceptances of the human behavior are based on verified data obtained from one or more of personal profiles, cultural traits, historical norms, societal preferences, personal prejudices, societal biases, or habits.

2. The machine-readable medium of claim 1 , wherein the operations further comprise:
detecting the sensor data through the one or more sensors including one or more of a camera, a microphone, a touch sensor, a capacitor, a radio component, a radar component, a scanner, and an accelerometer; and
monitoring the human labeler data to determine one or more of human behaviors and human variables, wherein the human labeler data is obtained through multiple sources including one or more of the one or more sensors, historical data, categorical data, and personal profiles.

3. The machine-readable medium of claim 2 , wherein the operations further comprise prior to classifying the sensor data, evaluate the human behaviors and human variables and their association with the sensor data.

4. The machine-readable medium of claim 3 , wherein the operations further comprise:
filtering out one or more of the human variables associated with one or more of inaccuracies, unintended consequences, and biases, wherein the filtered out one or more human variables include one or more of age, gender, race, ethnicity, national origin, religion, and sexual orientation, and wherein the filtered out one or more human variables further include one or more of accidental acts and coincidental items; and
recognizing and considering one or more variances between first human variables and second human variables in application of the first and second human variables in training the machine learning model.

5. The machine-readable medium of claim 1 , wherein the unified machine learning model is created and trained in an early fusion machine learning environment.

6. The machine-readable medium of claim 1 , wherein the operations further comprise creating and training multiple machine learning models such that each of the multiple machine learning models is based on first features of the features associated with the sensor data or second features of the features associated with human labeler data.

7. The machine-readable medium of claim 6 , wherein the operations further comprise computing scores based on average outcomes obtained from the multiple machine learning models associated with the sensor data and the human labeler data, wherein the scores are maintained in one or more databases to be used with creation and training of future machine learning models, wherein the computing device includes one or more processors comprising one or more of a graphics processor and an application processor, wherein the graphics processor and the application processor are co-located on a common semiconductor package.

8. A method comprising:
detecting sensor data via one or more sensors of a computing device;
accessing human labeler data at one or more databases coupled to the computing device;
evaluating relevance between the sensor data and the human labeler data, wherein the relevance identifies meaning of the sensor data based on human behavior corresponding to the human labeler data;
associating, based on the relevance, human labeler data with the sensor data to classify the sensor data as labeled data; and
training, based on the labeled data, a machine learning model to extract human influences from the human labeler data, and embed one or more of the human influences in one or more environments representing one or more physical scenarios involving one or more humans;
wherein training is further to facilitate the machine learning model to interpret, based on the human labeler data, the human influences according to multiple environments prior to embedding the one or more human influences in the one or more environments, wherein the interpretation of the human influences is based on acceptances of the human behavior and exceptions to the human behavior as derived from the human labeler data and based on the relevance, and wherein the acceptances of the human behavior are based on verified data obtained from one or more of personal profiles, cultural traits, historical norms, societal preferences, personal prejudices, societal biases, or habits.

9. The method of claim 8 , wherein the exceptions to the human behavior are based on unverified data obtained from one or more of coincidences, accidents, inaccuracies, flukes, and unintended consequences; and
wherein one or more of the human behaviors are filtered out based on one or more of the exceptions to avoid associating inaccuracies to the human influences.

10. The method of claim 8 , wherein the relevance is further based on a human-variables portion of the human behavior, wherein the human-variables portion is based on human variables that incite personal prejudices or societal biases, wherein the human variables include one or more of age, gender, race, ethnicity, national origin, political affiliation, religious association, and sexual orientation.

11. The method of claim 8 , wherein the machine learning model includes a unified machine learning model based on the sensor data and the human labeler data, wherein the unified machine learning model is employed during an early fusion scheme of a multimodal machine learning environment, wherein the early fusion scheme represents early fusing of the sensor data and the human labeler data.

12. The method of claim 8 , wherein the machine learning model includes separate machine learning models, wherein a first machine learning model of the separate machine learning models is based on the sensor data and not the human labeler data, wherein a second machine learning model of the separate machine learning models is based on the human labeler data and not the sensor data, and wherein the separate machine learning models are employed during a late fusion scheme of a multimodal machine learning environment, wherein the late fusion scheme represents late fusing of the sensor data and the human labeler data.

13. The method of claim 12 , further comprising:
obtaining a first score from the first machine learning model associated with the sensor data;
obtaining a second score from the second machine learning model associated with the human labele

Assignees

• Intel Corp

Inventors

• Anderson Glen J

Classifications

• G06V30/1916

  Validation; Performance evaluation · CPC title

• G06V10/811

  the classifiers operating on different input data, e.g. multi-modal recognition · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06V30/19147

  Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82

  using neural networks · CPC title