Systems and methods for expert guided semi-supervision with contrastive loss for machine learning models

Patent metadata

Field	Value
Publication number	US-12536437-B2
Application number	US-202217888849-A
Country	US
Kind code	B2
Filing date	Aug 16, 2022
Priority date	Aug 16, 2022
Publication date	Jan 27, 2026
Grant date	Jan 27, 2026

Abstract

A method includes, in response to at least one convergence criterion not being met: receiving a labeled dataset that includes a plurality of labeled samples; receiving an unlabeled dataset that includes a plurality of unlabeled samples; identifying a plurality of labeled-unlabeled sample pairs; applying a data augmentation transformation to each labeled sample and each corresponding unlabeled sample; computing, for each least one labeled-unlabeled sample pair, latent representation spaces using the machine learning model; generating, using the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair; computing a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions; applying an optimization function to each respective loss function; and updating a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs responsive to applying the optimization function.

First claim

What is claimed is:

1 . A method for semi-supervised training of a machine learning model, the method comprising:
determining whether at least one convergence criterion is met; and in response to the at least one convergence criterion not being met:
receiving a labeled dataset that includes a plurality of labeled samples;
receiving an unlabeled dataset that includes a plurality of unlabeled samples;
identifying a plurality of labeled-unlabeled sample pairs, each labeled-unlabeled sample pair including a respective labeled sample of the labeled samples and a corresponding unlabeled sample of the plurality of unlabeled samples;
applying a data augmentation transformation to each labeled sample and each corresponding unlabeled sample for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs;
computing, for each least one labeled-unlabeled sample pair, latent representation spaces using the machine learning model;
generating, using the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair;
computing a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions, wherein the loss function includes a contrastive loss term that utilizes a cosine similarity metric between the latent representations;
applying a stochastic gradient descent optimization function to minimize each respective loss function, including the contrastive loss term; and
updating a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs responsive to applying the optimization function.

2 . The method of claim 1 , wherein the machine learning model includes a feature extractor and one or more predictor networks.

3 . The method of claim 1 , further comprising training the machine learning model using the labeled dataset and the unlabeled dataset having samples propagated with predicted labels.

4 . The method of claim 1 , wherein identifying the plurality of labeled-unlabeled sample pairs includes identifying the plurality of labeled-unlabeled sample pairs using a similarity graph associated with the labeled dataset and the unlabeled dataset.

5 . The method of claim 4 , wherein the similarity graph is generated based on at least an expert derived similarity graph.

6 . The method of claim 1 , wherein the loss function includes a a mean squared error of each label prediction.

7 . The method of claim 1 , wherein the machine learning model is configured to perform at least one classification task.

8 . The method of claim 1 , wherein the machine learning model is configured to perform at least one regression task.

9 . A system for semi-supervised training of a machine learning model, the system comprising:
a processor; and a memory including instructions that, when executed by the processor, cause the processor to:
determine whether at least one convergence criterion is met; and
in response to the at least one convergence criterion not being met:
receive a labeled dataset that includes a plurality of labeled samples;
receive an unlabeled dataset that includes a plurality of unlabeled samples;
identify a plurality of labeled-unlabeled sample pairs, each labeled-unlabeled sample pair including a respective labeled sample of the labeled samples and a corresponding unlabeled sample of the plurality of unlabeled samples;
apply a data augmentation transformation to each labeled sample and each corresponding unlabeled sample for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs;
compute, for each least one labeled-unlabeled sample pair, latent representation spaces using the machine learning model;
generate, using the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair;
compute a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions, wherein the loss function includes a contrastive loss term that utilizes a cosine similarity metric between the latent representations;
apply a stochastic gradient descent optimization function to minimize each respective loss function, including the contrastive loss term; and
update a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs responsive to applying the optimization function.

10 . The system of claim 9 , wherein the machine learning model a feature extractor and one or more predictor networks.

11 . The system of claim 9 , wherein the instructions further cause the processor to train the machine learning model using the labeled dataset and the unlabeled dataset having samples propagated with predicted labels.

12 . The system of claim 9 , wherein the instructions further cause the processor to identify the plurality of labeled-unlabeled sample pairs by identifying the plurality of labeled-unlabeled sample pairs using a similarity graph associated with the labeled dataset and the unlabeled dataset.

13 . The system of claim 12 , wherein the similarity graph is generated based on at least an expert derived similarity graph.

14 . The system of claim 9 , wherein the loss function includes a a mean squared error of each label prediction.

15 . The system of claim 9 , wherein the machine learning model is configured to perform at least one classification task.

16 . The system of claim 9 , wherein the machine learning model is configured to perform at least one regression task.

17 . An apparatus for semi-supervised training of a machine learning model, the apparatus comprising:
a processor; and a memory including instructions that, when executed by the processor, cause the processor to:
determine whether at least one convergence criterion is met; and
in response to the at least one convergence criterion not being met:
receive a labeled dataset that includes a plurality of labeled samples;
receive an unlabeled dataset that includes a plurality of unlabeled samples;
identify a plurality of labeled-unlabeled sample pairs, each labeled-unlabeled sample pair including a respective labeled sample of the labeled samples and a corresponding unlabeled sample of the plurality of unlabeled samples;
apply a data augmentation transformation to each labeled sample and each corresponding unlabeled sample for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs;
compute, for each least one labeled-unlabeled sample pair, latent representation spaces using a feature extractor of the machine learning model;
generate, using one or more predictor networks of the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair;
compute a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions, the loss function including a combination of a mean squared error of each label prediction and a contrastive loss that utilizes a cosine similarity metric between the latent representations;
apply a stochastic gradient descent optimization function to minimize each respective loss function, including the contrastive loss term; and
update a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled

Assignees

• Bosch Gmbh Robert

Inventors

• Fathony Rizal Zaini Ahmad
• Cabrita Condessa Filipe J
• Soren Bijay Kumar
• Schorn Felix
• Lang Florian
• Alber Thomas
• Kuka Michael
• Henke Andreas

Classifications

• G06F16/23

  Updating · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title

• G06F16/2282

  Tablespace storage structures; Management thereof · CPC title

• G06N3/08 Primary

  Learning methods · CPC title