Systems and methods implementing an intelligent machine learning tuning system providing multiple tuned hyperparameter solutions

Patent metadata

Field	Value
Publication number	US-11270217-B2
Application number	US-201816194192-A
Country	US
Kind code	B2
Filing date	Nov 16, 2018
Priority date	Nov 17, 2017
Publication date	Mar 8, 2022
Grant date	Mar 8, 2022

Abstract

Systems and methods include receiving a tuning work request for tuning hyperparameters of a third-party model or system; performing, by a machine learning-based tuning service, a first tuning of the hyperparameters in a first tuning region; identifying tuned hyperparameter values for each of the hyperparameters based on results of the first tuning; setting a failure region based on the tuned hyperparameter values of the first tuning; performing, by the machine learning-based tuning service, a second tuning of the hyperparameters in a second tuning region that excludes the failure region; identifying additional tuned hyperparameter values for each of the hyperparameters based on results of the second tuning; and returning the tuned hyperparameter values and the additional hyperparameter values for implementing the third-party model or system with one of the tuned hyperparameter values and the additional hyperparameter values.

First claim

What is claimed is:

1. A system to tune hyperparameters to improve an effectiveness including one or more of accuracy and computational performances of a machine learning model, the system comprising:
memory; and
a machine learning-based tuning service that is hosted on a distributed networked system, the machine learning-based tuning service to:
access a tuning work request to tune two or more hyperparameters of the machine learning model;
perform a first tuning of the two or more hyperparameters in a hyperparameter space, the hyperparameter space including a plurality of hyperparameter values for the two or more hyperparameters;
identify, within a first tuning region of the hyperparameter space, ones of the hyperparameter values for the two or more hyperparameters based on the first tuning;
set a multidimensional shaped failure region within the hyperparameter space as a polygonal shape based on identified ones of the hyperparameter values as coordinates of the polygonal shape, the multidimensional shaped failure region including:
(i) the identified ones of the hyperparameter values for the two or more hyperparameters, and
(ii) a subset of the plurality of hyperparameter values that encompasses the identified ones of the hyperparameter values;

use the multidimensional shaped failure region to set: (a) the subset of the plurality of hyperparameter values, and (b) the identified ones of the hyperparameter values within the multidimensional shaped failure region, by setting both (a) and (b) to failure for a second tuning of the two or more hyperparameters;
perform the second tuning of the two or more hyperparameters in a second tuning region of the hyperparameter space based on setting the multidimensional shaped failure region;
identify diverse ones of the hyperparameter values for the two or more hyperparameters based on not selecting any of the subset of the plurality of hyperparameter values within the multidimensional shaped failure region; and
return the identified ones of the hyperparameter values and the diverse ones of the hyperparameter values for the machine learning model.

2. The system according to claim 1 , wherein the first tuning region includes a predetermined range of hyperparameter values for ones of the two or more hyperparameters of the tuning work request.

3. The system according to claim 2 , wherein the second tuning region includes the predetermined range of hyperparameter values for the ones of the two or more hyperparameters of the tuning work request excluding the subset of hyperparameter values for the ones of the two or more hyperparameters within the multidimensional shaped failure region.

4. The system according to claim 1 , wherein: (i) the identified ones of the hyperparameter values define a point on a multidimensional coordinate system, and (ii) setting the multidimensional shaped failure region as the polygonal shape includes identifying an area in the hyperparameter space that surrounds the identified ones of the hyperparameter values.

5. The system according to claim 1 , wherein the machine learning-based tuning service is to set the multidimensional shaped failure region by:
identifying a distance from the identified ones of the hyperparameter values; and
defining the polygonal shape encompassing the identified ones of the hyperparameter values based on the distance, the multidimensional shaped failure region is set to an area within the polygonal shape and the subset of the plurality of hyperparameter values within the polygonal shape are set to failure during the second tuning.

6. The system according to claim 5 , wherein the machine learning-based tuning service is to set the multidimensional shaped failure region by:
setting dimensions of the multidimensional shaped failure region including identifying an elastic region that surrounds the identified ones of the hyperparameter values of the first tuning, the multidimensional shaped failure region is set to an area within the elastic region and the subset of the plurality of hyperparameter values within the polygonal shape are set to failure during the second tuning.

7. The system according to claim 5 , wherein the distance is derived based on subscriber data of a subscriber to the machine learning-based tuning service.

8. The system according to claim 1 , wherein the machine learning-based tuning service is to: use the multidimensional shaped failure region to adjust one or more parameters of one or more tuning sources operated by the machine learning-based turning service to identify the ones of the hyperparameter values for the two or more hyperparameters of the tuning work request.

9. The system according to claim 1 , wherein performing the second tuning includes:
setting a tuning distance that defines a position of the second tuning region away from the first tuning region and the multidimensional shaped failure region, the second tuning satisfies or exceeds a diversity threshold.

10. The system according to claim 1 , wherein the machine learning-based tuning service implements an intelligent hyperparameter tuning system, the intelligent hyperparameter tuning system including:
a cluster of distinct machine learning tuning sources that perform distinct tuning operations of the two or more hyperparameters of the machine learning model;
a plurality of queue worker machines that selectively operate one or more of the cluster of distinct machine learning tuning sources based on a receipt of the tuning work request, the plurality of queue worker machines including a plurality of distinct queue worker machines that operate asynchronously to perform disparate tuning operations using one or more of the cluster of distinct machine learning tuning sources;
a shared work queue that is accessible by each of the plurality of distinct queue worker machines, the shared work queue including an asynchronous queue that enables asynchronous tuning operations by the plurality of queue worker machines; and
a platform database including a central repository that collects tuning data generated during tuning sessions of the two or more hyperparameters of the machine learning model.

11. The system according to claim 1 , wherein the performing of the second tuning includes:
applying the multidimensional shaped failure region to a distribution of the plurality of hyperparameter values; and
excluding the subset of the plurality of hyperparameter values and the identified ones of the hyperparameter values from the distribution based on the applying of the multidimensional shaped failure region.

12. The system according to claim 1 , wherein the machine learning-based tuning service is to:
use the multidimensional shaped failure region to set tuning parameters that cause the machine learning-based tuning service to avoid, during the second tuning, generating the subset of the plurality of hyperparameter values that lie within a multidimensional shape of the multidimensional shaped failure region as hyperparameter values for the diverse ones of the hyperparameter values for the two or more hyperparameters.

13. The system according to claim 1 , wherein the machine learning-based tuning service is to: normalize a dimension of a first hyperparameter of the two or more hyperparameters to a dimension of a second hyperparameter of the two or more hyperparameters, the multidimensional shaped failure region is set in response to the normalization.

14. The system according to claim 1 , wherein the machine learning-based tuning service is to:
generate, during the second tuning, additional hyperparameter values that lie within the multidimensional shaped failur

Assignees

• Intel Corp

Inventors

• Tee Kevin
• Mccourt Michael
• Hayes Patrick
• Clark Scott

Classifications

• G06N20/20 Primary

  Ensemble learning · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title