Machine learning feature recommendation

Patent metadata

Field	Value
Publication number	US-2022019936-A1
Application number	US-202016931906-A
Country	US
Kind code	A1
Filing date	Jul 17, 2020
Priority date	Jul 17, 2020
Publication date	Jan 20, 2022
Grant date	—

Abstract

A specification of a desired target field for machine learning prediction and one or more tables storing machine learning training data are received. Within the one or more tables, eligible machine learning features for building a machine learning model to perform a prediction for the target field are identified. The eligible machine learning features are evaluated using a pipeline of different evaluations to successively filter out one or more of the eligible machine learning features to identify a set of recommended machine learning features among the eligible machine learning features. The set of recommended machine learning features is provided for use in building the machine learning model.

First claim

What is claimed is:

1 . A method, comprising:
receiving a specification of a desired target field for machine learning prediction and one or more tables storing machine learning training data; identifying within the one or more tables eligible machine learning features for building a machine learning model to perform a prediction for the desired target field; evaluating the eligible machine learning features using a pipeline of different evaluations to successively filter out one or more of the eligible machine learning features to identify a set of recommended machine learning features among the eligible machine learning features; and providing the set of recommended machine learning features for use in building the machine learning model.

2 . The method of claim 1 , further comprising:
training the machine learning model using the provided set of recommended machine learning features; applying the trained machine learning model to determine a classification result; and performing a server-side action based on the determined classification result.

3 . The method of claim 2 , wherein the determined classification result is an incident classification of a support incident event.

4 . The method of claim 3 , wherein the performed server-side action is an assignment action to designate a party responsible for the support incident event.

5 . The method of claim 1 , wherein the one or more tables storing machine learning training data include historical customer data.

6 . The method of claim 1 , wherein the provided set of recommended machine learning features are ranked based on an evaluation of an impact to an accuracy of the machine learning model.

7 . The method of claim 1 , further comprising providing a different performance metric associated with each machine learning feature of the set of recommended machine learning features.

8 . The method of claim 7 , wherein at least one of the performance metrics is based on an increased amount of an area under a precision-recall curve associated with the machine learning model.

9 . The method of claim 1 , further comprising identifying a set of useless features from the eligible machine learning features.

10 . The method of claim 1 , wherein providing the set of recommended machine learning features for use in building the machine learning model includes providing a web service user interface to display the set of recommended machine learning features.

11 . The method of claim 10 , wherein the web service user interface allows a user to select one or more features from the displayed set of recommended machine learning features for training the machine learning model.

12 . The method of claim 1 , further comprising:
receiving a selection of machine learning features from the provided set of recommended machine learning features; and training the machine learning model using the selection of machine learning features.

13 . The method of claim 12 , further comprising:
preparing a training dataset for training the machine learning model using a subset of data from the received one or more tables storing machine learning training data.

14 . The method of claim 13 , wherein preparing the training dataset for training the machine learning model includes excluding data for features not belonging to the selection of machine learning features.

15 . The method of claim 1 , wherein identifying within the one or more tables the eligible machine learning features for building the machine learning model to perform the prediction for the desired target field includes determining a data type associated with each column of the one or more tables.

16 . The method of claim 15 , wherein the determined data type is a text, nominal, or numeric data type.

17 . The method of claim 1 , wherein the pipeline of different evaluations includes a first evaluation step to determine an impact score and a second evaluation step to determine a performance metric.

18 . The method of claim 17 , wherein the impact score is based on determining a weighted information gain score of one of the eligible machine learning features and the performance metric is determined including by applying an offline trained model to the impact score to determine the performance metric.

19 . A system, comprising:
a processor; and a memory coupled to the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
receive a specification of a desired target field for machine learning prediction and data from one or more tables storing machine learning training data;
identify within the data from the one or more tables eligible machine learning features for building a machine learning model to perform a prediction for the desired target field;
evaluate the eligible machine learning features using a pipeline of different evaluations to successively filter out one or more of the eligible machine learning features to identify a set of recommended machine learning features among the eligible machine learning features; and
provide the set of recommended machine learning features for use in building the machine learning model.

20 . A computer program product, the computer program product being embodied in a non-transitory computer readable medium and comprising computer instructions for:
receiving a specification of a desired target field for machine learning prediction and one or more tables storing machine learning training data; identifying within the one or more tables eligible machine learning features for building a machine learning model to perform a prediction for the desired target field; evaluating the eligible machine learning features using a pipeline of different evaluations to successively filter out one or more of the eligible machine learning features to identify a set of recommended machine learning features among the eligible machine learning features; and providing the set of recommended machine learning features for use in building the machine learning model.

Assignees

• Servicenow Inc

Inventors

• Sarda Gopal
• Ramachandran Sravan
• Subramanian Seganrasan
• Jayaraman Baskar

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N5/00

  Computing arrangements using knowledge-based models · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06N7/005

  Physics · mapped topic