System and method for field value recommendations based on confidence levels in analyzed dataset

Patent metadata

Field	Value
Publication number	US-10915827-B2
Application number	US-201816198449-A
Country	US
Kind code	B2
Filing date	Nov 21, 2018
Priority date	Sep 24, 2018
Publication date	Feb 9, 2021
Grant date	Feb 9, 2021

Abstract

A method of training a predictive model to predict a likely field value for one or more user selected fields within an application. The method comprises providing a user interface for user selection of the one or more user selected fields within the application; analyzing a pre-existing, user provided data set of objects; training, based on the analysis, the predictive model; determining, for each user selected field based on the analysis, a confidence function for the predictive model that identifies the percentage of cases predicted correctly at different applied confidence levels, the percentage of cases predicted incorrectly at different applied confidence levels, and the percentage of cases in which the prediction model could not provide a prediction at different applied confidence levels; and providing a user interface for user review of the confidence functions for user selection of confidence threshold levels to be used with the predictive model.

First claim

What is claimed is:

1. A prediction system configured to offer recommended field values based on an analyzed data set, the case prediction system comprising a processor configured by programming instructions encoded in non-transient computer readable media, the prediction system configurable to:
provide a user interface for user selection of one or more fields within an application for which to automatically predict a field value;
analyze a user provided data set of objects for relationships between the user selected one or more fields and content in the objects in the data set;
train, based on the analysis, a predictive model configurable to predict a likely field value for each of the user selected one or more fields when the application receives a new object and further configured to calculate a predicted confidence level for each predicted likely field value;
determine, for each user selected field based on the analysis, a confidence function for the predictive model that identifies the percentage of cases predicted correctly at different applied confidence levels, the percentage of cases predicted incorrectly at different applied confidence levels, and the percentage of cases in which the prediction model could not provide a prediction at different applied confidence levels;
provide a user interface for user review of the confidence function for a user selected field and for user selection of a confidence threshold level to be used with the predictive model for the user selected field; and
analyze an object received by the application using the predictive model and provide to the application predicted field values for user selected fields based on the user selected confidence threshold level.

2. The prediction system of claim 1 , wherein the application provides user-selectable field value choices for the fields and wherein the user-selectable field value choices are selected via a picklist, lookup or checkbox.

3. The prediction system of claim 1 , further configured to provide, for user selection via the user interface, an option for the predictive model to identify a predicted field value as a best recommendation.

4. The prediction system of claim 3 , wherein the confidence threshold level is used to determine the best recommendation, wherein predicted field values determined by the predictive model that have an associated confidence level that is below the confidence threshold level will not be recommended as a best recommendation and predicted field values determined by the predictive model that have an associated confidence level that is equal to or above the confidence threshold level will be recommended by the predictive model as a best recommendation via a visual indication.

5. The prediction system of claim 3 , further configured to provide an option, for user selection via the user interface, for the predictive model to automatically apply the best recommendation as a field value without user confirmation of the application of the best recommendation as a field value.

6. The prediction system of claim 1 , further configured to recommend the confidence threshold level.

7. The prediction system of claim 1 , further configured to provide an option, via the user interface, to activate the predictive model for use with the application.

8. A computing system comprising:
an application configurable for use in analyzing customer objects;
a predictive model configurable to analyze an object received by the application, predict a likely field value for one or more user selected fields within the application based on the object analysis, provide the predicted field values to the application, and calculate a predicted confidence level for each predicted field value; and
a model generation module configurable to:
provide a user interface for user selection of the one or more user selected fields within the application;
analyze a pre-existing, user provided data set of objects for relationships between the one or more user selected fields and content in the objects in the data set;
train, based on the analysis, the predictive model;
determine, for each user selected field based on the analysis, a confidence function for the predictive model that identifies the percentage of cases predicted correctly at different applied confidence levels, the percentage of cases predicted incorrectly at different applied confidence levels, and the percentage of cases in which the prediction model could not provide a prediction at different applied confidence levels; and
provide a user interface for user review of the confidence function for a user selected field and for user selection of a confidence threshold level to be used with the predictive model for the user selected field.

9. The computing system of claim 8 , wherein the application provides user-selectable field value choices for the fields and wherein the user-selectable field value choices are selected via a picklist, lookup or checkbox.

10. The computing system of claim 8 , wherein the model generation module is configured to provide, for user selection via the user interface, an option for the predictive model to identify a predicted field value as a best recommendation.

11. The computing system of claim 10 , wherein the confidence threshold level is used to determine the best recommendation, wherein predicted field values determined by the predictive model that have an associated confidence level that is below the confidence threshold level will not be recommended as a best recommendation and predicted field values determined by the predictive model that have an associated confidence level that is equal to or above the confidence threshold level will be recommended by the predictive model as a best recommendation via a visual indication.

12. The computing system of claim 10 , wherein the model generation module is configured to provide an option, for user selection via the user interface, for the predictive model to automatically apply the best recommendation as a field value without user confirmation of the application of the best recommendation as a field value.

13. The computing system of claim 8 , wherein the model generation module is configured to recommend the confidence threshold level.

14. The computing system of claim 8 , wherein the model generation module is configured to provide an option, via the user interface, to activate the predictive model for use with the application.

15. A method of training a predictive model to predict a likely field value for one or more user selected fields within an application, the method comprising:
providing a user interface for user selection of the one or more user selected fields within the application;
analyzing a pre-existing, user provided data set of objects for relationships between the one or more user selected fields and content in the objects in the data set;
training, based on the analysis, the predictive model;
determining, for each user selected field based on the analysis, a confidence function for the predictive model that identifies the percentage of cases predicted correctly at different applied confidence levels, the percentage of cases predicted incorrectly at different applied confidence levels, and the percentage of cases in which the prediction model could not provide a prediction at different applied confidence levels; and
providing a user interface for user review of the confidence function for a user selected field and for user selection of a confidence threshold level to be used with the predictive model for the user selected field.

16. The method of claim 15 , furth

Assignees

• Salesforce Com Inc

Inventors

• Muramoto Kristen Noriko
• Chang Son Thanh
• Tussoit Clement Jacques Antoine
• Hoang Melissa
• Malla Chaitanya
• Kjellberg Orjan N
• Jimenez Carlos Enrique Mogollan
• Hu George

Classifications

• G06F9/451 Primary

  Execution arrangements for user interfaces · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06N99/00

  Subject matter not provided for in other groups of this subclass · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F3/04847

  Interaction techniques to control parameter settings, e.g. interaction with sliders or dials · CPC title