Intelligent decision support system

Patent metadata

Field	Value
Publication number	US-11386358-B2
Application number	US-201916386402-A
Country	US
Kind code	B2
Filing date	Apr 17, 2019
Priority date	Apr 17, 2019
Publication date	Jul 12, 2022
Grant date	Jul 12, 2022

Abstract

Various embodiments are provided for implementing intelligent decision support system in a computing environment by a processor. Data of historical decisions may be collected and examples of decisions by domain experts may be generated. One or more machine learning models may be generated using different splits of the historical data and the annotated data. The one or more machine learning models may be combined and used to generate ensemble machine learning models that generate recommendations for the decisions. Users interact with a user interface displaying the data, recommendations, reasons for recommendations and a conversational dialog system for querying about the data, recommendations and guidance for decision making.

First claim

What is claimed is:

1. A method for implementing intelligent decision support for decision making in a computing environment by a processor, comprising:
collecting data about a plurality of historical data;
generating one or more samples of annotated data by one or more domain experts;
generating one or more machine learning models using different splits of the data and the annotated data to generate recommendations for one or more decisions;
combining the one or more machine learning models to generate a recommendation for the one or more decisions using one or more ensemble machine learning models;
generating one or more natural language reasons for the recommendations by one or more machine learning models and providing one or more reasons for the recommendation using a dialog system and
performing each of the following upon the one or more machine learning models being an ensemble of decision trees:
for each constituent decision tree model in the ensemble, determining a set of machine learning rules that were used in a path by a decision tree to arrive at an output recommendation;
mapping each machine learning rule in the path by each of the constituent decision tree models to a natural language reason;
for each data point for which the recommendations are being generated, creating a set of reasons by computing a union of a set of natural language reasons mapped from paths followed by a subset of the constituent decision tree models that participated in a final recommendation by the ensemble of decision trees for the data point; and

for each of the data points for which recommendations are being generated, displaying the set of natural language reasons as the reasons for the recommendation by the ensemble machine learning model; and
interacting with a user via the dialog system and engaging in a dialog with the user in relation to the questions about the data and the recommendation.

2. The method of claim 1 , further including:
displaying structured and unstructured data via the dialog system pertaining to one or more entities and information from one or more data sources;
displaying an analysis of decisions up to a current time period;
providing evidence for the one or more reasons for the recommendations;
engaging in a series of queries with the user in relation to the data, the recommendation, or a combination thereof;
enabling the user to enter or validate data; and
grouping one or more attributes of one of a plurality of entities to reflect one or more priorities of an entity relating to the data, the recommendation, or a combination thereof.

3. The method of claim 1 , further including:
automatically determining an intended entity or group of entities pertaining to requested information by the user via the dialog system;
automatically retrieving the requested information about the intended entity or the group of entities;
identifying specific data about the intended entity or group of entities according to one or more policies in effect and specific queries by the users; and
providing generic data applicable to a plurality of users interacting with the dialog system according to the one or more policies in effect and specific queries by the user.

4. The method of claim 1 , further including:
selecting sampled data sets to be used by machine learning models based on a degree of similarity with a reference data set and using the sampled data set as additional data for training the machine learning models;
determining the degree of similarity between data points in the reference data set and the data points in the sampled data sets;
determining the degree of similarity between all pairs of data points in the reference data set;
determining a smallest value, a largest value, and an average values of the degree of similarity between the pairs of data points in the reference data set; and
selecting the data points from the sampled data sets if the degree of similarity is lower than the smallest value, the largest value, or the average values of the degree of similarity between any two data points in the reference data set.

5. The method of claim 1 , further including:
collecting the historical data;
generating the annotated data using one or more domain experts;
deriving compatible semantics across historical data and the annotated data adjusting for differences;
deriving different splits of the historical data and the annotated data;
training the machine learning models for generating one or more recommendations using the different splits; and
creating ensemble machine learning models by combining the one or more recommendations from each of the machine learning models to generate the recommendation.

6. The method of claim 1 , further including initiating a machine learning to perform one or more machine learning operations to perform a semantic analysis, train a classifier, learn one or more machine learning rules, learn contextual data associated with the dialog system, learn and train the one or more prediction models using the historical data and the annotated data, generate one or more recommendations or predictions from the one or more prediction models, and assist with engaging in communication using the dialog system, or perform a combination thereof.

7. A system for implementing intelligent decision support for decision making in a computing system, comprising:
one or more computers with executable instructions that when executed cause the system to:
collect data about a plurality of historical data;
generate one or more samples of annotated data by one or more domain experts;
generate one or more machine learning models using different splits of the data and the annotated data to generate recommendations for one or more decisions;
combine the one or more machine learning models to generate a recommendation for the one or more decisions using one or more ensemble machine learning models;
generate one or more natural language reasons for the recommendations by one or more machine learning models and providing one or more reasons for the recommendation using a dialog system and
perform each of the following upon the one or more machine learning models being an ensemble of decision trees:
for each constituent decision tree model in the ensemble, determining a set of machine learning rules that were used in a path by a decision tree to arrive at an output recommendation;
mapping each machine learning rule in the path by each of the constituent decision tree models to a natural language reason;
for each data point for which the recommendations are being generated, creating a set of reasons by computing a union of a set of natural language reasons mapped from paths followed by a subset of the constituent decision tree models that participated in a final recommendation by the ensemble of decision trees for the data point; and
for each of the data points for which recommendations are being generated, displaying the set of natural language reasons as the reasons for the recommendation by the ensemble machine learning model; and

interact with a user via the dialog system and engaging in a dialog with the user in relation to the questions about the data and the recommendation.

8. The system of claim 7 , wherein the executable instructions:
display displaying structured and unstructured data via the dialog system pertaining to one or more entities and information from one or more data sources;
display an analysis of decisions up to a current time period;
provide evidence for the one or more reasons for the recommendations;
engage in a series of queries with the user in relation to the data, the recommendation, or a combination thereof;
enable the user to enter or validate data; and
g

Assignees

• IBM

Inventors

• Sheopuri Anshul
• Rieder Binny
• Arya Pooja
• Lamoreaux Nickle
• Daly Joanna
• Mukhopadhyay Saikat
• Dasgupta Priyanka
• Kambhatla Nandakishore
• Telang Aditya
• Agrawal Priyanka

Classifications

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F16/9035

  Filtering based on additional data, e.g. user or group profiles · CPC title

• G06N5/025

  Extracting rules from data · CPC title

• G06F40/56 Primary

  Natural language generation · CPC title