Machine model generation systems and methods

Patent metadata

Field	Value
Publication number	US-11848101-B2
Application number	US-202117363605-A
Country	US
Kind code	B2
Filing date	Jun 30, 2021
Priority date	Apr 26, 2016
Publication date	Dec 19, 2023
Grant date	Dec 19, 2023

Abstract

A method includes defining model attributes of a machine model that organizes feedback data into topic groups based on similarities in concepts in the feedback data. The model attributes include a topic model number that defines how many topic groups are to be created, a hyperparameter optimization alpha value, and/or a hyperparameter optimization beta value. The method also includes generating the machine model using the model attributes that are defined and the feedback data, and applying the machine model to the feedback data to divide different portions of the feedback data into the different topic groups based on contents of the feedback data, the topic model number, the hyperparameter optimization alpha value, and/or the hyperparameter optimization beta value.

First claim

What is claimed is:

1. A method comprising:
defining model attributes including a training iteration value that defines a number of training iterations to be used in machine learning to associate portions of feedback data with different combinations of topic groups based on similarities in concepts conveyed in the feedback data, the feedback data provided to pharmacies from a plurality of various data sources, the feedback data having confidential identifying information removed with the confidential identifying information that was removed including one or more of pharmacy claims data, drug code numbers of medications in prescriptions filled by the pharmacies, financial costs of the medications in the prescriptions filled by the pharmacies, copay amounts required by the pharmacies under one or more prescription benefit plans, or member eligibilities under the one or more prescription benefit plans;
receiving a topic model number selection that indicates a number of the topic groups;
using machine learning to train a machine model using a defined default parameter and based on the model attributes and the topic model number selection, wherein the machine model is trained by defining different relationships between selected concepts for each of the training iterations, wherein, for each iteration of the training iterations, the feedback data is transformed by applying an ontology to reduce terms in the feedback data by eliminating duplicative entries in the feedback data and sorting the portions of the feedback data having a relationship with each other into the same topic group of the topic groups based on the topic model number selection; and
creating one or more software applications to include the machine model that is trained, the one or more software applications configured to generate a display showing one or both of a topic cluster graph and a word cloud based on the machine model that is trained.

2. The method of claim 1 , wherein the machine model that is trained is configured to automatically update the topic groups of the feedback data as updated, new, or different feedback data is provided from the plurality of various data sources.

3. The method of claim 1 , wherein the model attributes include one or more of:
a number of the topic groups in the machine model, or
a similarity requirement for the concepts conveyed in the feedback data to be included in a common topic group of the topic groups by the machine model that is trained.

4. The method of claim 1 , wherein the defined default parameter that is used to generate the machine model includes one or more of:
a previously defined number of the topic groups that the feedback data provided to the pharmacies is to be divided into by the machine model, or
a similarity requirement for concepts conveyed in the feedback data to be included in a common topic group of the topic groups by the machine model that is trained.

5. The method of claim 1 , wherein the display that is generated based on the machine model also visually presents one or more of:
a distribution of service provider feedback data among different topic groupings, or
a list of the feedback data in the selected topic group.

6. The method of claim 1 , wherein the number of training iterations defined by the training iteration value dictate how many of the different combinations of the topic groups are defined and associated with the portions of the feedback data.

7. The method of claim 1 , further comprising:
receiving a hyperparameter optimization alpha value that defines how likely one or more of the portions of the feedback data are to be included in a single topic group of the different topic groups,
wherein the machine model is trained using the defined default parameter and the hyperparameter optimization alpha value, and based on the model attributes and the topic model number selection.

8. The method of claim 1 , further comprising:
receiving a hyperparameter optimization beta value that defines how broadly each of the different topic groups are defined relative to the feedback data,
wherein the machine model is trained using the defined default parameter and the hyperparameter optimization beta value, and based on the model attributes and the topic model number selection.

9. The method of claim 1 , further comprising:
removing the confidential identifying information from the feedback data by eliminating the duplicative entries of the feedback data that are received from diverse sources in the plurality of various data sources.

10. The method of claim 1 , further comprising:
receiving the feedback data from the plurality of various data sources that include one or more of emails, surveys, website form entries, transcripts of verbally communicated information, or social media data.

11. The method of claim 1 , wherein the feedback data from the plurality of various data sources is unstructured data.

12. A method comprising:
defining model attributes including a training iteration value that defines a number of training iterations to be used in machine learning to associate portions of feedback data with different combinations of topic groups based on similarities in concepts conveyed in the feedback data, the feedback data provided to pharmacies from a plurality of various data sources, the feedback data having confidential identifying information removed with the confidential identifying information that was removed including one or more of pharmacy claims data, drug code numbers of medications in prescriptions filled by the pharmacies, financial costs of the medications in the prescriptions filled by the pharmacies, copay amounts required by the pharmacies under one or more prescription benefit plans, or member eligibilities under the one or more prescription benefit plans;
training a machine model using machine learning and a defined default parameter, the machine model trained based on the model attributes, wherein the machine model is trained by defining different relationships between selected concepts for each of the training iterations, wherein, for each iteration of the training iterations, the feedback data is transformed by applying an ontology to reduce terms in the feedback data by eliminating duplicative entries in the feedback data and sorting the portions of the feedback data having a relationship with each other into the same topic group of the topic groups; and
creating one or more software applications to include the machine model that is trained, the one or more software applications configured to generate a display showing one or both of a topic cluster graph and a word cloud based on the machine model that is trained,
wherein the number of training iterations defined by the training iteration value dictate how many of the different combinations of the topic groups are defined and associated with the portions of the feedback data with at least one of the topic groups being different in each of the training iterations.

13. The method of claim 12 , wherein the machine model that is trained is configured to automatically update the topic groups of the feedback data as updated, new, or different feedback data is provided from the plurality of various data sources.

14. The method of claim 12 , wherein the model attributes include one or more of:
a number of the topic groups in the machine model, or
a similarity requirement for the concepts conveyed in the feedback data to be included in a common topic group of the topic groups by the machine model that is trained.

15. The method of claim 12 , wherein the defined default parameter that is used to generat

Assignees

• Express Scripts Strategic Dev Inc

Inventors

• Shah Pritesh J
• Markson Christopher R
• Meltabarger Logan R

Classifications

• G16H50/20 Primary

  for computer-aided diagnosis, e.g. based on medical expert systems · CPC title

• G06N20/00

  Machine learning · CPC title

• G06Q50/22 Primary

  Social work or social welfare, e.g. community support activities or counselling services · CPC title

• G16H10/60

  for patient-specific data, e.g. for electronic patient records · CPC title

• G16H40/20

  for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms · CPC title