Versioning of Trained Models Used To Deliver Cognitive Services

Patent metadata

Field	Value
Publication number	US-2016358098-A1
Application number	US-201514730537-A
Country	US
Kind code	A1
Filing date	Jun 4, 2015
Priority date	Jun 4, 2015
Publication date	Dec 8, 2016
Grant date	—

Abstract

A method, system, and/or computer program product manages the lifecycle of trained models used to deliver cognitive services. One or more processors obtain and deploy a cognitive engine that utilizes artificial intelligence (AI), machine learning, and/or similar algorithms. One or more processors obtain and deploy a version of a trained model that includes data that supports cognitive operations of the cognitive engine within a cognitive service. In response to changes to the input used to produce the trained model, one or more processors obtain and deploy a subsequent version of the trained model in support of the cognitive service.

First claim

What is claimed is:

1 . A computer implemented method of managing versions of trained models used to deliver cognitive services, the computer implemented method comprising:
obtaining and deploying, by one or more processors, a cognitive engine to a cognitive service, wherein the cognitive engine utilizes machine learning algorithms; obtaining and deploying, by one or more processors, a first trained model to the cognitive service, wherein the first trained model is created by a training engine in a training pipeline, and wherein the training engine in the training pipeline utilizes training data to create the first trained model; and in response to a change to a training input used by the training pipeline, obtaining and deploying, by one or more processors, a second trained model to the cognitive service, wherein the second trained model incorporates the change to the training input used by the training pipeline.

2 . The computer implemented method of claim 1 , wherein the change to the training input is a change to the training data that was used by the training engine to create the first trained model.

3 . The computer implemented method of claim 1 , wherein the change to the training input is a change to software code that defines the training engine.

4 . The method of claim 1 , further comprising:
obtaining, by one or more processors, multiple trained models using disparate data; ranking, by one or more processors, the multiple trained models according to an effectiveness level in enabling the cognitive engine to reach predefined goals; and deploying, by one or more processors, a highest-ranked trained model from the multiple trained models to the cognitive service.

5 . The method of claim 1 , further comprising:
receiving, by one or more processors, a request for a new service from the cognitive service; and triggering, by one or more processors, generation of the second trained model by the request for the new service from the cognitive service.

6 . The method of claim 1 , further comprising:
receiving, by one or more processors, feedback indicating that the cognitive service is not meeting predefined performance parameters; and in response to receiving feedback indicating that the cognitive service is not meeting the predefined performance parameters, obtaining and deploying, by one or more processors, a third trained model to the cognitive service.

7 . The method of claim 1 , further comprising:
receiving, by one or more processors, feedback indicating that the cognitive service is not meeting predefined performance parameters; and in response to receiving feedback indicating that the cognitive service is not meeting the predefined performance parameters, retiring, by one or more processors, the second trained model.

8 . The method of claim 1 , further comprising:
receiving, by one or more processors, feedback indicating that the cognitive service is not meeting predefined performance parameters; and in response to receiving feedback indicating that the cognitive service is not meeting the predefined performance parameters, retiring, by one or more processors, the cognitive engine.

9 . The method of claim 1 , further comprising:
applying, by one or more processors, a statistical analysis to the change to the training input; and in response to determining by the statistical analysis that the change to the training input exceeds a predefined limit, triggering, by one or more processors, generation of an alternate trained model.

10 . The method of claim 1 , further composing:
applying, by one or more processors, a machine learning technique to determine a change to predetermined characteristics of the training input; and in response to determining by the machine learning technique that a degree of change to the training input exceeds a predefined limit, triggering, by one or more processors, generation of an alternate trained model.

11 . The method of claim 1 , further comprising:
receiving, by one or more processors, an electronic signal from a hardware sensor, wherein the electronic signal describes an environment of the hardware sensor, and wherein the second trained model is generated using information from the electronic signal; and in response to the electronic signal from the hardware sensor indicating that the cognitive service is not meeting predefined performance parameters, obtaining and deploying, by one or more processors, a third trained model to the cognitive service.

12 . The method of claim 1 , further comprising:
receiving, by one or more processors, an electronic signal from a hardware sensor, wherein the electronic signal describes an environment of the hardware sensor, and wherein the second trained model is generated using information from the electronic signal; and in response to the electronic signal from the hardware sensor indicating that the cognitive service is not meeting predefined performance parameters, obtaining and deploying, by one or more processors, an alternate cognitive engine paired with new training data that supports the alternate cognitive engine.

13 . A computer program product for versioning trained models used to deliver cognitive services, the computer program product comprising a non-transitory computer readable storage medium having program code embodied therewith, the program code readable and executable by a processor to perform a method comprising:
obtaining and deploying a cognitive engine to a cognitive service, wherein the cognitive engine utilizes machine learning algorithms; obtaining and deploying a first trained model to the cognitive service, wherein the first trained model is created by a training engine in a training pipeline, and wherein the training engine in the training pipeline utilizes training data to create the first trained model; and in response to a change to a training input used by the training pipeline, obtaining and deploying a second trained model to the cognitive service, wherein the second trained model incorporates the change to the training input used by the training pipeline.

14 . The computer program product of claim 13 , wherein the change to the training input is a change to the training data that was used by the training engine to create the first trained model.

15 . The computer program product of claim 13 , wherein the change to the training input is a change to software code that defines the training engine.

16 . The computer program product of claim 13 , wherein the method further comprises:
generating multiple trained models using disparate data; ranking the multiple trained models according to an effectiveness level in enabling the cognitive engine to reach predefined goals; and deploying a highest-ranked trained model from the multiple trained models to the cognitive service.

17 . The computer program product of claim 13 , wherein the method further comprises:
receiving a request for a new service from the cognitive service; and triggering generation of the second trained model by the request for the new service from the cognitive service.

18 . The computer program product of claim 13 , wherein the method further comprises:
receiving feedback indicating that the cognitive service is not meeting predefined performance parameters; and in response to receiving feedback indicating that the cognitive service is not meeting the predefined performance parameters, retiring the second trained model.

19 . The computer program product of claim 13 , wherein the met

Assignees

• IBM

Inventors

• Duesterwald Evelyn
• Ebling Maria R
• Fong Liana L

Classifications

• G06F8/71

  Version control (security arrangements therefor G06F21/57); Configuration management · CPC title

• G06F9/44505

  Configuring for program initiating, e.g. using registry, configuration files · CPC title

• G06N20/20

  Ensemble learning · CPC title

• G06N99/005 Primary

  Physics · mapped topic

• G06N20/00 Primary

  Machine learning · CPC title