Identifying a deployed machine learning model

Patent metadata

Field	Value
Publication number	US-2019213503-A1
Application number	US-201815863982-A
Country	US
Kind code	A1
Filing date	Jan 8, 2018
Priority date	Jan 8, 2018
Publication date	Jul 11, 2019
Grant date	—

Abstract

A computer querying an application programming interface with each of multiple synthetic samples, each of the synthetic samples representing a separate sample assigned an original class from among multiple classes classified by a particular machine learning model and distorted to induce the particular machine learning model to misclassify the separate sample as a different class from among the classes. The computer accumulating, by the computer, a score of a number of results returned by the application programming interface that match an expected class label assignment of the different class for each of the synthetic samples. The computer, in response to the score exceeding a threshold, verifying, by the computer, that a service provided by the application programming interface is running the particular machine learning model.

First claim

What is claimed is:

1 . A method comprising:
querying, by a computer system, an application programming interface with each of a plurality of synthetic samples, each of the plurality of synthetic samples representing a separate sample assigned an original class from among a plurality of classes classified by a particular machine learning model and distorted to induce the particular machine learning model to misclassify the separate sample as a different class from among the plurality of classes; accumulating, by the computer system, a score of a number of results returned by the application programming interface that match an expected class label assignment of the different class for each of the plurality of synthetic samples; and in response to the score exceeding a threshold, verifying, by the computer system, that a service provided by the application programming interface is running the particular machine learning model.

2 . The method according to claim 1 , further comprising:
sending, by the computer system, a separate query call to the application programming interface for each of the plurality of synthetic samples, wherein a user requesting to query the application programming interface with the plurality of synthetic samples is only able to access the service through queries to the application programming interface; and receiving, by the computer system, an output from the application programming interface for each separate query call comprising a result label of one of the plurality of classes.

3 . The method according to claim 1 , wherein accumulating, by the computer system, a score of a number of results returned by the application programming interface that match an expected class label assignment of the different class for each of the plurality of synthetic samples further comprises:
accumulating, by the computer system, the score of the number of results returned by the application programming interface that match an expected class label assignment associated with each of the plurality of synthetic samples in a matrix of expected class labels, the matrix of expected class labels created from a plurality of results of applying the plurality of synthetic samples to the particular machine learning model prior to deployment.

4 . The method according to claim 3 , wherein accumulating, by the computer system, the score of the number of results returned by the application programming interface that match an expected class label assignment associated with each of the plurality of synthetic samples in a matrix of expected class labels, the matrix of expected class labels created from a plurality of results of applying the plurality of synthetic samples to the particular machine learning model prior to deployment further comprises:
in response to each result returned by the application programming interface that matches the expected class label in the matrix of expected class labels associated with a selection of the plurality of synthetic samples, updating, by the computer system, the cumulative score with a success; and in response to each result returned by the application programming interface that does not match the expected class label in the matrix of expected class labels associated with an additional selection of the plurality of synthetic samples, updating, by the computer system, the cumulative score with lack of success.

5 . The method according to claim 1 , further comprising:
receiving, by the computer system, a selection from a user of a percentage probability of certainty requested; and dynamically adjusting, by the computer system, the threshold to a level that requires the score to reach a level of certainty that the service provided by the application programming interface is running the particular machine learning model reaches the percentage probability of certainty requested.

6 . The method according to claim 1 , further comprising:
creating, by the computer system, a cohort set of a plurality of additional machine learning models of one or more configuration that classify the same plurality of classes as the particular machine learning model; running, by the computer system, the plurality of synthetic samples on each of the plurality of additional machine learning models; for each of the plurality of additional machine learning models, accumulating, by the computer system, a separate score of a separate number of results that match the expected class label assignment of the different class for each of the plurality of synthetic samples; and applying, by the computer system, one or more calibration rules to each separate score to calibrate the threshold to assess the likelihood that the service provided by the application programming interface is running the particular machine learning model.

7 . The method according to claim 1 , wherein querying, by the computer system, an application programming interface with each of a plurality of synthetic samples, each of the plurality of synthetic samples representing a separate sample assigned an original class from among a plurality of classes classified by a particular machine learning model and distorted to induce the particular machine learning model to misclassify the separate sample as a different class from among the plurality of classes further comprising:
querying, by the computer system, the application programming interface with each of the plurality of synthetic samples as normal, valid inputs to the application programming interface that are not detectable by the application programming interface as test inputs to verify an identity of the particular machine learning model deployed and running behind the application programming interface.

8 . The method according to claim 1 , wherein in response to the score exceeding a threshold, verifying, by the computer system, that a service provided by the application programming interface is running the particular machine learning model further comprises:
in response to the score exceeding a threshold, verifying, by the computer system, by a percentage of probability associated with the threshold, that the service provided by the application programming interface is running the particular machine learning model.

9 . A computer system comprising one or more processors, one or more computer-readable memories, one or more computer-readable storage devices, and program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:
program instructions to query an application programming interface with each of a plurality of synthetic samples, each of the plurality of synthetic samples representing a separate sample assigned an original class from among a plurality of classes classified by a particular machine learning model and distorted to induce the particular machine learning model to misclassify the separate sample as a different class from among the plurality of classes; program instructions to accumulate a score of a number of results returned by the application programming interface that match an expected class label assignment of the different class for each of the plurality of synthetic samples; and program instructions, in response to the score exceeding a threshold, to verify that a service provided by the application programming interface is running the particular machine learning model.

10 . The computer system according to claim 8 , wherein the program instructions further comprise:
program instructions to send a separate query call to the application programming interface for each of the plurali

Assignees

• IBM

Inventors

• Navratil Jiri
• Murdock James W

Classifications

• G06N20/20

  Ensemble learning · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F16/90335

  Query processing · CPC title

• G06F9/541

  via adapters, e.g. between incompatible applications · CPC title

• G06F16/583 Primary

  using metadata automatically derived from the content · CPC title