Crowd-sourced artificial intelligence image processing services

Patent metadata

Field	Value
Publication number	US-10360482-B1
Application number	US-201715830952-A
Country	US
Kind code	B1
Filing date	Dec 4, 2017
Priority date	Dec 4, 2017
Publication date	Jul 23, 2019
Grant date	Jul 23, 2019

Abstract

Features related to systems and methods for generating a machine learning model that is a composite of at least two other models (e.g., crowd-sourced models contributed by users) are described. Each of the contributed models provide output values that may not be to scale. To account for these differences, a normalization factor for a first machine learning model is generated to adjust values produced by the first machine learning model to correspond with results from the second machine learning model. The crowd-sourced models along with the normalization factor are included in the new image model generated in the claims.

First claim

What is claimed is:

1. A computer-implemented method comprising:
under control of one or more processors,
receiving, from an electronic communication device, a request for an image processing model, wherein the request identifies:
(i) an object type to be identified by the image processing model, and
(ii) training data including a first image showing a first object of the object type and a second image showing a second object of the object type;

identifying, from a library of machine learning models, a first machine learning model based on the object type identified in the request;
identifying, from the library of machine learning models, a second machine learning model based on the object type identified in the request;
generating a first image processing result identifying the first object using the first machine learning model, wherein the first image processing result includes a first prediction confidence value;
generating a second image processing result identifying the second object using the second machine learning model, wherein the second image processing result includes a second prediction confidence value;
determining a normalization factor for the first machine learning model based on a comparison between the first prediction confidence value and the second prediction confidence value;
generating the image processing model using the first machine learning model, the second machine learning model, and the normalization factor,
wherein the image processing model provides an input image to each of the first machine learning model and the second machine learning model, and
wherein generating the image processing model includes:
generating a normalization layer that:
 (i) receives image processing results from the first machine learning model, and
 (ii) provides normalized image processing results based on the normalization factor, and
generating an output layer that:
 (i) receives image processing results from the second machine learning model,
 (ii) receives the normalized image processing results from the normalization layer, and
 (iii) provides a final image processing result for the input image; and

storing the image processing model in the library of machine learning models.

2. The computer-implemented method of claim 1 , further comprising:
storing interaction information for the image processing model, wherein interaction information for a specific interaction identifies which of the first machine learning model or the second machine learning model provided a highest confidence result for the specific interaction with the image processing model; and
determining an attribution amount for the first machine learning model, wherein the attribution amount based on a number of interactions identified by the interaction information which the first machine learning model provided the highest confidence result.

3. The computer-implemented method of claim 2 , further comprising storing the attribution amount for the first machine learning model and another attribution amount for a third machine learning model, and
wherein identifying the first machine learning model comprises:
determining that the first machine learning model and the third machine learning model provide equivalent predictions;
determining that the attribution amount for the first machine learning model exceeds the another attribution amount for the third machine learning model; and
selecting the first machine learning model.

4. The computer-implemented method of claim 1 , further comprising:
receiving, from an electronic computing device, training image data, wherein the training image data includes metadata describing an object type shown in images included in the training image data;
adjusting color channel values for pixels included in the images to generate scrambled training image data; and
training the first machine learning model using the scrambled training image data,
wherein generating the image processing model includes:
generating a scrambling layer that:
(i) receives the input image directed to the first machine learning model,
(ii) adjusts the color channel values for pixels included in the input image to generate a scrambled input image, and
(iii) provides the scrambled input image to the first machine learning model.

5. The computer-implemented method of claim 1 , further comprising:
receiving, from the electronic communication device, an image for processing by the image processing model;
retrieving the image processing model from the library;
processing the image using the image processing model to generate an image processing result, the image processing result including at least one of segmentation information or classification information for an object shown in the image; and
transmitting the image processing result to the electronic communication device.

6. A system comprising:
one or more computing devices having a processor and a memory, wherein the one or more computing devices execute computer-readable instructions to:
receive, from an electronic computing device, a request for an image model, the request indicating an object type to be identified by the image model;
select, from a library of machine learning models, a first machine learning model and a second machine learning model associated with the object type;
generate the image model using the first machine learning model, the second machine learning model, and a normalization factor for image processing results of the first machine learning model,
wherein the image model provides an input image to each of the first machine learning model and the second machine learning model, and
wherein generating the image model includes:
generating a normalization layer that:
(i) receives image processing results from the first machine learning model, and
(ii) provides normalized image processing results based on the normalization factor; and

generating an output layer that:
(i) receives image processing results from the second machine learning model,
(ii) receives the normalized image processing results from the normalization layer, and
(iii) provides a final image processing result for the input image; and

store the image model in the library of machine learning models.

7. The system of claim 6 , wherein the one or more computing devices execute computer-readable instructions to:
generate a first image processing result identifying a first object in a first image using the first machine learning model, wherein the first image processing result includes a first prediction confidence value;
generate a second image processing result identifying a second object in a second image using the second machine learning model, wherein the second image processing result includes a second prediction confidence value; and
determine the normalization factor for the first machine learning model based on a comparison between the first prediction confidence value and the second prediction confidence value.

8. The system of claim 6 , wherein the request includes information identifying: (i) a characteristic for the image model to identify, wherein the characteristic comprises one of: detection of an object within an image or pixels representing the object with the image, and (ii) a set of reference images.

9. The system of claim 6 , wherein the one or more computing devices execute computer-readable instructions to:
store interaction information for the image model, wherein interaction information for a specific interaction identifies which of the first machine learning model or the second machine learning model provided a highest confidence result for the specific interaction with the image model; and
determine an att

Assignees

• Amazon Tech Inc

Inventors

• Khare Vineet
• Swaminathan Gurumurthy
• Zhou Xiong

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06V30/19167

  Active pattern learning · CPC title

• G06V10/87

  using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F18/2185

  the supervisor being an automated module, e.g. intelligent oracle · CPC title