Provisioning information technology (IT) infrastructures based on images of system architecture diagrams

Patent metadata

Field	Value
Publication number	US-10997409-B1
Application number	US-201816001618-A
Country	US
Kind code	B1
Filing date	Jun 6, 2018
Priority date	Jun 6, 2018
Publication date	May 4, 2021
Grant date	May 4, 2021

Abstract

Techniques are described for using machine learning (ML) models to create information technology (IT) infrastructures at a service provider network based on image of IT system architecture diagrams. To create IT system architecture diagrams, system architects often use tools ranging from pen and paper and whiteboards to various types of software-based drawing programs. Based on a user-provided image of an IT system architecture diagram (for example, a digital scan of a hand drawn system diagram, an image file created by a software-based drawing program, or the like), a service provider network uses one or more ML models to analyze the image to identify the constituent elements of the depicted IT system architecture and to create an infrastructure template that can be used to automatically provision corresponding computing resources at the service provider network.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving an image depicting a diagram of an information technology (IT) system;
using at least one machine learning (ML) model to:
identify a plurality of graphical elements in the image each representing a component of the IT system, and
determine a respective type of computing resource to be used to implement each of the components of the IT system represented by the plurality of graphical elements identified in the image;

generating an infrastructure template describing the respective type of computing resource to be used to implement each of the components of the IT system represented by the graphical elements identified in the image; and
configuring a plurality of computing resources at a service provider network based on the infrastructure template.

2. The computer-implemented method of claim 1 , wherein the image includes at least one graphical element representing one of: a physical server, a virtual machine (VM) instance, a container, a serverless function, a load balancer, a database, an object storage resource, an in-memory data storage resource, a logging resource, a public subnet, a private subnet, a virtual private cloud (VPC), a routing table, a security group configuration, an auto scaling group, a software configuration.

3. The computer-implemented method of claim 1 , wherein at least one graphical element of the plurality of graphical elements in the image is associated with an annotation, the annotation including one or more of: an identifier of a type of computing resource represented by the at least one graphical element, an identifier of a number of computing resources represented by the at least one graphical element, and one or more configuration settings associated with a computing resource represented by the at least one graphical element.

4. A computer-implemented method comprising:
receiving an image depicting a diagram of an information technology (IT) system;
analyzing the image using at least one machine learning (ML) model to create a structured data representation of the IT system, the structured data representation of the IT system indicating types of computing resources to be used to implement the IT system at a service provider network, and using the at least one ML model to include in the structured data representation at least one system configuration that is not depicted in the image; and
configuring a plurality of computing resources at the service provider network based on the structured data representation of the IT system.

5. The computer-implemented method of claim 4 , wherein analyzing the image using the at least one ML model to create the structured data representation of the IT system includes:
identifying a plurality of graphical elements in the image each representing a component of the IT system; and
determining a respective type of computing resource to be used to implement each of the components of the IT system represented by the graphical elements identified in the image.

6. The computer-implemented method of claim 4 , wherein the structured data representation is an infrastructure template that describes a respective type of computing resource to be used to implement components of the IT system represented by the diagram.

7. The computer-implemented method of claim 4 , wherein the image includes at least one graphical element representing one or more of: a physical server, a virtual machine (VM) instance, a container, a serverless function, a load balancer, a database, an object storage resource, an in-memory data storage resource, a logging resource, a public subnet, a private subnet, a virtual private cloud (VPC), a routing table, a security group, an auto scaling group, a software configuration.

8. The computer-implemented method of claim 4 , wherein the image includes at least one graphical element that is associated with an annotation, the annotation including one or more of: an identifier of a type of computing resource represented by the at least one graphical element, an identifier of a number of computing resources represented by the at least one graphical element, one or more configurations associated with a computing resource represented by the at least one graphical element.

9. The computer-implemented method of claim 4 , wherein the ML model is unable to identify at least one graphical element included in the image, the method further comprising:
receiving user input indicating a type of computing resource represented by the at least one graphical element; and
using the user input to train the ML model to recognize the at least one graphical element.

10. The computer-implemented method of claim 4 , wherein the ML model is a neural network.

11. The computer-implemented method of claim 4 , further comprising training the ML model using training data including one or more of graphical element-to-computing resource type mappings, computing resource type-to-service provider network computing resource mappings, computing resource type-to-best practices settings mappings.

12. The computer-implemented method of claim 4 , further comprising:
receiving operational parameters for the IT system; and
wherein determining the respective type of computing resource to be used to implement each of the components of the IT system represented by a plurality of graphical elements identified in the image is based at least in part on the operational parameters.

13. The computer-implemented method of claim 4 , further comprising:
receiving input representing a single graphical element of the diagram;
using the at least one ML model to identify a type of computing resource represented by the single graphical element of the diagram; and
sending a standardized graphical element representing the type of computing resource identified by the ML model.

14. A system comprising:
an information technology (IT) infrastructure modeling service implemented by a first one or more electronic devices, the IT infrastructure modeling service comprising instructions which, when executed by the first one or more electronic devices, cause the IT infrastructure modeling service to:
receive an image depicting a diagram of an IT system,
send the image to a machine learning (ML) service,
receive a structured data representation of the IT system from the ML service, and
configure a plurality of computing resources at a service provider network based on the structured data representation of the IT system; and

a machine learning (ML) service implemented by a second one or more electronic devices, the ML service including instructions that upon execution cause the ML service to:
receive the image depicting an IT system architecture,
identify a plurality of graphical elements in the image each representing a component of the IT system,
determine a respective type of computing resource to be used to implement each of the components of the IT system represented by the plurality of graphical elements, and
generate a structured data representation of the IT system, the structured data representation indicating types of computing resources to be used to implement the IT system at the service provider network.

15. The system of claim 14 , wherein the image includes at least one graphical element representing one or more of: a physical server, a virtual machine (VM) instance, a container, a serverless function, a load balancer, a database, an object storage resource, an in-memory data storage resource, a logging resource, a public subnet, a private subnet, a virtual private cloud (VPC), a routing table, a secur

Assignees

• Amazon Tech Inc

Inventors

• Krishnamurthy Sandeep
• Singh Rajankumar
• Markham Aaron
• Wei Lai

Classifications

• G06V30/19133

  Interactive pattern learning with a human teacher · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06V30/19147

  Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T11/26

  Drawing of charts or graphs · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title