Systems and Methods for Efficient Data Preprocessing of Machine Learning Workloads
US-2024403138-A1 · Dec 5, 2024 · US
US9727383B2 · US · B2
| Field | Value |
|---|---|
| Publication number | US-9727383-B2 |
| Application number | US-201213400579-A |
| Country | US |
| Kind code | B2 |
| Filing date | Feb 21, 2012 |
| Priority date | Feb 21, 2012 |
| Publication date | Aug 8, 2017 |
| Grant date | Aug 8, 2017 |
A practical reading order for non-experts. Skip the full description unless you need deep technical detail.
What the patent document calls the invention.
A short plain-language summary of the technical disclosure.
Who owns or filed the patent and who is credited as inventor.
Filing, priority, publication, and grant dates set the timeline.
The legal scope of protection — read this for what is actually claimed.
Technology tags used to group this patent with similar filings.
Prior art links and similar publications in this corpus.
Official abstract text for this publication.
Methods of predicting datacenter performance to improve provisioning are described. In an embodiment, a resource manager element receives a request from a tenant which describes an application that the tenant wants executed by a multi-resource, multi-tenant datacenter. The request that has been received is mapped to a set of different candidate resource combinations within the datacenter, where each candidate resource combination can be used to execute the application in a manner which satisfies a high level constraint specified within the request. This mapping may, for example, be performed using a combination of benchmarking and an analytical model. In some examples, each resource combination may comprise a number of virtual machines and a bandwidth between those machines. Data relating to at least a subset (and in some examples, two or more) of the candidate resource combinations is then presented to the tenant.
Opening claim text (preview).
The invention claimed is: 1. A method comprising: receiving, by a datacenter provider, a user request comprising details of an application to be executed by a multi-resource, multi-tenant datacenter and a high level constraint associated with the application; mapping the user request to a set of different candidate resource combinations within the datacenter, each candidate resource combination specifying at least a number of virtual machines and a network bandwidth between virtual machines for executing the application to satisfy the high level constraint; presenting data relating to at least a subset of the candidate resource combinations to the datacenter provider to enable the datacenter provider to select one of the at least a subset of the candidate resource combinations to execute the application; and, selecting one of the at least a subset of the candidate resource combinations based on a value of a metric computed for each of the different candidate resource combinations, the metric describing an impact on the ability of the datacenter to accommodate subsequent requests after allocating the candidate resource combination based at least in part on an imbalance in utilization across all resources, wherein the imbalance is calculated as the summation of the square of the quotient of dividing the number of unallocated VM slots by the total number of VM slots and the square of the quotient of dividing unallocated outbound link capacity by the total outbound link capacity. 2. The method according to claim 1 , further comprising: selecting the subset of the candidate resource combinations from the set of different candidate resource combinations, and wherein data relating to the selected subset of the candidate resource combinations is presented to the datacenter provider. 3. The method according to claim 2 , wherein the subset comprises a single candidate resource combination and the method further comprising: sending a message to cause the selected candidate resource combination to be automatically allocated to the user for executing the application. 4. The method according to claim 1 , further comprising: computing a cost to the user associated with each candidate resource combination; and wherein the data relating to a candidate resource combination comprises the computed cost to the user for the candidate resource combination. 5. The method according to claim 4 , wherein the cost is computed based on a value of a metric computed for the candidate resource combination and wherein the metric describes an impact on the datacenter of allocating the candidate resource combination. 6. The method according to claim 1 , further comprising: receiving a datacenter provider input selecting one of the candidate resource combinations presented to the datacenter provider; and sending a message to cause the selected candidate resource combination to be allocated to the user for executing the application. 7. The method according to claim 1 , wherein the mapping is performed using datacenter state information. 8. The method according to claim 1 , wherein mapping the user request to a set of different candidate resource combinations within the datacenter comprises: profiling the application using a sample of data on a sample machine to determine one or more parameters; and using an analytical model and the one or more parameters to identify the set of different candidate resource combinations which satisfy the high level constraint. 9. The method according to claim 1 , wherein the high level constraint is defined in terms of at least one of performance and cost. 10. The method according to claim 1 , wherein the candidate resource combination further comprises a time parameter. 11. A method comprising: receiving, by a datacenter provider, a user request comprising details of an application to be executed by a multi-resource, multi-tenant datacenter and high level constraint associated with the application; mapping the user request to a set of different candidate resource combinations within the datacenter, each candidate resource combination specifying a combination of datacenter resources for executing the application to satisfy the high level constraint; presenting data relating to two or more candidate resource combinations to the datacenter provider, the data comprising a metric describing an imbalance of resources across the datacenter, the metric being calculated based at least in part on a summation of the square of the quotient of dividing the number of unallocated VM slots by the total number of VM slots and the square of the quotient of dividing unallocated outbound link capacity by the total outbound link capacity, presenting data relating to two or more candidate combinations enabling the datacenter provider to associate a price with each of the presented two or more candidate combinations; and, presenting the data relating to two or more candidate combinations and the associated prices to the user, presenting the data relating to two or more candidate combinations and associated prices enabling the user to select one of the two or more candidate resource combinations to execute the application. 12. The method according to claim 11 , further comprising: selecting the two or more candidate resource combinations from the set of different candidate resource combinations. 13. The method according to claim 12 , wherein the two or more candidate resource combinations is selected based on a value of a second metric computed for each of the different candidate resource combinations and wherein the second metric describes an impact on the datacenter of allocating a candidate resource combination. 14. The method according to claim 11 , further comprising: receiving a user input selecting one of the candidate resource combinations for which related data was presented to the user; and sending a message to cause the selected candidate resource combination to be allocated to the user for executing the application. 15. The method according to claim 11 , wherein mapping the user request to a set of different candidate resource combinations within the datacenter comprises: profiling the application using a sample of data on a sample machine to determine one or more parameters; and using an analytical model and the one or more parameters to identify the set of different candidate resource combinations which satisfy the high level constraint. 16. One or more storage media storing device-executable instructions that, when executed by a computing system, direct the computing system to perform steps comprising: receiving a user request comprising details of a data analytics job to be performed by a datacenter and a high level constraint associated with the job; mapping the user request to a set of different candidate resource combinations within the datacenter, each candidate resource combination specifying at least a number of virtual machines and a network bandwidth between virtual machines for executing the data analytics job to satisfy the high level constraint; presenting data relating to two or more candidate resource combinations to the datacenter provider, the data comprising a metric describing an imbalance of resources across the datacenter, the metric being calculated based at least in part on a summation of the square of the quotient of dividing the number of unallocated VM slots by the total number of VM slots and the square of the quotient of dividing unallocated outbound link capacity by the total outbound link capacity, presenting including enabling the datacenter provider to
considering hardware capabilities · CPC title
Monitor · CPC title
Operations research, analysis or management · CPC title
Partitioning or combining of resources · CPC title
Related publications grouped by family.
Answers are generated from the same data shown on this page.