Retrieval of authoritative measurement data from in-memory datastores

Patent metadata

Field	Value
Publication number	US-9910755-B2
Application number	US-201514752756-A
Country	US
Kind code	B2
Filing date	Jun 26, 2015
Priority date	Jun 26, 2015
Publication date	Mar 6, 2018
Grant date	Mar 6, 2018

Abstract

A computing resource monitoring service receives a request to retrieve measurement data for a metric. The computing resource monitoring service determines, based at least in part on information included in the request, one or more in-memory datastores where the measurement data is stored. If the information indicates that the computing resource monitoring service is to provide an authoritative subset of the measurement data, the computing resource monitoring service limits the data provided in response to the request to the authoritative subset of the measurement data. Otherwise, if the information does not specify this indication, the computing resource monitoring service will provide the measurement data that is available.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, from a customer computer system of a customer of the computing resource service provider, a first application programming interface call to retrieve a first set of measurements for a metric, the first application programming interface call including a fully qualified metric identifier generated by a hash function and metadata associated with the first set of measurements for the metric, and specifying a time range for the first set of measurements and a parameter indicating that the first set of measurements are to be authoritative;
determining, based at least in part on the fully qualified metric identifier and the specified time range for the first set of measurements, one or more datastores where the first set of measurements are stored, wherein at least a portion of the fully qualified metric identifier identifies the one or more datastores, while another portion of the fully qualified metric identifier is used for unique identification of the first set of measurements within the one or more datastores;
obtaining the first set of measurements from the determined one or more datastores where the first set of measurements are stored;
determining if the first set of measurements obtained from the determined one or more datastores are authoritative;
as a result of the first set of measurements being authoritative, providing the first set of measurements in response to the first application programming interface call;
receiving, from the customer computer system, a second application programming interface call to retrieve a second set of measurements for the metric, the second application programming interface call including the fully qualified metric identifier, and specifying a time range for the second set of measurements and a parameter indicating that the second set of measurements are to be obtained regardless of whether a time period for storage of measurements for the metric has elapsed;
determining, based at least in part on the fully qualified metric identifier and the specified time range for the second set of measurements, one or more datastores where the second set of measurements are stored, wherein at least a portion of the fully qualified metric identifier identifies the one or more datastores, while another portion of the fully qualified metric identifier is used for unique identification of the second set of measurements within the one or more datastores;
obtaining the second set of measurements from the determined one or more datastores where the second set of measurements are stored; and
providing the second set of measurements in response to the second application programming interface call.

2. The computer-implemented method of claim 1 , further comprising:
receiving, from the customer computer system, a third application programming interface call to store a measurement for a second metric obtained within a time period for the second metric, the third application programming interface call specifying a parameter indicating that the measurement to be stored is authoritative for the time period;
determining a fully qualified metric identifier for the measurement for the second metric;
determining, based at least in part on the fully qualified metric identifier for the measurement for the second metric, a datastore for the measurement for the second metric;
storing the measurement for the second metric within the determined datastore; and
as a result of the third application programming interface call specifying the parameter indicating that the measurements to be stored is authoritative for the time period, making the measurement for the second metric available for retrieval and indicating that the measurement for the second metric is authoritative for the time period.

3. The computer-implemented method of claim 2 , further comprising:
selecting, based at least in part on the fully qualified metric identifier for the measurement for the second metric, a logical partition of a plurality of logical partitions for placement of the measurement;
obtaining, from a metric mapping registry and based at least in part on a timestamp for the measurement for the second metric, a mapping of the plurality of logical partitions to a plurality of in-memory datastores; and
transmitting the measurement for the second metric from the logical partition to a corresponding aggregator sub-system comprising the in-memory datastore based at least in part on the mapping.

4. The computer-implemented method of claim 2 , further comprising updating a metric mapping registry to indicate that, as a result of the measurement being stored within the determined one or more datastores, the measurement is authoritative for the time period.

5. A system, comprising at least one computing device configured to implement one or more services, wherein the one or more services are configured to:
receive a request to retrieve measurement data for a metric for an interval of time;
determine, based at least in part on information in the request, one or more datastores where the measurement data is stored, the information in the request including a metric identifier and a timestamp information for the measurement data, the metric identifier having been generated by a hash function and metadata associated with the measurements data, wherein at least a portion of the metric identifier identifies the one or more datastores, while another portion of the metric identifier is used for unique identification of the measurement data within the one or more datastores; and
generate a response to the request such that:
if the request indicates that authoritative data is to be provided in the generated response, the generated response limits the measurement data to authoritative measurement data; and
if the request does not indicate that authoritative data is to be provided in the generated response, the generated response is not limited to authoritative measurement data.

6. The system of claim 5 , wherein the information in the request includes information indicating that the measurement data to be retrieved is authoritative.

7. The system of claim 5 , wherein the information in the request includes information indicating that measurement data to be retrieved is not limited to the authoritative measurement data.

8. The system of claim 5 , wherein:
the one or more services are further configured to:
obtain, for at least the interval of time, a mapping indicating the one or more datastores where the measurement data is stored.

9. The system of claim 5 , wherein the one or more services are further configured to:
receive a second request to store a measurement for a second metric;
determine, based at least in part on information in the second request, that measurement data for the second metric is authoritative for a time interval, the measurement data for the second metric including the measurement for the second metric; and
store the measurement for the second metric in a second in-memory datastore as authoritative for the time interval.

10. The system of claim 9 , wherein:
the second request to store the measurement for the second metric includes the measurement for the second metric and metadata specifying attributes of the measurement for the second metric; and
the one or more services are further configured to:
generate, based at least in part on the metadata, a metric identifier for the measurement for the second metric;
select, based at least in part on the metric identifier for the measurement for the second metric, a partition of a plurality of logical partitions for placement of the measurement for the second metric; and
transmit the me

Assignees

• Amazon Tech Inc

Inventors

• Taylor Gary
• Dennehy Brian
• Rodriguez Sebastien Alexandre Roland
• Mangas Julio Delgado

Classifications

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F11/3409 Primary

  for performance assessment · CPC title

• G06F11/3452

  Performance evaluation by statistical analysis · CPC title

• G06F17/40

  Data acquisition and logging (for input to computer G06F3/00) · CPC title

• G06F11/3433

  for load management (allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029) · CPC title