Compression-based detection of inefficiency in external services

Patent metadata

Field	Value
Publication number	US-2017336995-A1
Application number	US-201615160905-A
Country	US
Kind code	A1
Filing date	May 20, 2016
Priority date	May 20, 2016
Publication date	Nov 23, 2017
Grant date	—

Abstract

The disclosed embodiments provide a system for detecting and managing inefficiency in external services. During operation, the system obtains a snapshot of a data stream transmitted over an external service from a computer system at a first time. Next, the system applies a compression technique to the snapshot to obtain a set of inefficiency metrics for a set of data elements in the snapshot. The system then outputs the set of inefficiency metrics with additional attributes of the data stream to improve identification of inefficiency in the data stream.

First claim

What is claimed is:

1 . A method, comprising:
obtaining a snapshot of a data stream transmitted over an external service from a computer system at a first time; applying, by a processor, a compression technique to the snapshot to obtain a set of inefficiency metrics for a set of data elements in the snapshot; and outputting the set of inefficiency metrics with additional attributes of the data stream to improve management of inefficiency in the data stream.

2 . The method of claim 1 , further comprising:
grouping a first subset of the data elements by a first attribute; applying the compression technique to the first subset to obtain a first group inefficiency metric for the first subset; and including the first group inefficiency metric in the outputted set of inefficiency metrics.

3 . The method of claim 2 , further comprising:
grouping a second subset of the data elements within the first subset by a second attribute; applying the compression technique to the second subset to obtain a second group inefficiency metric for the second subset; and using the first and second group inefficiency metrics to analyze a cause of the inefficiency.

4 . The method of claim 2 , wherein the first attribute comprises at least one of:
a port; a service type; a protocol; a network stack layer; a data element type; an address; a source; and a destination.

5 . The method of claim 1 , wherein the inefficiency is at least one of:
a request strategy inefficiency; and a schema inefficiency.

6 . The method of claim 1 , wherein:
the external service comprises a networking service, and the first set of data elements comprises a network packet.

7 . The method of claim 1 , wherein the external service comprises an interconnect service.

8 . The method of claim 1 , wherein outputting the set of inefficiency metrics with the additional attributes of the first set of data elements comprises at least one of:
displaying a ranking of the first set of data elements by the first set of inefficiency metrics; and identifying a subset of the first set of data elements as candidates for improving the inefficiency.

9 . The method of claim 1 , wherein outputting the set of inefficiency metrics with the additional attributes of the data stream comprises:
outputting, based on the first set of inefficiency metrics, a security flaw in the data stream.

10 . The method of claim 1 , wherein the set of inefficiency metrics comprises at least one of:
a redundancy; and a compression ratio.

11 . An apparatus, comprising:
one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
obtain a snapshot of a data stream transmitted over an external service from a computer system at a first time;
apply a compression technique to the snapshot to obtain a set of inefficiency metrics for a set of data elements in the snapshot; and
output the set of inefficiency metrics with additional attributes of the data stream to improve identification of inefficiency in the data stream.

12 . The apparatus of claim 11 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
group a first subset of the data elements by a first attribute; apply the compression technique to the first subset to obtain a first group inefficiency metric for the first subset; and include the first group inefficiency metric in the outputted set of inefficiency metrics.

13 . The apparatus of claim 12 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
group a second subset of the data elements within the first subset by a second attribute; apply the compression technique to the second subset to obtain a second group inefficiency metric for the second subset; and use the first and second group inefficiency metrics to analyze a cause of the inefficiency.

14 . The apparatus of claim 12 , wherein the first attribute comprises at least one of:
a port; a service type; a protocol; a network stack layer; a data element type; an address; a source; and a destination.

15 . The apparatus of claim 11 , wherein the inefficiency is at least one of:
a request strategy inefficiency; and a schema inefficiency.

16 . The apparatus of claim 11 , wherein:
the external service comprises a networking service, and the first set of data elements comprises a network packet.

17 . The apparatus of claim 11 , wherein the external service comprises an interconnect service.

18 . The apparatus of claim 11 , wherein outputting the set of inefficiency metrics with the additional attributes of the data stream comprises:
outputting, based on the first set of inefficiency metrics, a security flaw in the data stream.

19 . A system, comprising:
an analysis module comprising a non-transitory computer-readable medium storing instructions that, when executed by, cause the system to:
obtain a snapshot of a data stream transmitted over an external service from a computer system at a first time; and
apply a compression technique to the snapshot to obtain a set of inefficiency metrics for a set of data elements in the snapshot; and
a management module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to output the set of inefficiency metrics with additional attributes of the data stream to improve identification of inefficiency in the data stream.

20 . The system of claim 19 , wherein the non-transitory computer-readable medium of the analysis module further stores instructions that, when executed, cause the system to:
group a first subset of the data elements by a first attribute; apply the compression technique to the first subset to obtain a first group inefficiency metric for the first subset; group a second subset of the data elements within the first subset by a second attribute; apply the compression technique to the second subset to obtain a second group inefficiency metric for the second subset; and use the first and second group inefficiency metrics to analyze a cause of the inefficiency.

Assignees

• Linkedin Corp

Inventors

• Nicol John W
• Maheshwari Ritesh
• Baggott Nicholas P
• Ramachandra Haricharan K
• Sridharan Badrinath K

Classifications

• G06F11/3476 Primary

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F3/0673

  Single storage device · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/0619 Primary

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F11/302

  where the computing system component is a software system · CPC title