Method and apparatus for compaction of data received over a network

Patent metadata

Field	Value
Publication number	US-10015272-B2
Application number	US-201514911223-A
Country	US
Kind code	B2
Filing date	Mar 12, 2015
Priority date	Mar 12, 2015
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

Methods, apparatuses, and storage media associated with compaction of data from one or more computing devices are disclosed. In various embodiments, one or more Internet of Things (IoT) devices may transmit information to a computing system. The computing system may group together raw data received from these one or more IoT devices based on a shared attribute. The computing system may select a compaction scheme to represent the knowledge conveyed by a group of the raw data. The computing system may apply this compaction scheme to the group of raw data to generate data that is representative of the group of raw data. Other embodiments may be disclosed or claimed.

First claim

What is claimed is:

1. A computing system for processing raw data from Internet of Things (IoT) devices, the system comprising:
network interface circuitry to receive the raw data from the IoT devices over a network; and
one or more processors and physical memory, coupled with the one or more processors, to store a compaction module to be loaded into the physical memory for execution by the one or more processors, wherein:
the compaction module, coupled with the network interface circuitry, is to:
group values included in the raw data based on at least one attribute of corresponding ones of the IoT devices from which the raw data is obtained;
select at least one compaction scheme based on the values of the group;
generate representative data based on application of the selected at least one compaction scheme to the values of the group, wherein the application of the selected at least one compaction scheme is based on metadata that describes:
a type of representative data to be generated, wherein the type of representative data to be generated comprises a function, a bitmap index, or a principal value, and
how the selected at least one compaction scheme is to be applied to the values to generate the type of representative data.

2. The computing system of claim 1 , further comprising:
a database, wherein the compaction module is to store the representative data in a table of the database.

3. The computing system of claim 1 , wherein the compaction module is to discard the values of the raw data.

4. The computing system of claim 1 , wherein at least one of the IoT devices is included in a vehicle, a phone, a medical device, or a meter.

5. The computing system of claim 1 , wherein the computing system is included in an edge device or communicatively coupled with the edge device.

6. The computing system of claim 1 , wherein the compaction module is to discard the values of the group.

7. The computing system of claim 1 , wherein, when the representative data is a function, the compaction scheme includes a function fitting compression scheme or a piecewise compaction scheme, and wherein the compaction module is to:
fit a function to the values of the group,
generate the function as a linear function or a quadratic function when the compaction scheme includes the function fitting compression scheme, and
generate the function as a piecewise linear function when the compaction scheme includes the piecewise compaction scheme.

8. The computing system of claim 1 , wherein, when the representative data is the principal value, the compaction scheme includes deviation from the principal value.

9. The computing system of claim 8 , wherein the compaction module is to generate the representative data as respective percentage deviations of the values of the group from the principal value or as respective standard deviations of the values of the group from the principal value.

10. The computing system of claim 8 , wherein the compaction module is to generate the representative data as an indication of a number of values clustered around the representative value within a standard deviation.

11. The computing system of claim 1 , wherein, when the representative data is a bitmap index, the compaction module is to populate at least one bitmap associated with the bitmap index based on the values of the group.

12. The computing system of claim 1 , wherein the compaction module is to store at least a portion of the raw data.

13. The computing system of claim 1 , further comprising:
a query processing module, coupled with the network interface circuitry and the compaction module and to be loaded into the physical memory by execution by the one or more processors, to:
process a query received by the network interface circuitry;
identify responsive data based on the query, wherein the responsive data is to include at least one of raw data or uncompacted data; and
cause the network interface circuitry to transmit the responsive data.

14. The computing system of claim 13 , further comprising:
an uncompaction module, coupled with the compaction module and the query processing module and to be loaded in the physical memory for execution by the one or more processors, to generate the responsive data based on the representative data.

15. The computing system of claim 14 , wherein the uncompaction module is to generate the responsive data through a randomized algorithm for interpolation based on the representative data or application of a function indicated by the representative data.

16. The computing system of claim 1 , wherein, when the representative data is a bitmap index, the compaction scheme is a word-aligned hybrid bitmap compression scheme.

17. The computing system of claim 1 , wherein the representative data includes an indication of a number of values comprising the raw data to be represented by the representative data.

18. The computing system of claim 1 , wherein the compaction scheme is explicitly indicated or specified by a set of parameterized rules.

19. The computing system of claim 1 , wherein the metadata further describes how the representative data is to be stored.

20. A computer-implemented method for compacting data, the method comprising:
receiving, by a computing system, first data over a network from one or more Internet of Things (IoT) devices;
identifying, by the computing system, an attribute associated with the one or more of the IoT devices that is common to a first plurality of values included in the first data;
determining, by the computing system, a compaction scheme that is to indicate the first plurality of values;
applying, by the computing system, the compaction scheme to the first plurality of values to create compaction data, wherein the application of the determined compaction scheme is based on metadata that describes; a type of compaction data to be generated, wherein the type of compaction data to be generated comprises a function, a bitmap index, or a principal value, how the selected at least one compaction scheme is to be applied to the values to generate the type of compaction data, and how the compaction data is to be stored; and
storing, by the computing system, the compaction data in at least one table of a database.

21. The computer-implemented method of claim 20 , wherein when the type of compaction data is a bitmap index, the applying of the compaction scheme comprises:
populating, by the computing system, at least one bitmap to be associated with the bitmap index based on the first plurality of values; and
generating, by the computing system, the bitmap index based on the attribute and the first plurality of values.

22. The computer-implemented method of claim 20 , wherein, when the compaction data is a function, the applying of the compaction scheme comprises:
fitting, by the computing system, a function to the first plurality of values;
generating, by the computing system, the function as a linear function or a quadratic function when the compaction scheme includes a function fitting compression scheme; and
generating, by the computing system, the function as a piecewise linear function when the compaction scheme includes a piecewise compaction scheme.

23. The computer-implemented method of claim 20 , further comprising:
receiving, by the computing system over the network, a request for data associated with the one or more IoT devices;
determining, by the computing system, response data based on

Assignees

• Intel Corp

Inventors

• Zhong Tao
• Doshi Kshitij A
• Deng Gang
• Lou Ting
• Lu Zhongyan

Classifications

• H04L69/04 Primary

  Protocols for data compression, e.g. ROHC · CPC title

• G06F3/0638

  Organizing or formatting or addressing of data · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• H04L67/1095

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• H03M7/6088

  according to the data type · CPC title