System and method for data warehouse engine

Patent metadata

Field	Value
Publication number	US-10817528-B2
Application number	US-201615365755-A
Country	US
Kind code	B2
Filing date	Nov 30, 2016
Priority date	Dec 15, 2015
Publication date	Oct 27, 2020
Grant date	Oct 27, 2020

Abstract

A data warehouse engine (DWE) includes a central processing unit (CPU) core and a first data organization unit (DOU), where the first DOU is configured to aggregate read operations. The DWE also includes a first command queue coupled between the CPU core and the first DOU, where the first command queue is configured to convey commands from the CPU core to the first DOU.

First claim

What is claimed is:

1. A data warehouse engine (DWE) comprising:
a central processing unit (CPU) core;
a first data organization unit (DOU) comprising:
a cache storage of a non-uniform access engine of the DOU for non-uniform access,
a streaming buffer of a streaming access engine of the DOU for streaming access,

a command queue (cQ) message decoder coupled to the cache storage, wherein the cQ message decoder is configured to receive a command from the CPU core, and
a read interface coupled to the cache storage and the streaming buffer, wherein the read interface is configured to receive data from data storage and, in accordance with the command, store the data in the cache storage or forward the data for storage in the streaming buffer,
wherein the first DOU is configured to aggregate read operations, and to determine whether to store the data in the cache storage or the streaming buffer in accordance with the command sent from the CPU core; and

a first command queue coupled between the CPU core and the first DOU, wherein the first command queue is configured to convey the command from the CPU core to the first DOU.

2. The DWE of claim 1 , further comprising a response queue coupled between the first DOU and the CPU core, wherein the response queue is configured to convey responses from the first DOU to the CPU core.

3. The DWE of claim 1 , wherein the first DOU is further configured to perform on-the-fly data reorganization.

4. The DWE of claim 1 , further comprising:
a second DOU; and
a second command queue coupled between the CPU core and the second DOU.

5. The DWE of claim 1 , wherein the first DOU is further configured to perform at least one of: disaggregate write operations, disaggregate read operations, and aggregate write operations.

6. A data organization unit (DOU) comprising:
a source cache storage of a non-uniform access engine of the DOU;
a streaming buffer of a streaming access engine of the DOU;
a command queue (cQ) message decoder coupled to the source cache storage, wherein the cQ message decoder is configured to receive a message from a central processing unit (CPU) core; and
a read interface coupled to the source cache storage and the streaming buffer, wherein the read interface is configured to receive data from data storage and, in accordance with the message, store the data in the source cache storage or forward the data for storage in the streaming buffer.

7. The DOU of claim 6 , further comprising:
an elastic buffer coupled to the source cache storage; and
a destination cache storage coupled to the elastic buffer.

8. The DOU of claim 7 , further comprising:
a write manager coupled to the destination cache storage;
a write command queue coupled to the write manager; and
a write interface coupled to the write command queue.

9. The DOU of claim 8 , further comprising a response queue (rQ) message encoder coupled to the write interface, wherein the rQ message encoder is configured to send response messages to the CPU.

10. The DOU of claim 8 , further comprising:
a read data transformer coupled to the read interface between the read interface and the streaming buffer; and
a streaming write address generator coupled to the cQ message decoder and the stream buffer.

11. The DOU of claim 10 , further comprising a write transformer coupled between the streaming buffer and the write command queue.

12. The DOU of claim 6 , further comprising a transaction generator coupled to the cQ message decoder.

13. The DOU of claim 12 , further comprising a data storage allocator and tracker coupled between the transaction generator and the source cache storage.

14. The DOU of claim 12 , further comprising:
a streaming read address generator coupled to the cQ message decoder and to the transaction generator; and
a read command queue coupled to the streaming read address generator and to the read interface.

15. A method comprising:
receiving, by a data organization unit (DOU) from a central processing unit, a message;
obtaining, by the DOU from a source storage, first data, in accordance with the message;
determining whether to store the first data in a source cache storage or a streaming buffer in accordance with the message;
storing the first data in the source cache storage or the streaming buffer in accordance with the determination;
reorganizing the first data to produce second data; and
storing the second data in a destination storage.

16. The method of claim 15 , wherein the source storage is the destination storage.

17. The method of claim 15 , wherein reorganizing the first data comprises scattering the first data using disaggregation.

18. The method of claim 17 , wherein scattering the first data comprises performing non-uniform scattering or streaming scattering.

19. The method of claim 15 , wherein reorganizing the first data comprises gathering the first data using aggregation.

20. The method of claim 19 , wherein gathering the first data comprises performing non-uniform gathering or streaming scattering.

Assignees

• Futurewei Technologies Inc

Inventors

• Shrivastava Ashish Rai
• Chandra Alex Elisa
• Brown Mark
• Bhattacharya Debashis
• Gatherer Alan

Classifications

• G06F16/283 Primary

  Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP · CPC title

• G06F16/211

  Schema design and management · CPC title

• G06F16/252 Primary

  between a Database Management System and a front-end application · CPC title