Custom partitioning of a data stream

Patent metadata

Field	Value
Publication number	US-9244978-B2
Application number	US-201414302031-A
Country	US
Kind code	B2
Filing date	Jun 11, 2014
Priority date	Jun 11, 2014
Publication date	Jan 26, 2016
Grant date	Jan 26, 2016

Abstract

Techniques for partitioning data streams are provided. In some examples, a query for processing at least a portion of a data stream may be identified. The data stream may be associated with a user. Additionally, in some examples, code identifying an attribute of the identified stream may be received. The code may be capable of configuring the query based at least in part on the attribute. Further, in some aspects, the code may be configured to partition the data stream into at least a sub-stream based at least in part on the attribute.

First claim

What is claimed is:

1. A system, comprising:
a memory storing computer-executable instructions; and
a processor configured to access the memory and execute the computer-executable instructions to at least:
identify a data stream associated with a user;
receive a query for processing at least a portion of the data stream;
identify a rate of events of the data stream, the rate of events comprising a number of events received by the processor per time period;
determine that the rate of events is above a threshold; and
when the rate of events is above the threshold:
provide, to the user, a software development kit for generating a java class;
enable the user to create a partition class in java using the software development kit, the partition class configured to:
receive each event of the data stream at runtime; and

partition the data stream into a plurality of sub-stream portions;

receive, from the user, an attribute of the data stream;
receive, from the user, the partition class generated by the user for partitioning the data stream, the partition class is based at least in part on the software development kit and the attribute of the data stream;
associate, by the computer system, the partition class with the query;
process each event of the data stream with the partition class at runtime to partition the data stream based at least in part on the association of the partition class with the query; the process comprising:
partitioning the data stream into the plurality of sub-stream portions based at least in part on the attribute;
provide at least one sub-stream portion of the plurality of sub-stream portions to the query;
process the at least one sub-stream portion, using the query, in parallel with at least a second sub-stream portion of the plurality of sub-stream portions that is received as an output from the partition class; and
provide results of the query based at least in part on the at least one sub-stream portion and the second sub-stream portion.

2. The system of claim 1 , wherein associating the partition class with the query comprises tying the query to the partition class.

3. The system of claim 2 , wherein the processor is further configured to execute the computer-executable instructions to at least:
receive an event of the data stream based at least in part on the query;
provide the event to the partition class; and
receive an output from the partition class.

4. The system of claim 3 , wherein the data stream is partitioned based at least in part on the output from the partition class.

5. The system of claim 1 , wherein the attribute comprises at least one of a first identifier of a computing device of a customer of the user, a second identifier of an item associated with the user, or a third identifier of an item attribute associated with the item.

6. The system of claim 5 , wherein the identifier of the computing device of the customer indicates an Internet Protocol address of the computing device of the customer or a geographic region in which the computing device of the customer is located.

7. A non-transitory computer-readable storage memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising:
instructions to identify a query for processing at least a portion of a data stream associated with a user;
instructions to identify the data stream associated with the user;
instructions to identify a rate of events of the data stream, the rate of events comprising a number of events received per time period;
instructions to determine that the rate of events is above a threshold; and
when the rate of events is above the threshold, the plurality of instructions further comprise:
instructions to provide a software development kit to the user;
instructions to enable the user to create a partition class in java using the software development kit, the partition class is configured to:
receive each event of the data stream at runtime; an
partition the data stream into a plurality of sub-stream portions;

instructions to receive, from the user, an attribute of the data stream;
instructions to receive, from the user, the partition class generated by the user for partitioning the data stream, the partition class is identifying the attribute of the data stream and is generated based at least in part on the software development kit;
instructions to process each event of the data stream with the partition class at runtime to partition the data stream based at least in part on an association of the partition class with the query, the processing of each event comprising:
partitioning the data stream into the plurality of sub-stream portions based at least in part on the attribute; an

instructions to configure the query to process at least one sub stream portion of the plurality of sub-stream portions in parallel with at least a second sub-stream portion of the plurality of sub-stream portions that is received as an output from the partition class; and
instructions to provide results of the query based at least in part on the at least one sub-stream portion and the at least one second sub-stream portion.

8. The non-transitory computer-readable storage memory of claim 7 , wherein the plurality of instructions further comprise:
instructions to receive an event of the data stream based at least in part on the query;
instructions to provide the event to the partition class;
instructions to implement the partition class to process the event based at least in part on the attribute; and
instructions to receive an output from the partition class.

9. The non-transitory computer-readable storage memory of claim 8 , wherein the output from the partition class is at least one of the plurality of sub-stream portions.

10. The non-transitory computer-readable storage memory of claim 7 , wherein the attribute is specified by the user prior to receipt of the partition class.

11. The non-transitory computer-readable storage memory of claim 7 , wherein at least one sub-stream of the plurality of sub-stream portions is generated based at least in part on a hashing function performed on the at least a portion of the data stream.

12. A computer-implemented method, comprising:
identifying, by a computing system, a data stream associated with a user,
receiving a query for processing at least a portion of the data stream;
identifying a rate of events of the data stream, the rate of events comprising a number of events received by the computing system per time period;
determining that the rate of events is above a threshold; and
when the rate of events is above the threshold:
receiving, from the user, a partition class in java identifying an attribute of the data stream, the partition class is based at least in part on a software development kit received by the user, and the partition class is configured to:
receive each event of the data stream; and
partition the data stream into a plurality of sub-stream portions based at least in part on the attribute;

executing the partition class at runtime to partition the data stream into at least one sub-stream portion of the plurality of sub-stream portions based at least in part on the attribute and an association of the partition class with the query;
processing the at least one sub-stream using the query in parallel with at least a second sub-stream portion of the plurality of sub-stream portions that is received as an output from the partition class; and
providing results of the query based at least in part on the at least one sub-stream portion and the second sub-stream portion.

13. The computer-i

Assignees

• Oracle Int Corp

Inventors

• Alves Alexandre
• Bishnoi Sandeep

Classifications

• G06F16/24568

  Data stream processing; Continuous queries · CPC title

• G06F16/903

  Querying (for retrieval from the web G06F16/953) · CPC title

• G06F16/2457

  with adaptation to user needs · CPC title

• G06F16/24534 Primary

  Query rewriting; Transformation · CPC title

• G06F17/30516

  Physics · mapped topic