Systems and methods for providing real-time streaming data processing at edge servers

Patent metadata

Field	Value
Publication number	US-11178203-B2
Application number	US-202017019561-A
Country	US
Kind code	B2
Filing date	Sep 14, 2020
Priority date	Aug 19, 2019
Publication date	Nov 16, 2021
Grant date	Nov 16, 2021

Abstract

Systems and methods provide synchronizing edge server code among a plurality of edge servers. Systems and methods provision, to a plurality of edge servers, an updated version of transform code adapted to perform real-time processing on real-time streaming data that are received by the plurality of edge servers, receive real-time transformed data from at least one of the plurality of edge servers, detect that the received real-time transformed data from the at least one of the plurality of edge servers was generated using a previous version of transform code and perform one or more transform operations corresponding to the updated version of transform code on the received real-time transformed data, until the received transformed data is in a form consistent with the updated version of provisioned transform code.

First claim

What is claimed is:

1. A method for synchronizing edge server code among a plurality of edge servers comprising:
provisioning, to a plurality of edge servers, an updated version of transform code adapted to perform real-time processing on real-time streaming data that are received by the plurality of edge servers;
receiving real-time transformed data from at least one of the plurality of edge servers;
detecting that the received real-time transformed data from the at least one of the plurality of edge servers was generated using a previous version of transform code; and
performing one or more transform operations corresponding to the updated version of transform code on the received real-time transformed data, until the received transformed data is in a form consistent with the updated version of provisioned transform code;
wherein the method is performed using one or more processors of a central server.

2. The method of claim 1 comprising tracking a transform code version of each of the plurality of edge servers and determining whether to perform a newer transformation on the real-time transformed data received from an edge server that is using an older version of the transform code prior to completing an upgrade.

3. The method of claim 1 comprising using raw cached real-time streaming data provided from multiple edge servers and testing different transforms to determine a best transform to provide to a set of edge servers in the network.

4. A central server for synchronizing edge server code comprising:
one or more processors;
memory comprising executable instructions that when executed by the one or more processors causes the one or more processors to:
provision, to a plurality of edge servers, an updated version of transform code adapted to perform real-time processing on real-time streaming data that are received by the plurality of edge servers;
receive real-time transformed data from at least one of the plurality of edge servers;
detect that the received real-time transformed data from the at least one of the plurality of edge servers was generated using a previous version of transform code; and
perform one or more transform operations corresponding to the updated version of transform code on the received real-time transformed data, until the received transformed data is in a form consistent with the updated version of provisioned transform code.

5. The central server of claim 4 wherein the memory comprises instructions that when executed by the one or more processors causes the one or more processors to track a transform code version of each of the plurality of edge servers and determine whether to perform a newer transformation on the real-time transformed data received from an edge server that is using an older version of the transform code prior to completing an upgrade.

6. The central server of claim 4 wherein the memory comprises instructions that when executed by the one or more processors causes the one or more processors to use raw cached real-time streaming data provided from multiple edge servers and test different transforms to determine a best transform to provide to a set of edge servers in the network.

7. A system for providing real-time streaming data, the system comprising:
a plurality of edge servers;
a storage associated with each of the plurality of edge servers;
wherein each of the plurality of edge servers is configured to:
receive real-time streaming data from at least a respective first client device;
process, within a first time duration that is in real-time, the received real-time streaming data to produce first edge processed data;
store the first edge processed data in the respective storage;
receive a request for real-time streaming data from at least a second client device;
provide in real-time the first edge processed data to the requesting second client device; and
forward second edge processed data to a central server for processing, the second edge processed data being different from or the same as the first edge processed data; and

the central server, configured to:
provision, to the plurality of edge servers, an updated version of transform code adapted to perform real-time processing on real-time streaming data that are received by the plurality of edge servers;
receive real-time transformed data from at least one of the plurality of edge servers;
detect that the received real-time transformed data from the at least one of the plurality of edge servers was generated using a previous version of transform code; and
perform one or more transform operations corresponding to the updated version of transform code on the received real-time transformed data, until the received transformed data is in a form consistent with the updated version of provisioned transform code.

8. The system of claim 7 wherein the central server is configured to track a transform code version of each of the plurality of edge servers and determine whether to perform a newer transformation on the real-time transformed data received from an edge server that is using an older version of the transform code prior to completing an upgrade.

9. The system of claim 7 wherein the central server is configured to use raw cached real-time streaming data provided from multiple edge servers and test different transforms to determine a best transform to provide to a set of edge servers in the network.

10. The system of claim 7 wherein at least one of the plurality of edge servers are configured to process, within a second time duration, the received real-time streaming data to produce the second edge processed data, the second time duration being longer than or equal to the first time duration.

11. The system of claim 10 wherein at least one of the plurality of edge servers is configured to:
receive real-time streaming data from the at least the second client device;
process, within the first time duration that is in real-time, the received real-time streaming data from the second client device to produce the first edge processed data from real-time streaming data from both the at least first and second client devices; and
broadcast the first edge processed data to the at least first and second client device in real-time.

12. The system of claim 11 wherein at least one of the plurality of edge servers is configured to process, within the first time duration that is in real-time by transforming the real-time streaming data and caching the transformed data and process within the second time duration by using the transformed data as input to a windowed compute operation to produce the second edge processed data.

13. The system of claim 12 wherein at least one of the edge servers is configured to provide the second edge processed data from the edge server to the requesting second client device.

14. The system of claim 7 wherein the received real-time streaming data is in a format different from a format needed by the second client device and wherein at least one of the edge servers is configured to process the received real-time streaming data by normalizing the data in real-time and provide, in real-time, the first edge processed data from the at least one edge server to the requesting second client device by broadcasting the normalized real-time streaming data to the second client device.

15. The system of claim 7 wherein at least one the edge server is configured to process the received real-time streaming data by performing one or more of: generating alerts for geo-fencing operations associated with the first and second client devices, format conversion of location information r

Assignees

• Palantir Technologies Inc

Inventors

• Wilczynski Peter
• Zangri Daniel
• Martin Jeffrey
• Imig Robert
• Hou Wenshuai

Classifications

• H04L65/1073 Primary

  Registration or de-registration · CPC title

• H04L65/765 Primary

  intermediate · CPC title

• H04L65/61

  for supporting one-way streaming services, e.g. Internet radio · CPC title

• H04L67/52

  specially adapted for the location of the user terminal · CPC title

• H04L65/762

  at the source  (reformatting of additional data in video distribution servers H04N21/2355) · CPC title