Publish-subscribe platform for cloud file distribution

Patent metadata

Field	Value
Publication number	US-9485300-B2
Application number	US-201213419304-A
Country	US
Kind code	B2
Filing date	Mar 13, 2012
Priority date	Mar 13, 2012
Publication date	Nov 1, 2016
Grant date	Nov 1, 2016

Abstract

Methods and apparatus for managing file distribution between publishing and subscribing devices are provided. At a transporter system, a publish request for publishing a version of the dataset is received from a publisher device, wherein the publish request has a predefined format that includes an identity of the dataset, an identity of the version of the dataset, and an identity of a location for the version of the dataset. The transporter system updates a registry to associate the dataset with the identity of the dataset, version, and location, and the transporter system automatically publishes the dataset to any subscribers devices that have previously sent subscribe requests for such dataset to the transporter system.

First claim

What is claimed is:

1. A computer implemented method of managing file distribution between publishing and subscribing devices, comprising:
at a transporter system, receiving from a publisher device a publish request for publishing a version of a dataset, wherein the publish request has a predefined format that includes an identity of the dataset, an identity of the version of the dataset, and an identity of a location for the version of the dataset, wherein the version of the dataset includes a complete dataset or an incremental update to a previous version of the dataset;
the transporter system updating a registry to associate the version of the dataset with the identity of the dataset, version, and location; and
the transporter system publishing the version of the dataset to any subscribers devices that have previously sent subscribe requests for such dataset to the transporter system such that the complete dataset or incremental update to the previous version of the dataset is sent to the subscriber devices that have previously sent subscribe requests for such dataset to the transporter system;
wherein the publisher device has no knowledge of any subscriber devices to the dataset, and any subscriber devices to such dataset have no knowledge of the publisher device.

2. The method of claim 1 , the method further comprising:
the transporter system generating a checksum for the version of the dataset;
the transporter system updating the registry to associate the generated checksum with the version of the dataset; and
using the checksum to validate the version of the dataset prior to automatically publishing the version of the dataset, wherein the version of the dataset is only automatically published if the versioned dataset is valid.

3. The method of claim 1 , further comprising the transporter system storing the dataset in a cloud storage network, and wherein the transporter system is scalable along multiple dimensions, including dataset size, number of datasets, and number of subscribers to a dataset, and wherein automatically publishing includes the transporter system publishing to any subscribers directly or through multiple tiered distributors.

4. The method of claim 1 , wherein the predefined format of the publish request further includes information specifying how long the versioned dataset is to be retained by the transporter system and how many versions of the dataset are to be retained by the transporter system, and metadata specifying custom key/value pairs to be associated with the versioned dataset, the method further comprising:
determining whether the publish request is for a duplicate version of the dataset that has already been published; and
sending an error message to the publisher device if the publication request is for a duplicate version, and
wherein the operation for updating and automatically publishing are only performed if the version of the publish request is determined to not be a duplicate version.

5. The method of claim 1 , further comprising:
at a transporter system, receiving from one or more subscribing devices a subscribe request for the dataset, wherein the subscribe request has a predefined format that includes the identity of the dataset and a destination identity for receiving published updates of the dataset; and
after receiving each subscribe request, the transporter system updating the registry to associate the destination identity of such subscribe request with the identity of the dataset.

6. The method of claim 5 , further comprising:
repeating the operations for receiving from a publisher device, updating the registry, and automatically publishing for a plurality of publish requests for different datasets and different dataset versions from a plurality of different publisher devices, wherein at least one dataset is published by notifying a plurality of associated subscribers of individual chunks of such dataset so that such subscribers can share the individual chunks with each other;
repeating the operations for receiving from one or more subscriber devices and updating the registry for a plurality of subscribe requests for the different datasets;
if a query request to discover dataset versions is received, providing information from the registry for one or more datasets to a sender of such query call; and
if a remove request for a particular identified dataset version is received from a publisher device, removing information for the particular identified dataset version from the registry so that the particular identified dataset version is not published to any subscribers.

7. The method of claim 6 , wherein the predefined format of the publish request further includes a namespace identity that is also associated with the dataset in the registry and the predefined format of each subscribe request for the dataset also includes the namespace identity, an executable and associated arguments for a command to be executed for each updated version of the dataset that is published, wherein the destination identity specifies a host device and a consumer identity, the method further comprising:
for each automatically published updated version of each dataset, automatically invoking the command associated with such dataset, wherein the command is invoked at the specified host device.

8. An apparatus comprising at least a processor and a memory, wherein the processor and/or memory are configured to perform the following operations:
receiving from a publisher device a publish request for publishing a version of a dataset, wherein the publish request has a predefined format that includes an identity of the dataset, an identity of the version of the dataset, and an identity of a location for the version of the dataset, wherein the version of the dataset includes a complete dataset or an incremental update to a previous version of the dataset;
updating a registry to associate the version of the dataset with the identity of the dataset, version, and location; and
publishing the version of the dataset to any subscribers devices that have previously subscribed to such dataset such that the complete dataset or incremental update to the previous version of the dataset is sent to the subscriber devices that have previously sent subscribe requests for such dataset to the transporter system;
wherein the publisher device has no knowledge of any subscriber devices to the dataset, and any subscriber devices to such dataset have no knowledge of the publisher device.

9. The apparatus of claim 8 , wherein the processor and/or memory are further configured for:
generating a checksum for the version of the dataset;
updating the registry to associate the generated checksum with the version of the dataset; and
using the checksum to validate the version of the dataset prior to automatically publishing the version of the dataset, wherein the version of the dataset is only automatically published if the version of the dataset is valid.

10. The apparatus of claim 8 , wherein the processor and/or memory are further configured for storing the dataset in a cloud storage network, and wherein the transporter system is scalable along multiple dimensions, including dataset size, number of datasets, and number of subscribers to a dataset, and wherein automatically publishing includes the transporter system publishing to any subscribers directly or through multiple tiered distributors.

11. The apparatus of claim 8 , wherein the predefined format of the publish request further includes information specifying how long the version of the dataset is to be retained and how many versions of the dataset are to be retained, and metadata specifying custom key/value pairs to be associat

Assignees

• Feng Andrew An
• Chandra Rohit
• Suryanarayanan Lakshmanan
• Crowder Timothy R
• Lam Victor J
• Yahoo Inc

Inventors

• Feng Andrew An
• Chandra Rohit
• Suryanarayanan Lakshmanan
• Crowder Timothy R
• Lam Victor J

Classifications

• H04L29/08072

  Electricity · mapped topic

• H04L29/06

  Electricity · mapped topic

• H04L29/0809

  Electricity · mapped topic

• G06Q30/02

  Marketing; Price estimation or determination; Fundraising · CPC title

• H04L67/06 Primary

  specially adapted for file transfer, e.g. file transfer protocol [FTP] · CPC title