Tiered cache filling

Patent metadata

Field	Value
Publication number	US-11010341-B2
Application number	US-201615067099-A
Country	US
Kind code	B2
Filing date	Mar 10, 2016
Priority date	Apr 30, 2015
Publication date	May 18, 2021
Grant date	May 18, 2021

Abstract

One embodiment of the present invention sets forth a technique for replicating a file within a network of servers. The technique includes determining one or more regional master servers included in a regional cluster of servers and causing each regional master server to retrieve a file from a fill source. The technique further includes, for at least one local cluster of servers included in the regional cluster of servers, determining one or more local master servers included in the at least one local cluster of servers, and causing each local master server to retrieve the file from one of the one or more regional master servers.

First claim

What is claimed is:

1. A method, comprising:
performing a first hashing operation on a plurality of server identifiers (IDs) associated with servers included in a regional cluster of servers to generate a plurality of hash values for the server IDs, wherein each server ID included in the plurality of server IDs identifies a different server included in the regional cluster of servers;
performing a second hashing operation on a file ID associated with a content file stored at a third-party fill source to generate a first hash value;
performing one or more operations based on the first hash value to identify one or more regional master servers included in the regional cluster of servers for the content file based on comparing the first hash value to one or more hash values included in the plurality of hash values for the server IDs that identify the one or more regional master servers;
causing each regional master server included in the one or more regional master servers to retrieve the content file from the third-party fill source via a network connection; and
for at least one local cluster of servers included in the regional cluster of servers:
performing one or more operations to identify one or more local master servers included in the at least one local cluster of servers; and
causing each local master server included in the one or more local master servers to retrieve the content file from one of the one or more regional master servers instead of retrieving the content file from the third-party fill source.

2. The method of claim 1 , further comprising, for each manifest cluster of servers included in the at least one local cluster of servers:
performing one or more operations to identify one or more manifest master servers included in the manifest cluster of servers; and
causing each manifest master to retrieve the content file from a local master server associated with the local cluster of servers.

3. The method of claim 2 , further comprising causing at least one local master server to wait a first predetermined period of time before retrieving the content file from one of the one or more regional master servers, and causing at least one manifest master server to wait a second predetermined period of time before retrieving the content file from the at least one local master server.

4. The method of claim 1 , wherein performing one or more operations to identify the one or more local master servers comprises performing a filtering operation on a hash data structure based on the first hash value.

5. The method of claim 4 , further comprising, for each manifest cluster of servers included in the at least one local cluster of servers:
filtering the hash data structure based on the first hash value to identify one or more manifest master servers included in the manifest cluster of servers; and
causing each manifest master to retrieve the content file from a local master server associated with the local cluster of servers.

6. The method of claim 4 , wherein the hash data structure comprises a hash ring, and the first hashing operation and the second hashing operation comprise consistent hashing operations.

7. The method of claim 4 , wherein performing the first hashing operation comprises, for each server ID:
combining the server ID with a plurality of different values to generate a plurality of different server ID instances;
performing the first hashing operation on each of the plurality of server ID instances to generate the plurality of hash values for the server IDs; and
storing the plurality of hash values for the server IDs in the hash data structure.

8. The method of claim 1 , wherein a number of regional master servers and a number of local master servers are based on at least one replication factor associated with the content file.

9. A control server, comprising:
one or more memories storing a control application; and
one or more processors coupled to the one or more memories, wherein, when executed by the one or more processors, the control application configures the one or more processors to:
perform a first hashing operation on a plurality of server identifiers (IDs) associated with servers included in a regional cluster of servers to generate a plurality of hash values for the server IDs, wherein each server ID included in the plurality of server IDs identifies a different server included in the regional cluster of servers;
perform a second hashing operation on a file ID associated with a content file stored at a third-party fill source to generate a first hash value;
perform one or more operations based on the first hash value to identify a first plurality of regional master servers included in the regional cluster of servers for the content file based on comparing the first hash value to one or more hash values included in the plurality of hash values for the server IDs that identify the first plurality of regional master servers;
cause each regional master server included in the first plurality of regional master servers to retrieve the content file from the third-party fill source via a network connection; and
for at least one local cluster of servers included in the regional cluster of servers:
perform one or more operations to identify one or more local master servers included in the at least one local cluster of servers; and
cause each local master server included in the one or more local master servers to retrieve the content file from one of the first plurality of regional master servers instead of retrieving the content file from the third-party fill source.

10. The control server of claim 9 , wherein the control application further configures the one or more processors to, for each manifest cluster of servers included in the at least one local cluster of servers:
performing one or more operations to identify one or more manifest master servers included in the manifest cluster of servers; and
causing each manifest master to retrieve the content file from a local master server associated with the local cluster of servers.

11. The control server of claim 10 , wherein the control application further configures the one or more processors to cause at least one local master server to wait a first predetermined period of time before retrieving the content file from one of the first plurality of regional master servers, and cause at least one manifest master server to wait a second predetermined period of time before retrieving the content file from the at least one local master server.

12. The control server of claim 9 , wherein the control application further configures the one or more processors to:
perform a third hashing operation based on a second file ID associated with a second content file stored at the third-party fill source to generate a second hash value;
compare the second hash value to one or more hash values included in the plurality of hash values for the server IDs to identify a second plurality of regional master servers included in the regional cluster of servers, wherein the second plurality of regional master servers includes at least one server that is not included in the first plurality of regional master servers;
cause each regional master server to retrieve the second content file from the third-party fill source; and
for the at least one local cluster of servers included in the regional cluster of servers:
perform a filtering operation on a hash data structure based on the second hash value to identify a plurality of local master servers included in the local cluster of servers, wherein the plurality of local master servers includes at least one server that is not included in the one or more local m

Assignees

• Netflix Inc

Inventors

• Chen Andrew
• Brand Christopher
• Ellis Daniel
• Gutarin Alex

Classifications

• G06F16/172 Primary

  Caching, prefetching or hoarding of files · CPC title

• H04L67/1023 Primary

  based on a hash applied to IP addresses or costs · CPC title

• G06F16/1824 Primary

  implemented using Network-attached Storage [NAS] architecture (distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097) · CPC title

• G06F16/183

  Provision of network file services by network file servers, e.g. by using NFS, CIFS (network file access protocols H04L67/1097) · CPC title

• G06F16/137

  Hash-based (content-based indexing of textual data G06F16/31) · CPC title