Tiered cache filling

Patent metadata

Field	Value
Publication number	US-11675740-B2
Application number	US-202117322714-A
Country	US
Kind code	B2
Filing date	May 17, 2021
Priority date	Apr 30, 2015
Publication date	Jun 13, 2023
Grant date	Jun 13, 2023

Abstract

One embodiment of the present invention sets forth a technique for replicating a file within a network of servers. The technique includes determining one or more regional master servers included in a regional cluster of servers and causing each regional master server to retrieve a file from a fill source. The technique further includes, for at least one local cluster of servers included in the regional cluster of servers, determining one or more local master servers included in the at least one local cluster of servers, and causing each local master server to retrieve the file from one of the one or more regional master servers.

First claim

What is claimed is:

1. A computer-implemented method for caching file data within a computer network, comprising:
performing a first hashing operation on a file ID associated with a content file stored at a third-party fill source to generate a first hash value;
performing a second hashing operation on a plurality of server identifiers (IDs) to generate a plurality of hash values for the server IDs;
comparing the first hash value to the plurality of hash values for the server IDs to identify a first regional master server included in a regional cluster of servers;
causing the first regional master server to retrieve the content file from the third-party fill source via a network connection; and
for a first local cluster of servers included in the regional cluster of servers:
performing one or more operations to identify a first local master server included in the first local cluster of servers; and
causing the first local mater server to retrieve the content file from the first regional mater server instead of retrieving the content file from the third-party fill source.

2. The computer-implemented method of claim 1 , further comprising, for a first manifest cluster of servers included in the first local cluster of servers:
performing one or more operations to identify a first manifest master server included in the first manifest cluster of servers; and
causing the first manifest master server to retrieve the content file from the first local master server.

3. The computer-implemented method of claim 2 , further comprising causing the first local master server to wait a first predetermined period of time before retrieving the content file from the first regional master server, and causing the first manifest master server to wait a second predetermined period of time before retrieving the content file from the first local master server.

4. The computer-implemented method of claim 1 , wherein the plurality of server IDs identify different servers include in the regional cluster of servers.

5. The computer-implemented method of claim 1 , further comprising performing a filtering operation on a hash data structure based on the first hash value.

6. The computer-implemented method of claim 5 , further comprising, for a first manifest cluster of servers included in the first local cluster of servers:
filtering the hash data structure based on the first hash value to identify a first manifest master server included in the first manifest cluster of servers; and
causing the first manifest master server to retrieve the content file from the first local master server.

7. The computer-implemented method of claim 5 , wherein the hash data structure comprises a hash ring, and the first hashing operation and the second hashing operation comprise consistent hashing operations.

8. The computer-implemented method of claim 5 , wherein performing the second hashing operation comprises, for each server ID:
combining the server ID with a plurality of different values to generate a plurality of different server ID instances;
hashing each of the plurality of server ID instances to generate the plurality of hash values for the server IDs; and
storing the plurality of hash values for the server IDs in the hash data structure.

9. The computer-implemented method of claim 1 , wherein a number of regional master servers and a number of local master servers are based on at least one replication factor associated with the content file.

10. A computer system, comprising:
one or more memories storing instructions; and
one or more processors that are coupled to the one or more memories and, when executing the instructions, perform the steps of:
performing a first hashing operation on a file ID associated with a content file stored at a third-party fill source to generate a first hash value;
performing a second hashing operation on a plurality of server identifiers (IDs) to generate a plurality of hash values for the server IDs;
comparing the first hash value to the plurality of hash values for the server IDs to identify a first regional master server included in a regional cluster of servers;
causing the first regional master server to retrieve the content file from the third-party fill source via a network connection; and
for a first local cluster of servers included in the regional cluster of servers:
performing one or more operations to identify a first local master server included in the first local cluster of servers; and
causing the first local mater server to retrieve the content file from the first regional mater server instead of retrieving the content file from the third-party fill source.

11. The computer system of claim 10 , wherein the one or more processors, for a first manifest cluster of servers included in the first local cluster of servers, further perform the steps of:
performing one or more operations to identify a first manifest master server included in the first manifest cluster of servers; and
causing the first manifest master server to retrieve the content file from the first local master server.

12. The computer system of claim 11 , wherein the one or more processors further perform the steps of causing the first local master server to wait a first predetermined period of time before retrieving the content file from the first regional master server, and causing the first manifest master server to wait a second predetermined period of time before retrieving the content file from the first local master server.

13. The computer system of claim 11 , wherein the plurality of server IDs identify different servers include in the regional cluster of servers.

14. The computer system of claim 10 , wherein the one or more processors further perform the step of performing a filtering operation on a hash data structure based on the first hash value.

15. The computer system of claim 14 , wherein the one or more processors, for a first manifest cluster of servers included in the first local cluster of servers, further perform the steps of:
filtering the hash data structure based on the first hash value to identify a first manifest master server included in the first manifest cluster of servers; and
causing the first manifest master server to retrieve the content file from the first local master server.

16. The computer system of claim 14 , wherein the hash data structure comprises a hash ring, and the first hashing operation and the second hashing operation comprise consistent hashing operations.

17. The computer system of claim 14 , wherein, when performing the second hashing operation comprises, the one or more processors, for each server ID, further perform the steps of:
combining the server ID with a plurality of different values to generate a plurality of different server ID instances;
hashing each of the plurality of server ID instances to generate the plurality of hash values for the server IDs; and
storing the plurality of hash values for the server IDs in the hash data structure.

18. The computer system of claim 10 , wherein a first number of servers included in a first plurality of regional master servers is based on a first replication factor associated with a regional master server tier, and a second number of servers included in a plurality of local master servers is based on a second replication factor associated with a local master server tier.

19. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to per

Assignees

• Netflix Inc

Inventors

• Chen Andrew
• Brand Christopher
• Ellis Daniel P
• Gutarin Alex

Classifications

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F16/172 Primary

  Caching, prefetching or hoarding of files · CPC title

• H04L67/1023 Primary

  based on a hash applied to IP addresses or costs · CPC title

• G06F16/137

  Hash-based (content-based indexing of textual data G06F16/31) · CPC title

• G06F16/1824 Primary

  implemented using Network-attached Storage [NAS] architecture (distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097) · CPC title