Batch-optimized render and fetch architecture utilizing a virtual clock

Patent metadata

Field	Value
Publication number	US-9984130-B2
Application number	US-201414521206-A
Country	US
Kind code	B2
Filing date	Oct 22, 2014
Priority date	Jun 26, 2014
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

Implementations include a batch-optimized render and fetch architecture. An example method performed by the architecture includes receiving a request from a batch process to render a web page and initializing a virtual clock and a task list for rendering the web page. The virtual clock stands still when a request for an embedded item is outstanding and when a task is ready to run. The method may also include generating a rendering result for the web page when the virtual clock matches a run time for a stop task in the task list, and providing the rendering result to the batch process. Another example method includes receiving a request from a batch process to render a web page, identifying an embedded item in the web page, and determining, based on a rewrite rule, that the embedded item has content that is duplicative of content for a previously fetched embedded item.

First claim

What is claimed is:

1. A computer system comprising:
at least one processor; and
memory storing:
a data store of content for embedded items, and
instructions that, when executed by the at least one processor, cause the system to:
receive a request, from a batch process, to render a web page,
initialize a virtual clock for the web page,
generate a task list for rendering the web page, wherein each task in the task list has an associated run time, wherein generating the task list includes adding a task for processing the content of the web page and adding a stop task for rendering the web page, the stop task having a run time set to a predetermined time added to the virtual clock,
perform the tasks in the task list according to the virtual clock for the web page, wherein the virtual clock for the web page advances by being set to a time represented by a next-occurring task in a task list for the web page, the virtual clock remaining unchanged while a pending task in the task list has a run time matching the virtual clock, wherein at least one task in the task list causes the system to:
identify an embedded item in the web page,
determine, based on a rewrite rule, that the embedded item has content that is duplicative of content for a previously fetched embedded item, and
in response to the determination, provide the content for the previously fetched embedded item from the data store, and

generate a rendering result for the web page when the virtual clock-matches a run time for the stop task in the task list, and
provide the rendering result to the batch process.

2. The system of claim 1 , wherein as part of determining that the embedded item has content that is duplicative of content for a previously fetched embedded item, the instructions further include instructions that, when executed by the at least one processor, cause the system to:
match the embedded item to a template of the rewrite rule, the rewrite rule also including a redirect identifier,
wherein providing the content for the previously fetched embedded item includes using the redirect identifier to locate the content for the previously fetched embedded item.

3. The system of claim 2 , wherein the template includes a URL without a query string.

4. The system of claim 1 , wherein the embedded item is a first embedded item and the instructions further include instructions that, when executed by the at least one processor, cause the system to:
identify a second embedded item in the web page;
determine whether the second embedded item is blacklisted;
return an error when the second embedded item is blacklisted without fetching content for the second embedded item; and
generate the rendering result without the content for the second embedded item.

5. The system of claim 1 , wherein the virtual clock does not advance while waiting for the provided content of the previously fetched embedded item.

6. A system comprising:
at least one processor; and
memory storing:
a data store of content for embedded items, and
instructions that, when executed by the at least one processor, cause the system to:
receive a request, from a batch process, to render a web page;
initialize a virtual clock for the web page;
generate a task list for rendering the web page, wherein each task in the task list has an associated run time, wherein generating the task list includes adding a task for processing the content of the web page and adding a stop task for rendering the web page, the stop task having a run time set to a predetermined time added to the virtual clock;
perform the tasks in the task list according to the virtual clock for the web page, wherein the virtual clock for the web page advances by being set to a time represented by a next-occurring task in a task list for the web page, the virtual clock remaining unchanged while a pending task in the task list has a run time matching the virtual clock, wherein at least one task in the task list causes the system to:
identify a first embedded item in the web page,
identify a first embedded item in the web page,
determine that the first embedded item includes an image, and
generate a mock image with empty content that specifies dimensions from a dimension table entry for the second embedded item;

generate a rendering result for the web page when the virtual clock matches a run time for the stop task in the task list, the rendering result using the content for the previously fetched embedded item and the mock image; and
provide the rendering result to the batch process.

7. A method comprising:
receiving a request, from a batch process, to render a web page;
initializing, using at least one processor, a virtual clock for the web page;
generating a task list for rendering the web page, wherein each task has an associated run time and wherein generating the task list includes adding a stop task with a run time set to a predetermined time added to the virtual clock;
performing the tasks in the task list according to the virtual clock for the web page, wherein the virtual clock for the web page advances by being set to a time represented by a next-occurring task in the task list, the virtual clock remaining unchanged while a pending task in the task list has a run time matching the virtual clock;
generating, using the at least one processor, a rendering result for the web page when the virtual clock matches a run time for a stop task in the task list; and
providing the rendering result to the batch process.

8. The method of claim 7 , wherein the predetermined time added to the virtual clock is between 5 and 20 seconds.

9. The method of claim 7 , wherein the batch processes includes an indexing engine and the method further comprises using the rendering result to rank tokens in an index.

10. The method of claim 7 , wherein outstanding requests for embedded items are pending tasks.

11. The method of claim 7 , further comprising:
identifying an embedded image in the web page;
requesting content for the embedded image;
receiving, in response to the request, a mock image that specifies dimensions for the embedded image but has empty content; and
using the mock image in generating the rendering result.

12. The method of claim 7 , wherein the batch process is an indexing engine and the method further comprises demoting a rank for the web page based on information in the rendering result.

13. The method of claim 7 , wherein the batch process is an indexing engine and the method further comprises using the rendering result to index dynamically generated content.

14. The method of claim 7 , wherein the rendering result includes an image and a layout that identifies coordinates of elements in the web page in the image.

15. The method of claim 7 , wherein the run time for the stop task represents an average web page loading time.

16. The system of claim 1 , wherein the instructions further include instructions that, when executed by the at least one processor, cause the system to:
add a task to the task list, responsive to receiving the content for the embedded item, the task having a respective run time set equal to a current value of the virtual clock.

17. The system of claim 16 , wherein the task added to the task list runs script identified in the content for the embedded item.

18. The system of claim 16 , wherein the task added to the task list fetches a second embedded item identified in the content.

19. The system of claim 1 , wherein the embedded item is a first embedded item and the instructi

Assignees

• Google Inc
• Google Llc

Inventors

• Xu Hui
• Kapoor Rupesh
• Hendriks Erik Arjan
• Fang Hao
• Tapus Cristian

Classifications

• G06F40/131

  Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces · CPC title

• G06F16/00 Primary

  Information retrieval; Database structures therefor; File system structures therefor · CPC title

• G06F40/186

  Templates · CPC title

• G06F16/9566

  URL specific, e.g. using aliases, detecting broken or misspelled links · CPC title

• G06F16/957

  Browsing optimisation, e.g. caching or content distillation · CPC title