System and method for content selection for web page indexing

Patent metadata

Field	Value
Publication number	US-10324984-B2
Application number	US-201113287568-A
Country	US
Kind code	B2
Filing date	Nov 2, 2011
Priority date	May 5, 2009
Publication date	Jun 18, 2019
Grant date	Jun 18, 2019

Abstract

An indexing system for documents such as web pages divides a document into elements, such as document object model elements. User attention data from prior interactions with the document are analyzed to determine those elements of a document that satisfy a threshold requirement of user attention. Elements meeting the user attention threshold requirement are added to a set of indexable content for the document. Furthermore, document sections are determined based on attention data and each section is indexed separately. Indexing is per section and based only on the indexable content, thereby enhancing the index relevance, increasing the efficiency of search engines and reducing spamdexing.

First claim

What is claimed is:

1. A method of selecting content of a web page for indexing, the method comprising:
transmitting, by an indexer server, a request to retrieve web page content from a web page data store, the request including a uniform resource locator (URL) of a web page, wherein the indexer server comprises one or more computer systems configured to performing indexing of web page documents within web page data stores;
receiving, by the indexer server, the web page content from the web page data store, based on the URL;
requesting, by the indexer server, interaction data associated with the URL from an interaction data store;
receiving, by the indexer server, interaction data from the interaction data store for the URL that identifies one or more attention events, each attention event based on a previous user interface event detected within the web page content of the URL, and each attention event including a location within the web page where the respective user interface event was detected;
dividing, by the indexer server, the web page content into a plurality of non-overlapping content sections;
for each particular content section of the plurality of content sections:
analyzing, by the indexer server, the locations of the one or more attention events in the received interaction data to determine whether the particular content section is associated with one or more of the attention events;
determining whether the particular content section of the web page is found within one or more additional web pages stored within the web page data store;
in response to determining that the particular content section of the web page is found within one or more additional web pages stored within the web page data store:
(a) analyzing the locations of a plurality of additional attention events associated with the one or more additional web pages, to determine whether the particular content section is associated with one or more of the plurality of additional attention events; and
(b) summing the number of attention events associated with the particular content section within the web page, and the number of additional attention events associated with the particular content section within the one or more additional web pages; and

in response to a determination that the sum of the numbers of attention events associated with the particular content section within the web page and within the additional web pages, is greater than a threshold of attention events, adding the particular content section to an indexable content of the web page; and

outputting, by the indexer server, the indexable content corresponding to the web page.

2. The method of claim 1 , further comprising:
indexing the web page based on the indexable content corresponding to the web page.

3. The method of claim 1 , wherein dividing the web page content into the plurality of non-overlapping content sections comprises dividing the web page into a plurality of document object elements associated with a corresponding document object model (DOM), and wherein the method further comprises:
retrieving a plurality of DOM element identifiers (IDs) for all logged interactions;
analyzing the plurality of DOM element IDs; and
selecting a DOM element of the web page.

4. The method of claim 3 , further comprising:
determining whether the selected DOM element is found in an interaction data set of the plurality of DOM element IDs;
selecting a next DOM element for the web page if the selected DOM element is found in the interaction data set of DOM element IDs;
analyzing the interaction data for the DOM element to determine if the DOM element is associated with attention events above the threshold if the interaction data does exist for the selected DOM element; and
adding the DOM element to an indexable content of the web page if the DOM element satisfies the threshold.

5. The method of claim 4 , wherein the threshold is based on human classified attention events.

6. The method of claim 1 , further comprising:
processing each of the particular content sections separately and indexing the corresponding web page sections into separate section indexes separate from other section indexes.

7. The method of claim 6 , further comprising:
identifying an attention history of each of a plurality of elements of each section; and
selecting each of the plurality of elements which meet a predefined attention history requirement; and
adding the plurality of selected elements to indexable content for a respective content section of the web page.

8. An apparatus configured to select content of a web page, the apparatus comprising:
a transmitter configured to transmit a request to retrieve web page content from a web page data store, the request including a uniform resource locator (URL) of a web page; and
a processor configured to
receive the web page content based on the URL;
request interaction data associated with the URL from an interaction data store;
receive interaction data for the URL that identifies one or more attention events, each attention event based on a previous user interface event detected within the web page content of the URL, and each attention event including a location within the web page where the respective user interface event was detected;
divide the web page content into a plurality of non-overlapping content sections;
for each particular content section of the plurality of content sections:
analyzing the locations of the one or more attention events in the received interaction data to determine whether the particular content section is associated with one or more of the attention events;
determining whether the particular content section of the web page is found within one or more additional web pages stored within the web page data store;
in response to determining that the particular content section of the web page is found within one or more additional web pages stored within the web page data store:
(a) analyzing the locations of a plurality of additional attention events associated with the one or more additional web pages, to determine whether the particular content section is associated with one or more of the plurality of additional attention events; and
(b) summing the number of attention events associated with the particular content section within the web page, and the number of additional attention events associated with the particular content section within the one or more additional web pages; and

in response to a determination that the sum of the numbers of attention events associated with the particular content section within the web page and within the additional web pages, is greater than a threshold of attention events, add the particular content section to an indexable content of the web page; and
output the indexable content corresponding to the web page.

9. The apparatus of claim 8 , wherein the processor is further configured to:
index the web page based on the indexable content corresponding to the web page.

10. The apparatus of claim 8 , wherein dividing the web page content into the plurality of non-overlapping content sections comprises dividing the web page into a plurality of document object elements associated with a corresponding document object model (DOM), and wherein the processor is further configured to:
retrieve a plurality of DOM element identifiers (IDs) for all logged interactions;
analyze the plurality of DOM element IDs; and
select a DOM element of the web page.

11. The apparatus of claim 10 , wherein the processor is further configured to:
determine whether the selected DOM element is found in an interaction data set of the plurality of DOM e

Assignees

• Hauser Robert R
• Oracle America Inc

Inventors

• Hauser Robert R

Classifications

• G06F16/951 Primary

  Indexing; Web crawling techniques · CPC title

• G06F16/81

  Indexing, e.g. XML tags; Data structures therefor; Storage structures · CPC title