A/B testing for search engine optimization

Patent metadata

Field	Value
Publication number	US-10839406-B2
Application number	US-201816022291-A
Country	US
Kind code	B2
Filing date	Jun 28, 2018
Priority date	Jun 28, 2018
Publication date	Nov 17, 2020
Grant date	Nov 17, 2020

Abstract

The disclosed embodiments provide a system for performing A/B testing for search engine optimization (SEO). During operation, the system divides a set of web pages into a control group, an inbound treatment group, an outbound treatment group, and a full treatment group. Next, the system generates a first set of links from the outbound treatment group to the inbound treatment group and a second set of links within the full treatment group. The system then applies an A/B test to pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group. Finally, the system outputs a result of the A/B test for use in assessing an effect of the first and second sets of links on search engine results associated with the set of web pages.

First claim

What is claimed is:

1. A method comprising:
dividing, by one or more computer systems, a set of web pages into a control group, an inbound treatment group, an outbound treatment group, and a full treatment group;
wherein the control group includes one or more webpages that are not affected by inbound links or outbound links in an A/B test;
wherein the inbound treatment group includes one or more webpages that are affected by inbound links in the A/B test but not affected by outbound links in the A/B test;
wherein the outbound treatment group includes one or more webpages that are affected by outbound links in the A/B test but not affected by inbound links in the A/B test;
wherein the full treatment group includes one or more webpages that are affected by inbound links and outbound links in the A/B test;
generating, by the one or more computer systems, a first set of links from the outbound treatment group to the inbound treatment group and a second set of links within the full treatment group;
applying the A/B test to pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group; and
outputting a result of the A/B test for use in assessing an effect of the first and second sets of links on search engine results, or user engagement metrics, associated with the set of web pages.

2. The method of claim 1 , wherein dividing the set of web pages into the control group, the inbound treatment group, the outbound treatment group, and the full treatment group comprises:
randomly assigning each web page in the set of web pages to a group selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group; and
verifying a statistical equivalence of one or more outcome variables across the control group, the inbound treatment group, the outbound treatment group, and the full treatment group.

3. The method of claim 2 , wherein the one or more outcome variables comprise at least one of:
a user engagement metric;
an index rate; or
a crawl rate.

4. The method of claim 1 , wherein generating the first and second sets of links comprises:
generating a set of links between pairs of web pages in the set of web pages; and
removing, from the set of links, inbound links to the control group and outbound links from the control group.

5. The method of claim 4 , wherein generating the first and second sets of links further comprises:
removing, from the set of links, additional outbound links from the inbound treatment group and additional inbound links to the outbound treatment group.

6. The method of claim 1 , wherein generating the first and second sets of links comprises:
determining a set of relevance scores between pairs of web pages in the set of web pages; and
generating, based on the set of relevance scores, the set of links to connect pairs of relevant web pages in the set of web pages.

7. The method of claim 6 , wherein generating the set of links to connect the pairs of relevant web pages comprises:
producing, based on the set of relevance scores, a set of clusters of relevant web pages from the set of web pages; and
for each cluster in the set of clusters, adding links between the pairs of relevant web pages in said each cluster.

8. The method of claim 1 , wherein applying the A/B test to the pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group comprises:
performing the A/B test on the control group and the inbound treatment group to assess an inbound effect of the first set of links on the search engine results.

9. The method of claim 1 , wherein applying the A/B test to the pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group comprises:
performing the A/B test on the outbound treatment group and the full treatment group to assess an inbound effect of the first and second sets of links on the search engine results.

10. The method of claim 1 , wherein applying the A/B test to the pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group comprises:
performing the A/B test on the control group and the outbound treatment group to assess an outbound effect of the first set of links on the search engine results.

11. The method of claim 1 , wherein applying the A/B test to the pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group comprises:
performing the A/B test on the inbound treatment group and the full treatment group to assess an outbound effect of the first and second sets of links on the search engine results.

12. The method of claim 1 , wherein applying the A/B test to the pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group comprises:
performing the A/B test on the control group and the full treatment group to assess an overall effect of the second set of links on the search engine results.

13. The method of claim 1 , wherein applying the A/B test to the pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group comprises:
performing an A/A test on the control group to assess a network effect associated with the first and second sets of links.

14. The method of claim 1 , wherein the effect of the first and second sets of links is on search engine results associated with the set of web pages.

15. A system, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
divide a set of web pages into a control group, an inbound treatment group, an outbound treatment group, and a full treatment group;
wherein the control group includes one or more webpages that are not affected by inbound links or outbound links in an A/B test;
wherein the inbound treatment group includes one or more webpages that are affected by inbound links in the A/B test but not affected by outbound links in the A/B test;
wherein the outbound treatment group includes one or more webpages that are affected by outbound links in the A/B test but not affected by inbound links in the A/B test;
wherein the full treatment group includes one or more webpages that are affected by inbound links and outbound links in the A/B test;
generate a first set of links from the outbound treatment group to the inbound treatment group and a second set of links within the full treatment group;
apply the A/B test to pairs of groups selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group; and
output a result of the A/B test for use in assessing an effect of the first and second sets of links on search engine results, or user engagement metrics, associated with the set of web pages.

16. The system of claim 15 , wherein dividing the set of web pages into the control group, the inbound treatment group, the outbound treatment group, and the full treatment group comprises:
randomly assigning each web page in the set of web pages to a group selected from the control group, the inbound treatment group, the outbound treatment group, and the full treatment group; and
verifying a statistical equivalence of one or more outcome variables across the control group, the inb

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wang Quan
• Hoang Huan V
• Huang Shen
• Zhang Yongzheng
• Kuan Chi-Yi

Classifications

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title

• G06Q30/0201 Primary

  Market modelling; Market analysis; Collecting market data · CPC title

• G06F16/24578

  using ranking · CPC title