Orthogonal experimentation in a computing environment

Patent metadata

Field	Value
Publication number	US-9703685-B2
Application number	US-201615043873-A
Country	US
Kind code	B2
Filing date	Feb 15, 2016
Priority date	Mar 19, 2010
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

Various embodiments include at least one of systems, methods, and software to receive input configuring tests within a computing environment to expose users to standard application or website experiences or test experiences. In some embodiments, multiple tests may be configured to run orthogonally within user experiences without affecting the results of one another. Some such embodiments preserve the ability to execute certain tests in anon-orthogonal manner while other tests are allowed to execute orthogonally.

First claim

What is claimed is:

1. A system comprising:
at least one processor;
at least one memory device;
at least one network interface device; and
instructions held in the at least one memory device, the instructions retrievable from the at least one memory device and executable by the at least one processor to cause the system to perform a method comprising:
identifying a first test experience and a second test experience;

defining a spectrum of incoming traffic values, having a minimum value and a maximum value;
allocating a first range in the spectrum of incoming traffic values to the first test experience;
allocating a second range in the spectrum of incoming traffic values to the second text experience;
receiving a web page request including an identifier associated with a user;
converting the identifier into a value within the spectrum of incoming traffic values;
determining whether the value falls within the first range or the second range; and
responsive to determining that the value falls within the first range or the second range, processing the web page request according to the test experience associated with the range that the value falls into.

2. The system of claim 1 , wherein the method further comprises:
receiving data designating a start date and an end date for when users will be exposed at least one of the first test experience and the second test experience.

3. The system of claim 1 , wherein the received request for webpage related content is a request for a webpage to be built by at least one process that executes on the at least one processor.

4. The system of claim 1 , wherein the received web page request is a request received via the at least one network interface device from a logical webpage server that executes to build a webpage including at least one portion generated and provided via the at least one network interface device to the logical webpage server by the processing of the request for the webpage related content according to any of the identified test experiences.

5. The system of claim 1 , wherein the identifier associated with a user is a globally unique identifier (GUID).

6. The system of claim 5 , wherein the GUID is received with a request for webpage related content.

7. The system of claim 1 , wherein a third range in the spectrum of incoming traffic values is not assigned to any test experience.

8. The system of claim 1 , wherein converting the identifier into a value within the spectrum of incoming traffic values further comprising:
applying a hash value to the identifier to produce a value within the spectrum of values.

9. The system of claim 1 , wherein the first range and the second range have at least some overlapping values.

10. The system of claim 9 , further comprising responsive to determining that the value falls within the first range and the second range, processing the web page request according to both test experiences.

11. A method comprising:
identifying a first test experience and a second test experience;
defining a spectrum of incoming traffic values, having a minimum value and a maximum value;
allocating a first range in the spectrum of incoming traffic values to the first test experience;
allocating a second range in the spectrum of incoming traffic values to the second text experience;
receiving a web page request including an identifier associated with a user;
converting the identifier into a value within the spectrum of incoming traffic values;
determining whether the value falls within the first range or the second range; and
responsive to determining that the value falls within the first range or the second range, processing the web page request according to the test experience associated with the range that the value falls into.

12. The method of claim 11 , wherein the method further comprises:
receiving data designating a start date and an end date for when users will be exposed to at least one of the first test experience and the second test experience.

13. The method of claim 11 , wherein the received request for webpage related content is a request for a webpage to be built by at least one process that executes on the at least one processor.

14. The method of claim 11 , wherein the received web page request is a request received via the at least one network interface device from a logical webpage server that executes to build a webpage including at least one portion generated and provided via the at least one network interface device to the logical webpage server by the processing of the request for the webpage related content according to any of the identified test experiences.

15. The method of claim 11 , wherein the identifier associated with a user is a globally unique identifier (GUID).

16. The method of claim 15 , wherein the GUID is received with a request for webpage related content.

17. A non-transitory computer-readable storage medium, with instructions stored thereon, which when executed by at least one processor of a computer, causes the computer to perform a method comprising:
identifying a first test experience and a second test experience;
defining a spectrum of incoming traffic values, having a minimum value and a maximum value;
allocating a first range in the spectrum of incoming traffic values to the first test experience;
allocating a second range in the spectrum of incoming traffic values to the second text experience;
receiving a web page request including an identifier associated with a user;
converting the identifier into a value within the spectrum of incoming traffic values;
determining whether the value falls within the first range or the second range; and
responsive to determining that the value falls within the first range or the second range, processing the web page request according to the test experience associated with the range that the value falls into.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises:
receiving data designating a start date and an end date for when users will be exposed to at least one of the first test experience and the second test experience.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the received request for webpage related content is a request for a webpage to be built by at least one process that executes on the at least one processor.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the received web page request is a request received via the at least one network interface device from a logical webpage server that executes to build a webpage including at least one portion generated and provided via the at least one network interface device to the logical webpage server by the processing of the request for the webpage related content according to any of the identified test experiences.

Assignees

• Ebay Inc

Inventors

• Sahni Jasdeep Singh
• Madan Anil
• Nadig Deepak Seetharam
• Cheung Po
• Mistry Bhavesh
• Bodine John
• Lo Michael

Classifications

• G06F11/3696

  Methods or tools to render software testable · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F16/2255

  Hash tables · CPC title

• G06F16/958

  Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking · CPC title

• G06F16/954

  Navigation, e.g. using categorised browsing · CPC title