Orthogonal experimentation in a computing environment

Patent metadata

Field	Value
Publication number	US-10268569-B2
Application number	US-201715644626-A
Country	US
Kind code	B2
Filing date	Jul 7, 2017
Priority date	Mar 19, 2010
Publication date	Apr 23, 2019
Grant date	Apr 23, 2019

Abstract

Various embodiments include at least one of systems, methods, and software to receive input configuring tests within a computing environment to expose users to standard application or website experiences or test experiences. In some embodiments, multiple tests may be configured to run orthogonally within user experiences without affecting the results of one another. Some such embodiments preserve the ability to execute certain tests in a non-orthogonal manner while other tests are allowed to execute orthogonally.

First claim

What is claimed is:

1. A system comprising:
at least one processor;
at least one memory device;
at least one network interface device; and
instructions held in the at least one memory device, the instructions retrievable from the at least one memory device and executable by the at least one processor to cause the system to perform operations comprising:
defining a spectrum of incoming traffic values, having a minimum value and a maximum value;
receiving a webpage request including an identifier associated with a user;
converting the identifier into a value within the spectrum of incoming traffic values;
determining the value falls within a particular range from among a plurality of ranges in the spectrum of incoming traffic values, each of the plurality of ranges being allocated to one of a plurality of test experiences; and
responsive to determining that the value falls within the particular range, processing the webpage request according to the test experience associated with the particular range.

2. The system of claim 1 , wherein the operations further comprise:
identifying a first test experience and a second test experience;
allocating a first range in the spectrum of incoming traffic values to the first test experience;
allocating a second range in the spectrum of incoming traffic values to the second text experience;
wherein the determining the value falls within the particular range includes determining whether the value falls within the first range or the second range.

3. The system of claim 2 , wherein the operations further comprise allocating a third range in the spectrum of incoming traffic values to a non-test experience.

4. The system of claim 2 , wherein the first range and the second range have at least some overlapping values.

5. The system of claim 1 , wherein the operations further comprises:
receiving data designating a start date and an end date for when users will be exposed at least one of the plurality of test experiences.

6. The system of claim 1 , wherein the received webpage request is a request for a webpage to be built by at least one process that executes on the at least one processor.

7. The system of claim 1 , wherein the received webpage request is a request received via the at least one network interface device from a logical webpage server that executes to build a webpage including at least one portion generated and provided via the at least one network interface device to the logical webpage server by the processing of the webpage request according to the test experience associated with the particular range.

8. The system of claim 1 , wherein the identifier associated with a user is a globally unique identifier (GUID).

9. The system of claim 8 , wherein the GUID is received with a request for webpage related content.

10. The system of claim 1 , wherein converting the identifier into the value within the spectrum of incoming traffic values further comprising:
applying a hash value to the identifier to produce a value within the spectrum of incoming traffic values.

11. The system of claim 10 , further comprising responsive to determining that the value falls within at least two ranges from among the plurality of ranges in the spectrum, processing the webpage request according to at least two test experiences to which the at least two ranges are allocated.

12. A method comprising:
defining a spectrum of incoming traffic values, having a minimum value and a maximum value;
receiving a webpage request including an identifier associated with a user;
converting the identifier into a value within the spectrum of incoming traffic values;
determining the value falls within a particular range from among a plurality of ranges in the spectrum of incoming traffic values, each of the plurality of ranges being allocated to one of a plurality of test experiences; and
responsive to determining that the value falls within the particular range, processing the webpage request according to the test experience associated with the particular range.

13. The method of claim 12 , wherein the method further comprises:
receiving data designating a start date and an end date for when users will be exposed at least one of the plurality of test experiences.

14. The method of claim 12 , wherein the received webpage request is a request for a webpage to be built by at least one process that executes on the at least one processor.

15. The method of claim 12 , wherein the received webpage request is a request received via the at least one network interface device from a logical webpage server that executes to build a webpage including at least one portion generated and provided via the at least one network interface device to the logical webpage server by the processing of the webpage request according to the test experience associated with the particular range.

16. The method of claim 12 , wherein the identifier associated with a user is a globally unique identifier (GUID).

17. The method of claim 16 , wherein the GUID is received with the webpage request.

18. A non-transitory computer-readable storage medium, with instructions stored thereon, which when executed by at least one processor of a computer, causes the computer to perform a method comprising:
defining a spectrum of incoming traffic values, having a minimum value and a maximum value;
receiving a webpage request including an identifier associated with a user;
converting the identifier into a value within the spectrum of incoming traffic values;
determining the value falls within a particular range from among a plurality of ranges in the spectrum of incoming traffic values, each of the plurality of ranges being allocated to one of a plurality of test experiences; and
responsive to determining that the value falls within the particular range, processing the webpage request according to the test experience associated with the particular range.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the method further comprises:
receiving data designating a start date and an end date for when users will be exposed at least one of the plurality of test experiences.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the received webpage request is a request for a webpage to be built by at least one process that executes on the at least one processor.

Assignees

• Ebay Inc

Inventors

• Sahni Jasdeep Singh
• Madan Anil
• Nadig Deepak Seetharam
• Cheung Po
• Mistry Bhavesh
• Bodine John
• Lo Michael

Classifications

• G06F11/3696

  Methods or tools to render software testable · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F16/2255

  Hash tables · CPC title

• G06F16/958

  Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking · CPC title

• G06F16/954

  Navigation, e.g. using categorised browsing · CPC title