Automated test multiplexing system

Patent metadata

Field	Value
Publication number	US-11446570-B2
Application number	US-202016870496-A
Country	US
Kind code	B2
Filing date	May 8, 2020
Priority date	May 8, 2020
Publication date	Sep 20, 2022
Grant date	Sep 20, 2022

Abstract

An imitation learning system may learn how to play a video game based on user interactions by a tester or other user of the video game. The imitation learning system may develop an imitation learning model based, at least in part, on the tester's interaction with the video game and the corresponding state of the video game to determine or predict actions that may be performed when interacting with the video game. The imitation learning system may use the imitation learning model to control automated agents that can play additional instances of the video game. Further, as the user continues to interact with the video game during testing, the imitation learning model may continue to be updated. Thus, the interactions by the automated agents with the video game may, over time, almost mimic the interaction by the user enabling multiple tests of the video game to be performed simultaneously.

First claim

What is claimed is:

1. A computer-implemented method comprising:
as implemented by an interactive computing system configured with specific computer-executable instructions,
accessing an imitation learning model of a video game under test hosted by a user computing system, the imitation learning model generated at a first time based on a set of user interactions with a user-controlled instance of the video game; and
for each of a set of agent-controlled instances of the video game, wherein the set of agent-controlled instances of the video game are hosted by a set of computing systems separate from the user computing system:
receiving first state information at a second time for the agent-controlled instance of the video game, wherein the second time is later than the first time, and wherein the first state information is indicative of an execution state of the agent-controlled instance of the video game;
applying the first state information to the imitation learning model to obtain a first simulated user action; and
causing an imitation learning (IL) agent to perform the first simulated user action with respect to the agent-controlled instance of the video game,

wherein the first state information for at least one of the set of agent-controlled instances of the video game differs from the first state information for at least one other of the set of agent-controlled instances of the video game.

2. The computer-implemented method of claim 1 , further comprising:
receiving an indication of a user action performed with respect to the user-controlled instance of the video game; and
modifying the imitation learning model of the video game based at least in part on the indication of the user action to obtain an updated imitation learning model.

3. The computer-implemented method of claim 2 , wherein the user action is performed subsequent to the set of user interactions used to generate the imitation learning model.

4. The computer-implemented method of claim 2 , further comprising, for each of the set of agent-controlled instances of the video game:
receiving second state information at a third time for the agent-controlled instance of the video game, wherein the third time is later than the second time;
applying the second state information to the updated imitation learning model to obtain a second simulated user action; and
causing the imitation learning agent to perform the second simulated user action with respect to the agent-controlled instance of the video game.

5. The computer-implemented method of claim 2 , further comprising receiving state information for the user-controlled instance of the video game, wherein the imitation learning model of the video game is modified based at least in part on the indication of the user action and the state information.

6. The computer-implemented method of claim 5 , wherein the state information comprises pre-state information corresponding to the state of the user-controlled instance of the video game prior to performance of the user action, post-state information corresponding to the state of the user-controlled instance of the video game after performance of the user action, or both pre-state information and post-state information.

7. The computer-implemented method of claim 2 , further comprising receiving a weighting associated with the user action, wherein the modifying of the imitation learning model to obtain the updated imitation learning model is based at least in part on the indication of the user action and the weighting associated with the user action.

8. The computer-implemented method of claim 1 , wherein the set of agent-controlled instances of the video game comprises one or more instances of the video game.

9. The computer-implemented method of claim 1 , further comprising, upon detecting a trigger condition at an agent-controlled instance of the video game of the set of agent-controlled instances of the video game, performing a remediation action.

10. The computer-implemented method of claim 9 , wherein the remediation action comprises providing control of the agent-controlled instance of the video game to a user.

11. The computer-implemented method of claim 9 , wherein the remediation action comprises modifying a weighting of a state within the imitation learning model.

12. The computer-implemented method of claim 9 , wherein the trigger condition comprises: a threshold number of occurrences of a video game state, an error, or an occurrence of an unexpected state, wherein the unexpected state comprises a state not included in the imitation learning model.

13. A system comprising:
an electronic data store configured to store an imitation learning model generated based on a set of user interactions with a user-controlled instance of a video game hosted by a user computing system; and
a hardware processor of a test system in communication with the electronic data store, the hardware processor configured to execute specific computer-executable instructions to at least:
access the imitation learning model of the video game from the electronic data store;
receive first state information for an agent-controlled instance of the video game, wherein the agent-controlled instance of the video game is hosted by an imitation learning client computing system separate from the user computing system;
apply the first state information to the imitation learning model to obtain a first simulated user action, wherein the first state information is indicative of an execution state of the agent-controlled instance of the video game; and
provide the first simulated action to an imitation learning agent configured to test the agent-controlled instance of the video game, wherein the imitation learning model comprises a dynamically generated imitation learning model that is updated during execution of the agent-controlled instance of the video game based on a user interaction with the user-controlled instance of the video game.

14. The system of claim 13 , wherein the test system comprises the imitation learning client computing system.

15. The system of claim 13 , wherein the hardware processor is further configured to execute specific computer-executable instructions to at least:
receive an updated imitation learning model;
receive second state information for the agent-controlled instance of the video game, wherein the second state information is received later than the first state information;
apply the second state information to the updated imitation learning model to obtain a second simulated user action; and
provide the second simulated user action to the imitation learning agent to perform the second simulated user action with respect to the agent-controlled instance of the video game.

16. The system of claim 13 , wherein the imitation learning agent is hosted by the imitation learning client computing system comprising computer hardware that is separate from the test system.

17. The system of claim 13 , wherein the hardware processor is further configured to execute specific computer-executable instructions to at least:
receive an indication of the user interaction performed with respect to the user-controlled instance of the video game; and
modify the imitation learning model of the video game based at least in part on the indication of the user interaction to obtain an updated imitation learning model.

18. The system of claim 17 , wherein the hardware processor is further configured to execute specific computer-executable instructions to at least:
receive second state information fo

Assignees

• Electronic Arts Inc

Inventors

• Borovikov Igor
• Harder Jesse Hans Stokes
• O'Neill Thomas Patrick
• Rein Jonathan Albert
• Lee Avery H
• Wrotek Pawel Piotr
• Parker Graham Michael
• Vincent David

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• A63F13/60

  Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor · CPC title

• A63F13/67

  adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use · CPC title

• G06F11/3684

  for test design, e.g. generating new test cases · CPC title