Predictively controlling an environmental control system using upper confidence bound for trees

Patent metadata

Field	Value
Publication number	US-2016201934-A1
Application number	US-201514597198-A
Country	US
Kind code	A1
Filing date	Jan 14, 2015
Priority date	Jan 14, 2015
Publication date	Jul 14, 2016
Grant date	—

Abstract

In an embodiment, an electronic device may include a processor that may iteratively simulate candidate control trajectories using upper confidence bound for trees (UCT) to control an environmental control system (e.g., an HVAC system). Each candidate control trajectory may be simulated by selecting a control action at each of a plurality of time steps over a period of time that has the highest upper bound on possible performance using values from previous simulations and predicting a temperature for a next time step of the plurality of time steps that results from applying the selected control action using a thermal model. The processor may determine a value of each candidate control trajectory using a cost function, update the value of each control action selected in each candidate control trajectory, and select a candidate control trajectory with the highest value using UCT to apply to control the environmental control system.

First claim

1 . A non-transitory computer readable medium storing instructions thereon, the instructions, when executed by a processor of an electronic device, configured to:
simulate control trajectories to control an environmental control system of a structure for a forthcoming period of time including selecting a control action at each of a plurality of time steps of the forthcoming period of time using upper confidence bound for trees (UCT); determine a value of each control trajectory using a cost function and update the value of each control action selected in each control trajectory; and select a control trajectory with the highest value as the control trajectory to apply to control the environmental control system.

2 . The computer readable medium of claim 1 , wherein selecting the control action at each of the plurality of time steps includes selecting the control action that has the highest upper bound on possible performance using the value from previous simulations.

3 . The computer readable medium of claim 2 , wherein the instructions are configured to select a default action as the control action for each of the plurality of time steps using bang-bang control if there are no previous simulations.

4 . The computer readable medium of claim 2 , wherein selecting the control action at each of the plurality of time steps includes predicting temperatures at subsequent time steps that result from applying the selected control action at each of the plurality of time steps using a thermal model of the structure.

5 . The computer readable medium of claim 1 , wherein the cost function comprises at least three weighted factors related to environmental control system efficiency, temperature comfort level, and environmental control system wear-and-tear.

6 . The computer readable medium of claim 5 , wherein the weighted environmental control system efficiency factor represents the environmental control system runtime based on the control trajectories, the weighted temperature comfort level represents an error between a target temperature and predicted temperatures derived using a thermal model of a structure, and the weighted environmental control system wear-and-tear factor represents environmental control system cycle length based on the control trajectories.

7 . A method, comprising:
simulating, via a processor, control trajectories to control an environmental control system of a structure for a forthcoming period of time using upper confidence bound for trees (UCT), wherein each control trajectory is simulated by selecting a control action to control the environmental control system at each of a plurality of time steps for the forthcoming period of time based on the highest upper bound on possible performance using rewards from previous simulations; determining, via the processor, a reward for each control trajectory; and selecting, via the processor, a control trajectory with the highest reward as the control trajectory to apply to control the environmental control system.

8 . The method of claim 7 , comprising updating the reward of each control action selected in each control trajectory after the reward is determined for each control trajectory.

9 . The method of claim 7 , comprising predicting a temperature of a next time step using the selected control action at each time step and a thermal model to generate a trajectory of predicted temperatures.

10 . The method of claim 7 , wherein the reward is determined using a cost function.

11 . The method of claim 10 , wherein the cost function comprises weighted factors related to environmental control system efficiency, temperature comfort level provided by the environmental control system, and environmental control system wear-and-tear.

12 . The method of claim 11 , wherein the weights of the factors are adjusted based on user preference.

13 . A system, comprising:
an environmental control system; and an electronic device, comprising a processor, the processor being configured to:
control the environmental control system of a structure by applying a control trajectory, wherein the control trajectory is selected from a plurality of candidate control trajectories simulated using upper confidence bound for trees (UCT) based on having the highest value as determined by a cost function, each of the plurality of candidate control trajectories including a selected control action at each of a plurality of time steps over a period of time and a predicted temperature trajectory resulting from applying each control action at each of the plurality of time steps according to a thermal model of the structure.

14 . The system of claim 13 , wherein the thermal model includes factors related to a state of the structure comprising target temperature, current indoor temperature, indoor temperature on a previous time step, indoor temperature rate, current environmental control system state, number of seconds into current environmental control system state, time of day, outdoor temperature, current schedule mode, or some combination thereof.

15 . The system of claim 13 , wherein the processor is configured to determine whether to cancel the applied control trajectory based on whether a measured difference between the predicted temperature trajectory and an indoor temperature exceeds a threshold and, if so, to apply bang-bang control.

16 . The system of claim 13 , wherein the processor is configured to control the environmental control system according to the control trajectory while the indoor temperature is within a maintenance band surrounding a setpoint temperature and to apply bang-bang control when the indoor temperature exceeds the maintenance band.

17 . The system of claim 16 , wherein the processor is configured to expand the maintenance band prior to upcoming setpoint changes.

18 . The system of claim 13 , wherein the thermal model is dynamically generated by the processor and updated at least once a day with historical data, is selected from a number of preloaded thermal models on the electronic device based on the type of structure in which the environmental control system is installed or geographic location of the structure, is downloaded onto the electronic device from an external server, or some combination thereof.

19 . An electronic device, comprising:
a power source configured to provide operational power to the electronic device; and a processor coupled to the power source, the processor being configured to:
iteratively simulate candidate control trajectories to control an environmental control system of a structure using upper confidence bound for trees (UCT), wherein each candidate control trajectory is simulated by selecting a control action to control the environmental control system at each of a plurality of time steps over a period of time that has the highest upper bound on possible performance using value estimates from previous simulations and predicting a temperature for a next time step of the plurality of time steps that results from applying the selected control action using a thermal model of the structure;
determine a value estimate of each candidate control trajectory using a cost function and update the value estimate of each control action selected in each candidate control trajectory;
select a control trajectory with the highest value using UCT; and
apply the selected candidate control trajectory to control the environmental control system.

20 . The electronic device of claim 19 , wherein the cost function balances a temperature er

Assignees

• Google Inc

Inventors

• Hester Todd Andrew
• Fisher Evan Jarman
• Khandelwal Piyush

Classifications

• F24F11/48

  prior to normal operation, e.g. pre-heating or pre-cooling · CPC title

• F24F11/46 Primary

  Improving electric energy efficiency or saving · CPC title

• F24F11/58

  using Internet communication · CPC title

• F24F11/64

  using pre-stored data · CPC title

• F24F11/30

  for purposes related to the operation of the system, e.g. for safety or monitoring · CPC title