Building control system using reinforcement learning

Patent metadata

Field	Value
Publication number	US-12013673-B2
Application number	US-202117537179-A
Country	US
Kind code	B2
Filing date	Nov 29, 2021
Priority date	Nov 29, 2021
Publication date	Jun 18, 2024
Grant date	Jun 18, 2024

Abstract

Sensorized commercial buildings are a rich target for building a new class of applications that improve operational and energy efficiency of building operations that take into account human activities. Such applications, however, rarely experience widespread adoption due to the lack of a common descriptive schema that would enable porting these applications and systems to different buildings. Our demo presents Brick [4], a uniform schema for representing metadata in buildings. Our schema defines a concrete ontology for sensors, subsystems and relationships among them, which enables portable applications. Using a web application, we will demonstrate real buildings that have been mapped to the Brick schema, and show application queries that extracts relevant metadata from these buildings. The attendees would be able to create example buildings and write their own queries.

First claim

What is claimed is:

1. A building management system, comprising:
one or more memory devices configured to store instructions thereon that, when executed by one or more processors, cause the one or more processors to:
access a library of neural network models associated with a plurality of different business vertical categories;
retrieve a first neural network model from the library based on a first business vertical category associated with the building management system, wherein the first neural network model is trained on building operation data of a previous time period, wherein the building operation data of the previous time period is associated with the first business vertical category;
generate, using the first neural network model, a control decision for equipment operated by the building management system; and
control the equipment to affect a physical condition of a building in accordance with the control decision.

2. The building management system of claim 1 , wherein the first neural network model comprises a reinforcement learning model, and wherein the instructions cause the one or more processors to generate the control decision by determining a control policy comprising one or more state-spaces for the reinforcement learning model.

3. The building management system of claim 1 , wherein the plurality of different business vertical catagories comprise residential, data center, and office tower.

4. The building management system of claim 1 , wherein the instructions further cause the one or more processors to determine a second prediction model based on the first neural network model, and wherein the building management system is operated using the second prediction model.

5. The building management system of claim 4 , wherein the second prediction model comprises a second neural network model trained based on at least one output of the first neural network model.

6. The building management system of claim 4 , wherein the second prediction model comprises a second neural network model trained based on at least one state from a state-space of a control policy determined based on the first neural network model and an action of the control policy for the at least one state.

7. The building management system of claim 2 , wherein the control policy comprises one or more action-spaces, and wherein the one or more action-spaces comprises one or more of a potential zone-air temperature setpoints.

8. The building management system of claim 1 , wherein the building operation data is based on both of simulated data and real-time building operation data.

9. A method for a building management system, comprising:
providing a library of neural network models associated with a plurality of different business vertical categories;
selecting, by a processing circuit, a first neural network model from the library based on a first business vertical category associated with the building management system, wherein the first neural network model is trained based on building operation data of a previous time period, wherein the building operation data of the previous time period is associated with the first business vertical catagory; and
operating, by the processing circuit, the building management system using the first neural network model.

10. The method of claim 9 , further comprising providing a first control policy, wherein the first control policy comprises a baseline policy, wherein the first neural network model corresponds to a second control policy determined based on a reward performance, the method further comprising generating, by the processing circuit, a second prediction model based on the second control policy, and operating, by the processing circuit, the building management system using the second prediction model.

11. The method of claim 10 , wherein the second prediction model comprises a second neural network model trained based on at least one state of a state-space of the second control policy and an action of the second control policy based on the at least one state.

12. The method of claim 9 , wherein the building operation data includes one or more of current values, previous values, or forecasts for various points of a space of a building operatively connected to the processing circuit.

13. A building management system, comprising:
building equipment operatively connected to one or more processors; and
one or more memory devices configured to store instructions thereon that, when executed by the one or more processors, cause the one or more processors to:
access a library of neural network models associated with a plurality of different business vertical categories;
retrieve a first neural network model from the library based on a first business vertical category associated with the building management system, wherein the first neural network model is trained based on building operation data of a previous time period, wherein the building operation data is associated with the first business vertical category; and
control the building equipment using the first neural network model.

14. The building management system of claim 13 , wherein the first neural network model is configured to determine a first value for a setpoint for the building equipment based on a current state of the building equipment.

15. The building management system of claim 13 , wherein the first neural network model is a reinforcement learning model.

16. The building management system of claim 14 , wherein the library comprises a second neural network model corresponding to a second business vertical category, wherein the second neural network model is configured to determine a second value for the setpoint, wherein the second value is different than the first value.

Assignees

• Johnson Controls Tyco IP Holdings LLP
• Tyco Fire & Security Gmbh

Inventors

• Huber Michael M
• Asp Gerald A
• Mellenthin Daniel A
• Senzig Jeffrey J G

Classifications

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G06N3/08

  Learning methods · CPC title

• G05B2219/25011

  Domotique, I-O bus, home automation, building automation · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/092

  Reinforcement learning · CPC title