Systems and methods for conversational interaction with a building automation system

Patent metadata

Field	Value
Publication number	US-11264020-B2
Application number	US-201816028126-A
Country	US
Kind code	B2
Filing date	Jul 5, 2018
Priority date	Jul 5, 2017
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

A system for user interaction with a building automation system (BAS). The system includes a user device configured to accept vocal inputs and produce audio outputs, and a server in communication with the user device and a BAS controller. The server is configured to translate a vocal input to input text, and determine and store a conversation context based on the input text. The server is further configured to determine a desired output corresponding to the input text. Additionally, the server is configured to receive, from the BAS controller, BAS data corresponding to the desired output, and generate at least one audio output from the BAS data. The user device outputs the at least one audio output, and the server is configured to apply the conversation context to a subsequent vocal input.

First claim

What is claimed is:

1. A system for user interaction with a building automation system (BAS), the system comprising:
a user device configured to accept vocal inputs and produce audio outputs; and
a server in communication with the user device and a BAS controller, the server configured to:
query the BAS controller using one or more application programming interfaces (API) of the BAS controller with an element associated with input text based on vocal input, the query configured to cause the BAS controller to return an operational parameter stored by the BAS controller;
receive, from the BAS controller, BAS data including the operational parameter associated with a BAS device or a user corresponding to desired output, wherein the BAS data corresponds to a detected equipment fault;
generate at least one audio output related to the operational parameter from the BAS data, wherein the at least one audio output alerts a user of the detected equipment fault; and
submit a maintenance request based on the detected equipment fault.

2. The system of claim 1 , further comprising updating a BAS parameter based on a desired output associated with the input text.

3. The system of claim 2 , wherein the vocal input corresponds to a request to add a new user, and updating the BAS parameter comprises adding the new user to a BAS database and providing corresponding access rights.

4. The system of claim 1 , wherein the maintenance request comprises at least an equipment name, a location, and fault data.

5. The system of claim 1 , wherein the maintenance request comprises an order and payment for a replacement part.

6. The system of claim 1 , wherein the BAS data corresponds to a detected equipment fault, the server further configured to:
determine a plurality of potential fault causes;
calculate a probability for each potential fault cause; and
determine the potential fault with a highest probability;
wherein the at least one audio output corresponds to the potential fault with the highest probability.

7. A method of user interaction with a building automation system (BAS), the method comprising:
query a BAS controller using one or more application programming interfaces (API) of the BAS controller with an element associated with first user input, the query configured to cause the BAS controller to return an operational parameter stored by the BAS controller;
receiving BAS data from the BAS controller, the BAS data including the operational parameter associated with a BAS device or a user corresponding to the first user input;
providing, to a user, a first output corresponding to the operational parameter from the BAS data by displaying the first output via a user interface;
displaying a list of selectable options corresponding to the first output
determining a second user input by detecting a user selection corresponding to the second user input;
reprioritizing the list of selectable options for subsequent inputs based on the user selection;
forming a second output based on the second user input and a conversation context of the first user input; and
providing, to the user, the second output by displaying the second output via the user interface.

8. The method of claim 7 , further comprising:
determining a user ID; and
customizing each of the first output and the second output, based on the user ID.

9. The method of claim 8 , further comprising:
determining access rights corresponding to the user ID; and
updating a BAS parameter based on at least one of the first user input and the second user input,
wherein the access rights comprise updating the BAS parameter.

10. The method of claim 7 , further comprising:
accessing a BAS database to determine the BAS data;
comparing the second output to the first output to determine a preferred output corresponding to the first user input; and
updating the BAS database with the preferred output.

11. The method of claim 7 , wherein the conversation context comprises intent and entity parameters corresponding to the first user input.

12. The method of claim 7 , further comprising resetting the conversation context when a new user provides an input.

13. The method of claim 7 , further comprising resetting the conversation context after a threshold amount of time without subsequent user inputs.

14. A user device for communicating with a building automation system (BAS), the user device comprising:
a proximity sensor configured to determine a user location with respect to the user device;
a microphone configured to accept vocal inputs from a user;
a speaker configured to produce audio outputs to the user;
a processor in communication with the proximity sensor, the microphone, and the speaker, and configured to:
receive user location data from the proximity sensor;
receive a vocal input from the microphone;
store, in a memory, context parameters corresponding to the vocal input and the user location data;
determine a desired output corresponding to the vocal input;
query the BAS using one or more application programming interfaces (API) of the BAS with an element associated with the vocal input, the query configured to cause the BAS to return an operational parameter stored by the BAS;
receive BAS data including the operational parameter associated with a BAS device or a user, wherein the BAS data corresponds to a detected equipment fault;
update an operating parameter within the BAS based on the desired output; and
provide an audio output to the speaker, the audio output related to the operational parameter associated with the detected equipment fault from the BAS data;
submit a maintenance request based on the detected equipment fault;

wherein the processor analyzes a subsequent vocal input using the context parameters, prior to providing a subsequent audio output.

15. The user device of claim 14 , wherein the operating parameter corresponds to BAS equipment associated with the user location.

16. The user device of claim 14 , wherein the processor is further configured to:
determine a user ID from the vocal input;
determine access rights corresponding to the user ID; and
verify that the access rights comprise updating the operating parameter.

17. The user device of claim 14 , wherein the vocal input corresponds to a request to add a new user, and updating the operating parameter comprises adding the new user to a BAS database and providing corresponding access rights.

Assignees

• Johnson Controls Tyco IP Holdings LLP

Inventors

• Pourmohammad Sajjad
• Mueller Joseph M
• Park Youngchoon
• Sinha Sudhi R
• Mellenthin Daniel
• Asp Gerald A

Classifications

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G06F2221/2141

  Access rights, e.g. capability lists, access control lists, access tables, access matrices · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title