Secure enterprise access with voice assistant devices

Patent metadata

Field	Value
Publication number	US-11961523-B2
Application number	US-202017016367-A
Country	US
Kind code	B2
Filing date	Sep 9, 2020
Priority date	Jul 22, 2020
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

Systems and methods are provided for optimizing and securing an enterprise voice service accessed by an external voice assistant device. An enterprise voice assistant installed on a client device acts as an enterprise voice service for an external voice assistant device. The enterprise voice assistant receives a voice query from the external voice assistant device. The voice query is processed using a machine learning model to extract an intent and at least one slot. The extracted intent and at least one slot are used to determine whether a response to the voice query can be generated using local enterprise data that was previously received and stored by the client device from a management server. The response is generated based on the determination by using the local enterprise data or by sending the extracted intent and at least one slot to and receiving the response from the management server.

First claim

What is claimed is:

1. A system for securely accessing enterprise data using an external voice assistant device comprising:
a memory storage including a non-transitory, computer-readable medium comprising instructions; and
a processor that executes the instructions to carry out stages comprising:
receiving, at a client device connected locally with the external voice assistant over a local network, a voice query from the external voice assistant device over the local network, wherein the client device acts as a voice service for the external voice assistant device in connection with a wake word;
processing the voice query at the client device to extract an intent and at least one slot from the voice query;
determining, by a management agent executing on the client device, that local enterprise data on the client device is not fully responsive to the voice query, wherein the local enterprise data is previously received at the client device;
determining that an internet connection is unavailable;
based on the determination that the internet connection is unavailable, requesting additional enterprise data by:
transmitting the extracted intent and at least one slot to a management server over a cellular network; and
receiving the response to the voice query from the management server over the cellular network,
wherein requesting additional enterprise data is performed in an instance where a speaker of the voice query is authenticated, and
wherein the additional enterprise data includes encrypted content; and

generating the response to the voice query based on receiving the additional enterprise data from the management server, including causing the external voice assistant to read aloud the content.

2. The system of claim 1 , wherein the determination includes:
querying a local application on the client device based on the intent and slot; and
receiving a result from the local application that includes the local enterprise data, wherein the generated response includes at least a portion of the result.

3. The system of claim 1 , wherein generating the response to the voice query further comprises:
transmitting the extracted intent and at least one slot to a management server remote from the local network; and
receiving the response to the voice query from the management server.

4. The system of claim 1 , wherein generating the response to the voice query further comprises:
storing the extracted intent and at least one slot in the memory storage;
transmitting the extracted intent and at least one slot to a management server over the internet connection when the internet connection becomes available; and
receiving the response to the voice query from the management server over the internet connection.

5. The system of claim 1 , the stages further comprising forming a communication path between the client device and the external voice assistant device using a local discovery protocol.

6. The system of claim 1 , the stages further comprising, prior to generating the response, authenticating a speaker of the voice query by comparing the voice query with a voice sample stored in the memory storage.

7. A non-transitory, computer-readable medium comprising instructions that, when executed by a processor of a client device, perform stages for securely accessing enterprise data using an external voice assistant device, the stages comprising:
receiving, at a client device connected locally with the external voice assistant over a local network, a voice query from the external voice assistant device over the local network, wherein the client device acts as a voice service for the external voice assistant device in connection with a wake word;
processing the voice query at the client device to extract an intent and at least one slot from the voice query;
determining, by a management agent executing on the client device, that local enterprise data on the client device is not fully responsive to the voice query, wherein the local enterprise data is previously received at the client device;
determining that an internet connection is unavailable;
based on the determination that the internet connection is unavailable, requesting additional enterprise data by:
transmitting the extracted intent and at least one slot to a management server over a cellular network; and
receiving the response to the voice query from the management server over the cellular network,
wherein requesting additional enterprise data is performed in an instance where a speaker of the voice query is authenticated, and
wherein the additional enterprise data includes encrypted content; and

generating the response to the voice query based on receiving the additional enterprise data from the management server, including causing the external voice assistant device to read the content aloud.

8. The non-transitory, computer-readable medium of claim 7 , wherein the determination includes:
querying a local application on the client device based on the intent and slot; and
receiving a result from the local application that includes the local enterprise data, wherein the generated response includes at least a portion of the result.

9. The non-transitory, computer-readable medium of claim 7 , wherein generating the response to the voice query further comprises:
transmitting the extracted intent and at least one slot to a management server remote from the local network; and
receiving the response to the voice query from the management server.

10. The non-transitory, computer-readable medium of claim 7 , wherein generating the response to the voice query further comprises:
storing the extracted intent and at least one slot in the memory storage;
transmitting the extracted intent and at least one slot to a management server over the internet connection when the internet connection becomes available; and
receiving the response to the voice query from the management server over the internet connection.

11. The non-transitory, computer-readable medium of claim 7 , the stages further comprising forming a communication path between the client device and the external voice assistant device using a local discovery protocol.

12. The non-transitory, computer-readable medium of claim 7 , the stages further comprising, prior to generating the response, authenticating a speaker of the voice query by comparing the voice query with a voice sample stored in the memory storage.

13. A method for securely accessing enterprise data using an external voice assistant device comprising:
receiving, at a client device connected locally with the external voice assistant over a local network, a voice query from the external voice assistant device over the local network, wherein the client device acts as a voice service for the external voice assistant device in connection with a wake word;
processing the voice query at the client device to extract an intent and at least one slot from the voice query;
determining, by a management agent executing on the client device, that local enterprise data on the client device is not fully responsive to the voice query, wherein the local enterprise data is previously received at the client device;
determining that an internet connection is unavailable;
based on the determination that the internet connection is unavailable, requesting additional enterprise data by:
transmitting the extracted intent and at least one slot to a management server over a cellular network; and
receiving the response to the voice query from the management server over the cellular network,
wherein requesting additional enterprise data is performed in an instance where a speake

Assignees

• VMware LLC
• Vmware Inc

Inventors

• Aluvala Suman
• Panchapakesan Ramani
• Shetty Rohit Pradeep
• Kochhar Arjun

Classifications

• G10L17/04 Primary

  Training, enrolment or model building · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L17/22

  Interactive procedures; Man-machine interfaces · CPC title

• H04L67/125

  involving control of end-device applications over a network · CPC title

• G10L2015/221

  Announcement of recognition results · CPC title