Offline voice control

Patent metadata

Field	Value
Publication number	US-12374334-B2
Application number	US-202418404254-A
Country	US
Kind code	B2
Filing date	Jan 4, 2024
Priority date	Dec 20, 2019
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

As noted above, example techniques relate to offline voice control. A local voice input engine may process voice inputs locally when processing voice inputs via a cloud-based voice assistant service is not possible. Some techniques involve local (on-device) voice-assisted set-up of a cloud-based voice assistant service. Further example techniques involve local voice-assisted troubleshooting the cloud-based voice assistant service. Other techniques relate to interactions between local and cloud-based processing of voice inputs on a device that supports both local and cloud-based processing.

First claim

The invention claimed is:

1. A playback device comprising:
at least one audio transducer;
one or more microphones;
a network interface;
at least one processor;
a housing carrying the one or more microphones, the network interface, the at least one processor, and
at least one non-transitory computer-readable medium comprising program instructions that are executable by the at least one processor such that the playback device is configured to:
while the playback device is in an offline mode:
monitor, via a local voice assistant, a sound data stream from the one or more microphones for local keywords from a local natural language unit library of the local voice assistant, wherein in the offline mode, a voice assistant service (VAS) wake-word engine is inactive, and wherein while the playback device is in an online mode, the VAS wake-word engine is active;
generate a first local wake-word event corresponding to a first voice input when the local voice assistant detects sound data matching one or more first local keywords in a first portion of the sound data stream, wherein the one or more first local keywords comprise a local wake word;
determine, via the local voice assistant, an intent of the first voice input, wherein the determined intent represents to a command to setup smart devices;
according to the command to setup smart devices, setup the local voice assistant with a particular smart device connected to a local area network, wherein the playback device is disconnected from the Internet while in the offline mode;
generate a second local wake-word event corresponding to a second voice input when the local voice assistant detects sound data matching one or more second local keywords in a second portion of the sound data stream, wherein the one or more second local keywords comprise the local wake word;
determine, via the local voice assistant, an intent of the second voice input, wherein the determined intent of the second voice input represents a particular command for the particular smart device; and
send, via the network interface over the local area network to the particular smart device, data representing the particular command.

2. The playback device of claim 1 , wherein the program instructions that are executable by the at least one processor such that the playback device is configured to setup the local voice assistant with the particular smart device connected to the local area network comprise program instructions that are executable by the at least one processor such that the playback device is configured to:
discover, via the network interface, the particular smart device.

3. The playback device of claim 2 , wherein the program instructions that are executable by the at least one processor such that the playback device is configured to discover the particular smart device comprise program instructions that are executable by the at least one processor such that the playback device is configured to:
transmit, via the network interface over the local area network, one or more discovery requests; and
receive, via the network interface over the local area network from the particular smart device, (i) a response to at least one of the one or more discovery requests and (ii) data identifying the particular smart device.

4. The playback device of claim 1 , wherein the program instructions that are executable by the at least one processor such that the playback device is configured to setup the local voice assistant with the particular smart device connected to the local area network comprise program instructions that are executable by the at least one processor such that the playback device is configured to:
add one or more keywords corresponding to the particular smart device to the local natural language unit library of the local voice assistant.

5. The playback device of claim 4 , wherein the program instructions that are executable by the at least one processor such that the playback device is configured to add the one or more keywords corresponding to the particular smart device to the local natural language unit library of the local voice assistant comprise program instructions that are executable by the at least one processor such that the playback device is configured to:
add at least one keyword corresponding to respective identifiers of the particular smart device to the local natural language unit library of the local voice assistant.

6. The playback device of claim 4 , wherein the at least one non-transitory computer-readable medium further comprises program instructions that are executable by the at least one processor such that the playback device is configured to:
add a command keyword corresponding to a function of the particular smart device to the local natural language unit library of the local voice assistant.

7. The playback device of claim 6 , wherein the particular smart device is an additional playback device, and wherein the program instructions that are executable by the at least one processor such that the playback device is configured to add the command keyword corresponding to the function of the particular smart device to the local natural language unit library of the local voice assistant comprise program instructions that are executable by the at least one processor such that the playback device is configured to:
add at least one command keyword corresponding to a grouping command to the local natural language unit library of the local voice assistant, of the playback device, wherein the grouping command causes formation of synchrony groups among playback devices targeted by the grouping command.

8. The playback device of claim 6 , wherein the particular smart device is a smart thermostat, and wherein the program instructions that are executable by the at least one processor such that the playback device is configured to add the command keyword corresponding to the function of the particular smart device to the local natural language unit library of the local voice assistant comprise program instructions that are executable by the at least one processor such that the playback device is configured to:
add at least one command keyword corresponding to a temperature control command to the local natural language unit library of the local voice assistant, of the playback device, wherein the temperature control command causes adjustments of a temperature setting at the smart thermostat.

9. The playback device of claim 1 , wherein the at least one non-transitory computer-readable medium further comprises program instructions that are executable by the at least one processor such that the playback device is configured to:
while the playback device is in the online mode:
monitor, via the VAS wake-word engine, the sound data stream from the one or more microphones for one or more VAS wake words of a cloud-based voice assistant service; and
generate a VAS wake-word event corresponding to a third voice input when the VAS wake-word engine detects sound data matching a particular VAS wake word in a third portion of the sound data stream, wherein, when the VAS wake word event is generated, the playback device streams, via the network interface, sound data representing the third voice input to one or more servers of the cloud-based voice assistant service.

10. The playback device of claim 1 , wherein the at least one non-transitory computer-readable medium further comprises program instructions that are executable by the at least one processor such that the playback device is configured to:
before generation of the first local wake-word event, output, via the at least one audio transducer, an audible prompt to setup smart devices, and wherein the command to setup sm

Assignees

• Sonos Inc

Inventors

• Smith Connor

Classifications

• H04L43/0811

  by checking connectivity · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L2015/088

  Word spotting · CPC title

• G10L15/08

  Speech classification or search · CPC title

• G10L15/07

  to the speaker · CPC title