In-call virtual assistants

Patent metadata

Field	Value
Publication number	US-10134395-B2
Application number	US-201314037077-A
Country	US
Kind code	B2
Filing date	Sep 25, 2013
Priority date	Sep 25, 2013
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

Techniques for providing virtual assistants to assist users during a voice communication between the users. For instance, a first user operating a device may establish a voice communication with respective devices of one or more additional users, such as with a device of a second user. For instance, the first user may utilize her device to place a telephone call to the device of the second user. A virtual assistant may also join the call and, upon invocation by a user on the call, may identify voice commands from the call and may perform corresponding tasks for the users in response.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
receiving an indication that a first device of a first user and a second device of a second user are exchanging first voice-communication data;
establishing a connection between the first device, the second device, and a computing device hosting at least a portion of a virtual assistant based at least in part on the indication;
receiving, by the computing device, the first voice-communication data exchanged between the first device and the second device, the computing device being remote from both the first device and the second device, wherein the computing device is configured to communicate with the first device over one or more networks and configured to communicate with the second device over the one or more networks;
performing speech recognition on a first part of a first audio signal to generate first text, the first audio signal representing first audio of the first voice-communication data exchanged between the first device and the second device;
identifying a predefined utterance in the first text;
invoking the virtual assistant based at least in part on identifying the predefined utterance;
receiving identity information associated with the first device;
transmitting, to the first device and at least partially in response to invoking the virtual assistant, information indicating that second voice-communication data will not be transmitted to the second device;
transmitting, to the first device, a request for a password, the request based at least in part on the identity information;
receiving the second voice-communication data from the first device including a representation of the password;
preventing, based at least in part on the request for the password, the second voice-communication data from being transmitted to the second device;
determining that the representation of the password is associated with an identity of the first user;
receiving third voice-communication data exchanged between the first device and the second device, the third voice-communication data received subsequent to the first voice-communication data and the second voice-communication data;
at least partly in response to identifying the predefined utterance, performing speech recognition on a second audio signal to generate second text, the second audio signal representing second audio of the third voice-communication data exchanged between the first device and the second device;
identifying a voice command in the second text, the voice command being separate from and occurring after the predefined utterance;
performing a task corresponding to the voice command at least partly in response to identifying the voice command; and
sending an output audio signal to at least one of the first device or the second device, the output audio signal configured to cause audible output associated with the performing of the task on at least one of the first device or the second device.

2. A system as recited in claim 1 , wherein:
the voice command comprises a request for information;
the performing of the task comprises locating, as located information, the information; and
the sending of the output audio signal comprises sending a signal configured to cause the virtual assistant to state, to the first user and the second user, the located information.

3. A system as recited in claim 1 , wherein the first device, the second device, and the computing device are connected via a conference call.

4. A system as recited in claim 1 , wherein the computing device is associated with a telephony service that establishes voice communication between the first user and the second user.

5. A system as recited in claim 1 , the second audio associated with the third voice-communication data including a first part and a second part, the second text corresponding to the first part of the second audio signal, the acts further comprising, after sending the output audio signal to the at least one of the first device or the second device, refraining from performing speech recognition on the second part of the second audio signal, the second part being subsequent to the first part.

6. A system as recited in claim 1 , wherein the information indicating that second voice-communication data will not be transmitted to the second device includes at least data generated using text to speech.

7. A system as recited in claim 1 , wherein the identity information associated with the first device is based at least in part on a telephone number associated with the first device, a uniform resource identifier associated with the first device, a voice over internet protocol (VoIP) identifier associated with the first device, or a session initiation protocol (SIP) identifier associated with the first device.

8. A method comprising:
at a computing device hosting at least a portion of a virtual assistant:
receiving first voice-communication data exchanged between a first device of a first user and a second device of a second user, the computing device being remote from both the first device and the second device;
performing speech recognition on a first audio signal to generate first text, the first audio signal representing first audio of the first voice-communication data exchanged between the first device and the second device;
identifying a predefined utterance in the first text;
invoking the virtual assistant based at least in part on identifying the predefined utterance;
receiving identity information associated with the first device;
transmitting, to the first device and at least partially in response to invoking the virtual assistant, information indicating that second voice-communication data will not be transmitted to the second device;
transmitting a request for a password to the first device, the request based at least in part on the identity information;
receiving the second voice-communication data from the first device including a representation of the password;
preventing, based at least in part on the request for the password, the second voice-communication data from being transmitted to the second device;
determining that the representation of the password is associated with an identity of the first user;
receiving third voice-communication data exchanged between the first device and the second device, the third voice-communication data received subsequent to the first voice-communication data and the second voice-communication data;
at least partly in response to identifying the predefined utterance in the first text, performing speech recognition on a second audio signal to generate second text, the second audio signal representing second audio of the third voice-communication data exchanged between the first device and the second device;
identifying, from the second text, a voice command uttered by at least one of the first user or the second user, the voice command being separate from and occurring after the predefined utterance;
performing a task corresponding to the voice command at least partly in response to identifying the voice command; and
sending, over one or more networks, an output audio signal to the first device and the second device, wherein the output audio signal is associated with the performing the task corresponding to the voice command.

9. A method as recited in claim 8 , further comprising identifying a user that provided the voice command, and wherein audible content associated with the output audio signal is based at least in part on the identifying the user.

Assignees

• Amazon Tech Inc

Inventors

• Typrin Marcello

Classifications

• H04M2203/355

  Interactive dialogue design tools, features or methods · CPC title

• H04M3/493

  Interactive information services, e.g. directory enquiries {; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals} · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• H04M2203/357

  Autocues for dialog assistance · CPC title