Multi-tasking and skills processing

Patent metadata

Field	Value
Publication number	US-11295745-B1
Application number	US-201916560103-A
Country	US
Kind code	B1
Filing date	Sep 4, 2019
Priority date	Sep 4, 2019
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

Described herein is a system for enabling a user to multitask by allowing a user to pause or interrupt an on-going interaction with a skill. The system monitors a state of a skill session, and updates the state to allow the user or system to suspend the session. The user may provide an instruction to pause an active session, causing the system to place the session in a suspended state. The user may then provide an instruction to resume the suspended session, causing the system to place the session in an active state. In other cases, the user input may be a request during an active session that requires invoking another skill. The system may place the current session in a suspended state, and invoke a second skill session to monitor the interaction with a second skill. When the interaction with the second skill is completed, the system may resume the previous session by placing it in an active state.

First claim

What is claimed is:

1. A computer implemented method comprising:
receiving first audio data representing a first utterance, the first utterance associated with a user profile;
performing speech processing using the first audio data to determine a first intent represented by the first utterance;
determining a first skill to execute the first intent;
generating a first data record for monitoring a beginning and an end of a first skill session corresponding to the first utterance, the first data record including a first skill session identifier and a first skill identifier representing the first skill;
sending the first skill session identifier and an indicator of the first intent to a first component associated with the first skill;
associating first status data with the first data record, the first status data indicating the first skill session is active with respect to the user profile;
receiving second audio data representing a second utterance, the second utterance associated with the user profile;
performing speech processing using the second audio data to determine a second intent represented by the second utterance;
determining a second skill to execute the second intent;
generating a second data record for monitoring a beginning and an end of a second skill session corresponding to the second utterance, the second data record including a second skill session identifier and a second skill identifier representing the second skill;
sending the second skill session identifier and a second indicator of the second intent to a second component associated with the second skill;
associating second status data with the first data record, the second status data indicating that the first skill session is suspended with respect to the user profile;
associating third status data with the second data record, the third status data indicating the second skill session is active with respect to the user profile;
receiving, from the second component, a first message indicating completion of execution with respect to the second intent;
sending, to the first component, a second message indicating resumption of the first skill session, the second message including the first skill session identifier; and
associating fourth status data with the first data record, the fourth status data indicating that the first skill session is active with respect to the user profile.

2. The computer implemented method of claim 1 , further comprising:
determining that the second intent results in interrupting the first skill session;
determining, using stored policy data, that the first skill session is capable of being interrupted, the stored policy data indicating at least one of: when the first skill session is capable of being interrupted, an engagement level corresponding to the first skill session, or a type resource used by the first skill session; and
sending a third message to the first component indicating that the first skill session is being suspended with respect to the user profile, the third message including the first skill session identifier.

3. The computer implemented method of claim 1 , further comprising:
sending a third message to the first component indicating that the first skill session is being suspended with respect to the user profile, the third message including the first skill session identifier;
receiving, from the first component, resumption data indicating a duration of time within which the first skill session is capable of being resumed;
storing a first time representing when the second status data is associated with the first data record;
storing a second time representing when the first message is received;
determining time lapsed based on the first time and the second time; and
determining that the first skill session is capable of being resumed based at least in part on the duration of time and the time lapsed,
wherein sending the second message comprises sending the second message in response to determining that the first skill session is capable of being resumed.

4. The computer implemented method of claim 1 , further comprising:
storing association data indicating that the first skill session was interrupted by the second skill session; and
associating fifth status data with the second data record, the fifth status data indicating that the second skill session is terminated, the associating in response to receiving the first message;
wherein sending the second message comprises:
determining, using the association data, that the first skill session was interrupted by the second skill session;
determining, using the fifth status data, that the second skill session is terminated; and
generating the second message in response to determining that the second skill session is terminated.

5. A computer-implemented method comprising:
identifying a first data record corresponding to a first skill session, the first data record including a first skill session identifier and a first skill identifier associated with a first skill;
determining that first status data associated with the first data record indicates that the first skill session is suspended;
receiving an indication to resume the first skill session;
sending, to a first component associated with the first skill, a first message indicating resumption of the first skill session, the first message including the first skill session identifier; and
associating second status data with the first data record, the second status data indicating that the first skill session is active.

6. The method of claim 5 , wherein receiving the indication comprises:
receiving audio data representing an utterance associated with a user profile;
processing the audio data using automatic speech recognition (ASR) to determine text data; and
performing natural language understanding (NLU) on the text data to determine an intent to resume the first skill session with respect to the user profile,
wherein receiving the indication to resume the first skill session comprises receiving the intent.

7. The method of claim 5 , wherein receiving the indication comprises:
identifying a second data record corresponding to a second skill session;
determining that third status data associated with the second data record indicates that the second skill session is terminated; and
determining that the first skill session was interrupted by the second skill session,
wherein associating the second status data with the first data record occurs in response to determining that the third status data indicates the second skill session is terminated and determining that the first skill session was interrupted by the second skill session.

8. The method of claim 5 , further comprising:
receiving, from the first component, a time period within which the first skill session is capable of being resumed;
determining a duration of time between when the first status data is set to suspended and when the indication to resume the first skill session is received; and
determining that, based at least in part on the duration of time, the first skill session is capable of being resumed.

9. The method of claim 5 , further comprising:
receiving input data associated with a user profile;
determining that the input data corresponds to the first skill session;
determining that the second status data indicates that the first skill session is active; and
sending the input data to the first component based at least in part on determining that the first skill session is active.

10. The method of claim 5 , further comprising prior to identifying the first data record:
receiving input data associated with a user profile;
determining that the input data repr

Assignees

• Amazon Tech Inc

Inventors

• Roy Shiladitya
• Li Bo
• Gundeti Vikram Kumar
• Bhupathiraju Eswara Jnana Swaroop
• Leupold Nathan G
• Huntwork Andrew S

Classifications

• G10L2015/228

  of application context · CPC title

• G10L15/222 Primary

  Barge in, i.e. overridable guidance for interrupting prompts · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/1822

  Parsing for meaning understanding · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title