Streaming tasks in a multiple device environment

Patent metadata

Field	Value
Publication number	US-12562172-B2
Application number	US-202318379094-A
Country	US
Kind code	B2
Filing date	Oct 11, 2023
Priority date	May 8, 2023
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Systems and processes for operating an intelligent automated assistant are provided. An exemplary process includes, at a first user device with one or more processors and memory: receiving an audio input including a task and an application for performing the task; in response to a determination that the application is available to the first user device, performing the task with the application at the first user device; in response to a determination that the application is not available to the first user device: causing performance of the task with the application at a second user device; receiving the result of the task from the second user device; and providing the result of the task as an audio output.

First claim

What is claimed is:

1 . A first user device, comprising:
one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving an audio input including a task and an application for performing the task;
in response to a determination that the application is available to the first user device, performing the task with the application at the first user device;
in response to a determination that the application is not available to the first user device:
causing performance of the task with the application at a second user device;
receiving a result of the task from the second user device; and
providing the result of the task as an audio output.

2 . The first user device of claim 1 , wherein determining that the application is not available to the first user device includes determining whether the application is capable of being installed and/or enrolled on the first user device.

3 . The first user device of claim 2 , the one or more programs further including instructions for:
in response to a determination that the application is capable of being installed and/or enrolled on the first user device, providing an output including instructions to install and/or enroll the application on the first user device.

4 . The first user device of claim 2 , the one or more programs further including instructions for:
in response to a determination that the application is not capable of being installed and/or enrolled on the first user device, causing performance of the task with the application at the second user device.

5 . The first user device of claim 1 , the one or more programs further including instructions for:
in accordance with a determination that the application is not installed on the second user device, providing an audio output indicating that the application needs to be installed on the second user device; and forgoing performance of the task with the application at the second user device.

6 . The first user device of claim 1 , wherein the audio input includes an indication of a third user device, the one or more programs further including instructions for:
in accordance with a determination that the application is available to the third user device, causing performance of the task with the application at the third user device; and in accordance with a determination that the application is not available to the third user device:
causing performance of the task with the application at the second user device;
causing transmission of data representing the result of the task to the third user device; and
providing the result of the task as a second audio output at the third user device.

7 . The first user device of claim 1 , wherein causing performance of the task at the second user device further comprises:
providing a command representing the task to the second user device; and causing the second user device to execute the command.

8 . The first user device of claim 1 , wherein causing performance of the task at the second user device further comprises:
providing the audio input to the second user device; causing the second user device to determine the task from the audio input; and causing the second user device to execute a command corresponding to the determined task.

9 . The first user device of claim 1 , the one or more programs further including instructions for:
in accordance with a determination that a user providing the audio input is an authorized user of the first user device, performing the task with the application at the first user device; and in accordance with a determination that the user providing the audio input is not an authorized user of the first user device, providing an audio output indicating that the task cannot be performed.

10 . The first user device of claim 9 , wherein the determination that the user providing the audio input is an authorized user device is based on a speaker identification of the user.

11 . The first user device of claim 9 , wherein the determination that the user providing the audio input is an authorized user device is based on an enrollment process of the first user device.

12 . The first user device of claim 1 , the one or more programs further including instructions for:
in accordance with a determination that a user providing the audio input is not an authorized user of the first user device, performing the task with the application at the first user device.

13 . The first user device of claim 1 , wherein the audio output is a first audio output, the one or more programs further including instructions for:
while providing the first audio output:
detecting display of a second application at the second user device;
in accordance with a determination that the second application provides long form audio:
forgoing providing the first audio output at the first user device; and
providing a second audio output at the second user device; and

in accordance with a determination that the second application provides short form audio:
providing the second audio output at the second user device; and
continuing to provide the first audio output at the first user device.

14 . The first user device of claim 1 , wherein the audio input is a first audio input and the task is a first task, the one or more programs further including instructions for:
while providing the audio output:
receiving a second audio input including a second task;
in accordance with a determination that the second task is a request to provide an audio output other than the audio output, causing playback of the audio output other than the audio output based on whether the user is an authorized user; and
in accordance with a determination that the second task is a modification of the audio output, performing the modification without determining whether the user is an authorized user.

15 . The first user device of claim 1 , the one or more programs further including instructions for:
displaying a user interface associated with the application on a display of the second user device.

16 . The first user device of claim 15 , wherein the user interface includes text indicating a device providing the audio output.

17 . The first user device of claim 1 , wherein the audio output is a first audio output, the one or more programs further including instructions for:
prior to providing the first audio output:
providing a third audio output indicating that the first audio output will stream from the second user device to the first user device.

18 . The first user device of claim 17 , the one or more programs further including instructions for:
in accordance with a determination that audio input including the task has been received at least a predetermined number of times, forgoing providing the third audio output.

19 . The first user device of claim 1 , the one or more programs further including instructions for:
detecting that the second user device is greater than a predetermined distance away from the first user device; and in response to detecting that the second user device is greater than the predetermined distance away from the first user device, ceasing the audio output.

20 . The first user device of claim 1 , the one or more programs further including instructions for:
in accordance with a determination that the application

Assignees

• Apple Inc

Inventors

• Sinesio Andrew J
• Abramets Alexandr N
• Hong Ivana M
• Klems Ryan R
• Loo Catherine M
• Mandel Daniel J
• Orr Ryan M
• Saleh Zeyad H

Classifications

• G10L17/04

  Training, enrolment or model building · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/165

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title

• G10L17/00

  Speaker identification or verification techniques · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title