Intelligent automated assistant in a media environment

Patent metadata

Field	Value
Publication number	US-10956006-B2
Application number	US-201916394965-A
Country	US
Kind code	B2
Filing date	Apr 25, 2019
Priority date	Sep 8, 2015
Publication date	Mar 23, 2021
Grant date	Mar 23, 2021

Abstract

Systems and processes are disclosed for operating a digital assistant in a media environment. In an exemplary embodiment, a user can interact with a digital assistant of a media device while content is displayed by the media device. In one approach, a plurality of exemplary natural language requests can be displayed in response to detecting a user input of a first input type. The plurality of exemplary natural language requests can be contextually-related to the displayed content. In another approach, a user request can be received in response to detecting a user input of a second input type. A task that at least partially satisfies the user request can be performed. The performed task can depend on the nature of the user request and the content being displayed by the media device. In particular, the user request can be satisfied while reducing disruption to user consumption of media content.

First claim

The invention claimed is:

1. A plurality of non-transitory computer-readable storage media including a first non-transitory computer-readable storage medium storing one or more first programs and a second non-transitory computer-readable storage medium storing one of more second programs, the one or more first programs comprising first instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to:
display content on a display unit;
while displaying the content, detect a user input comprising a predetermined motion pattern on a second electronic device; and
in response to detecting the user input comprising a predetermined motion pattern on a second electronic device:
display a virtual keyboard interface on the display unit;
wherein the one or more second programs comprise second instructions, which when executed by one or more processors of the second electronic device, cause the second electronic device to:
in response to detecting the user input comprising the predetermined motion pattern on the second electronic device:
display a selectable affordance on a display of the second electronic device, wherein selection of the selectable affordance enables text input to be received by the first electronic device via a keyboard of the second electronic device.

2. The plurality of computer-readable storage medium of claim 1 , wherein the first instructions further cause the first electronic device to:
receive text input via the keyboard of the second electronic device, the text input representing a user request;
obtain results that at least partially satisfy the user request; and
display a user interface on the display unit, the user interface including at least a portion of the results.

3. The plurality of computer-readable storage medium of claim 1 , wherein the displayed content comprises a second user interface with a plurality of exemplary natural language requests.

4. The plurality of computer-readable storage medium of claim 3 , wherein the displayed content includes media content.

5. The plurality of computer-readable storage medium of claim 1 , wherein the displayed content comprises a third user interface with results from a previous user request, the third user interface occupying at least a majority of a display area of the display unit.

6. The plurality of computer-readable storage medium of claim 5 , wherein the virtual keyboard interface is overlaid on at least a portion of the third user interface.

7. The plurality of computer-readable storage medium of claim 1 , wherein the user input is detected via a remote control of the electronic device, and wherein the remote control and the second electronic device are different devices.

8. The plurality of computer-readable storage medium of claim 7 , wherein the user input comprises a predetermined motion pattern on a touch-sensitive surface of the remote control device.

9. The plurality of computer-readable storage medium of claim 1 , wherein the user input is detected via the second electronic device.

10. The plurality of computer-readable storage medium of claim 1 , wherein the first instructions further cause the first electronic device to:
in response to detecting the user input, display a search field on the display unit;
receive text input from the second electronic device; and
display the text input in the displayed search field.

11. A system, comprising:
a first electronic device, comprising:
one or more first processors; a first memory; and one or more first programs, wherein the one or more first programs are stored in the first memory and configured to be executed by the one or more first processors, the one or more first programs including first instructions for:
displaying content on a display unit;
while displaying the content, detecting a user input comprising a predetermined motion pattern on a second electronic device; and
in response to detecting the user input comprising a predetermined motion pattern on a second electronic device:
displaying a virtual keyboard interface on the display unit; and

the second electronic device, comprising:
one or more second processors; a second memory; and one or more second programs, wherein the one or more second programs are stored in the second memory and configured to be executed by the one or more second processors, the one or more second programs including second instructions for:
in response to detecting the user input comprising the predetermined motion pattern on the second electronic device:
displaying a selectable affordance on a display of the second electronic device, wherein selection of the affordance enables text input to be received by the first electronic device via a keyboard of the second electronic device.

12. The system of claim 11 , wherein the first instructions further cause the first electronic device to:
receive text input via the keyboard of the second electronic device, the text input representing a user request;
obtain results that at least partially satisfy the user request; and
display a user interface on the display unit, the user interface including at least a portion of the results.

13. The system of claim 11 , wherein the displayed content comprises a second user interface with a plurality of exemplary natural language requests.

14. The system of claim 13 , wherein the displayed content includes media content.

15. The system of claim 11 , wherein the displayed content comprises a third user interface with results from a previous user request, the third user interface occupying at least a majority of a display area of the display unit.

16. The system of claim 15 , wherein the virtual keyboard interface is overlaid on at least a portion of the third user interface.

17. The system of claim 11 , wherein the user input is detected via a remote control of the electronic device, and wherein the remote control and the second electronic device are different devices.

18. The system of claim 17 , wherein the user input comprises a predetermined motion pattern on a touch-sensitive surface of the remote control device.

19. The system of claim 11 , wherein the user input is detected via the second electronic device.

20. The system of claim 11 , wherein the first instructions further cause the first electronic device to:
in response to detecting the user input, display a search field on the display unit;
receive text input from the second electronic device; and
display the text input in the displayed search field.

21. A method, comprising:
at a first electronic device with one or more first processors and a first memory:
displaying content on a display unit;
while displaying the content, detecting a user input comprising a predetermined motion pattern on a second electronic device; and
in response to detecting the user input comprising a predetermined motion pattern on a second electronic device:
displaying a virtual keyboard interface on the display unit; and

at a second electronic device with one or more second processors and a second memory:
in response to detecting the user input comprising the predetermined motion pattern on the second electronic device:
displaying a selectable affordance on a display of the second electronic device, wherein selection of the affordance enables text input to be received by the first electronic device via a keyboard of the second electronic device.

22. The method of claim 21 , further com

Assignees

• Apple Inc

Inventors

• Napolitano Lia T
• Hwang Grace H
• Penha Henrique D
• Shaw Jeremiah D
• Fino Jorge S

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/04842 Primary

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title