Intelligent automated assistant for TV user interactions

Patent metadata

Field	Value
Publication number	US-9338493-B2
Application number	US-201414498503-A
Country	US
Kind code	B2
Filing date	Sep 26, 2014
Priority date	Jun 30, 2014
Publication date	May 10, 2016
Grant date	May 10, 2016

Abstract

Systems and processes are disclosed for controlling television user interactions using a virtual assistant. A virtual assistant can interact with a television set-top box to control content shown on a television. Speech input for the virtual assistant can be received from a device with a microphone. User intent can be determined from the speech input, and the virtual assistant can execute tasks according to the user's intent, including causing playback of media on the television. Virtual assistant interactions can be shown on the television in interfaces that expand or contract to occupy a minimal amount of space while conveying desired information. Multiple devices associated with multiple displays can be used to determine user intent from speech input as well as to convey information to users. In some examples, virtual assistant query suggestions can be provided to the user based on media content shown on a display.

First claim

What is claimed is:

1. A method for controlling television interactions using a virtual assistant, the method comprising:
at an electronic device:
receiving speech input from a user;
determining media content based on the speech input;
displaying a first user interface having a first size, wherein the first user interface comprises one or more selectable links to the media content;
receiving a selection of one of the one or more selectable links; and
in response to the selection, displaying a second user interface having a second size larger than the first size, wherein the second user interface comprises the media content associated with the selection.

2. The method of claim 1 , wherein the first user interface expands into the second user interface in response to the selection.

3. The method of claim 1 , wherein the first user interface is overlaid on playing media content.

4. The method of claim 1 , wherein the second user interface is overlaid on playing media content.

5. The method of claim 1 , wherein the speech input comprises a query, and the media content comprises a result of the query.

6. The method of claim 5 , wherein the first user interface comprises a link to results of the query beyond the one or more selectable links to the media content.

7. The method of claim 1 , further comprising:
in response to the selection, playing the media content associated with the selection.

8. The method of claim 1 , wherein the media content comprises a sporting event.

9. The method of claim 1 , wherein the second user interface comprises a description of the media content associated with the selection.

10. The method of claim 1 , wherein the first user interface comprises a link to purchase media content.

11. The method of claim 1 , further comprising:
receiving additional speech input from the user, wherein the additional speech input comprises a query associated with displayed content;
determining a response to the query associated with the displayed content based on metadata associated with the displayed content; and
in response to receiving the additional speech input, displaying a third user interface, wherein the third user interface comprises the determined response to the query associated with the displayed content.

12. The method of claim 1 , further comprising:
receiving an indication to initiate receipt of speech input; and
in response to receiving the indication, displaying a readiness confirmation.

13. The method of claim 1 , further comprising:
in response to receiving the speech input, displaying a listening confirmation.

14. The method of claim 1 , further comprising:
displaying a transcription of the speech input.

15. The method of claim 1 , wherein the electronic device comprises a television.

16. The method of claim 1 , wherein the electronic device comprises a television set-top box.

17. The method of claim 1 , wherein the electronic device comprises a remote control.

18. The method of claim 1 , wherein the electronic device comprises a mobile telephone.

19. The method of claim 1 , wherein the one or more selectable links in the first user interface comprise moving images associated with the media content.

20. The method of claim 19 , wherein the moving images associated with the media content comprise live feeds of the media content.

21. The method of claim 1 , further comprising:
determining whether currently displayed content comprises a moving image or a control menu;
in response to a determination that currently displayed content comprises a moving image, selecting a small size as the first size for the first user interface; and
in response to a determination that currently displayed content comprises a control menu, selecting a large size, larger than the small size, as the first size for the first user interface.

22. The method of claim 1 , further comprising:
determining alternative media content for display based on one or more of a user preference, a show popularity, and a status of a live sporting event; and
displaying a notification comprising the determined alternative media content.

23. A non-transitory computer-readable storage medium comprising computer-executable instructions for:
receiving speech input from a user;
determining media content based on the speech input;
displaying a first user interface having a first size, wherein the first user interface comprises one or more selectable links to the media content;
receiving a selection of one of the one or more selectable links; and
responsive to the selection, for displaying a second user interface having a second size larger than the first size, wherein the second user interface comprises the media content associated with the selection.

24. The non-transitory computer-readable storage medium of claim 23 , wherein the first user interface expands into the second user interface in response to the selection.

25. The non-transitory computer-readable storage medium of claim 23 , wherein the first user interface is overlaid on playing media content.

26. The non-transitory computer-readable storage medium of claim 23 , wherein the second user interface is overlaid on playing media content.

27. The non-transitory computer-readable storage medium of claim 23 , wherein the speech input comprises a query, and the media content comprises a result of the query.

28. The non-transitory computer-readable storage medium of claim 27 , wherein the first user interface comprises a link to results of the query beyond the one or more selectable links to the media content.

29. A system for controlling television interactions using a virtual assistant, the system comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving speech input from a user;
determining media content based on the speech input;
displaying a first user interface having a first size, wherein the first user interface comprises one or more selectable links to the media content;
receiving a selection of one of the one or more selectable links; and
responsive to the selection, for displaying a second user interface having a second size larger than the first size, wherein the second user interface comprises the media content associated with the selection.

30. The system of claim 29 , wherein the first user interface expands into the second user interface in response to the selection.

31. The system of claim 29 , wherein the first user interface is overlaid on playing media content.

32. The system of claim 29 , wherein the second user interface is overlaid on playing media content.

33. The system of claim 29 , wherein the speech input comprises a query, and the media content comprises a result of the query.

34. The system of claim 33 , wherein the first user interface comprises a link to results of the query beyond the one or more selectable links to the media content.

Assignees

• Apple Inc

Inventors

• Van Os Marcel
• Saddler Harry J
• Napolitano Lia T
• Russell Jonathan H
• Lister Patrick M
• Dasari Rohit

Classifications

• G06F16/738

  Presentation of query results · CPC title

• G06F16/73

  Querying · CPC title

• H04N21/84

  Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title