Intelligent automated assistant for TV user interactions

Patent metadata

Field	Value
Publication number	US-9668024-B2
Application number	US-201615085465-A
Country	US
Kind code	B2
Filing date	Mar 30, 2016
Priority date	Jun 30, 2014
Publication date	May 30, 2017
Grant date	May 30, 2017

Abstract

Systems and processes are disclosed for controlling television user interactions using a virtual assistant. A virtual assistant can interact with a television set-top box to control content shown on a television. Speech input for the virtual assistant can be received from a device with a microphone. User intent can be determined from the speech input, and the virtual assistant can execute tasks according to the user's intent, including causing playback of media on the television. Virtual assistant interactions can be shown on the television in interfaces that expand or contract to occupy a minimal amount of space while conveying desired information. Multiple devices associated with multiple displays can be used to determine user intent from speech input as well as to convey information to users. In some examples, virtual assistant query suggestions can be provided to the user based on media content shown on a display.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device having a processor, memory, and a first display, cause the device to:
receive speech input from a user;
determine a user intent of the speech input based on content displayed on the first display;
determine media content based on the user intent; and
play the media content on a second device associated with a second display.

2. The computer-readable storage medium of claim 1 , wherein the content displayed on the first display comprises an application interface.

3. The computer-readable storage medium of claim 2 , wherein the speech input comprises a request to display media associated with the application interface.

4. The computer-readable storage medium of claim 3 , wherein the application interface comprises a television program listing, and the media comprises a television program in the television program listing.

5. The computer-readable storage medium of claim 1 , wherein the instructions further cause the device to:
identify the user based on the speech input; and
determine the user intent of the speech input based on data associated with the identified user.

6. The computer-readable storage medium of claim 1 , wherein the instructions further cause the device to:
determine whether the user is an authorized user based on the speech input;
wherein the media content is played on the second device in response to a determination that the user is an authorized user.

7. The computer-readable storage medium of claim 6 , wherein determining whether the user is an authorized user comprises analyzing the speech input using voice recognition.

8. The computer-readable storage medium of claim 1 , wherein the instructions further cause the device to:
in response to determining that the user intent comprises a request for information, display information associated with the media content on the first display of the device; and
in response to determining that the user intent comprises a request to play the media content, play the media content on the second device.

9. The computer-readable storage medium of claim 1 , wherein the speech input comprises a request to play content on the second device; and
wherein the media content is played on the second device in response to the request to play content on the second device.

10. The computer-readable storage medium of claim 1 , wherein the instructions further cause the device to:
determine whether the determined media content should be displayed on the first display or the second display based on a media format, a user preference, or a default setting;
wherein the media content is displayed on the second display in response to a determination that the determined media content should be displayed on the second display; and
wherein the media content is displayed on the first display in response to a determination that the determined media content should be displayed on the first display.

11. The computer-readable storage medium of claim 1 , wherein the instructions further cause the device to:
determine a proximity of each of two or more devices, including the second device and a third device;
wherein the media content is played on the second device associated with the second display based on the proximity of the second device relative to the proximity of the third device.

12. The computer-readable storage medium of claim 1 , wherein the instructions further cause the device to:
display a list of display devices, including the second device associated with the second display; and
receive a selection of the second device in the list of display devices;
wherein the media content is displayed on the second display in response to receiving the selection of the second device.

13. The computer-readable storage medium of claim 1 , wherein the instructions further cause the device to:
determine whether headphones are attached to the device;
in response to a determination that headphones are attached to the device, display the media content on the first display; and
in response to a determination that headphones are not attached to the device, display the media content on the second display.

14. The computer-readable storage medium of claim 1 , wherein the instructions further cause the device to:
determine alternative media content for display based on one or more of a user preference, a show popularity, and a status of a live sporting event; and
display a notification comprising the determined alternative media content.

15. A method for controlling television interactions using a virtual assistant, the method comprising:
at an electronic device having a first display:
receiving speech input from a user;
determining a user intent of the speech input based on content displayed on the first display;
determining media content based on the user intent; and
playing the media content on a second device associated with a second display.

16. The method of claim 15 , further comprising:
identifying the user based on the speech input; and
determining the user intent of the speech input based on data associated with the identified user.

17. The method of claim 15 , further comprising:
in response to determining that the user intent comprises a request for information, displaying information associated with the media content on the first display of the device; and
in response to determining that the user intent comprises a request to play the media content, playing the media content on the second device.

18. The method of claim 15 , further comprising:
determining whether the determined media content should be displayed on the first display or the second display based on a media format, a user preference, or a default setting;
wherein the media content is displayed on the second display in response to a determination that the determined media content should be displayed on the second display; and
wherein the media content is displayed on the first display in response to a determination that the determined media content should be displayed on the first display.

19. The method of claim 15 , further comprising:
displaying a list of display devices, including the second device associated with the second display; and
receiving a selection of the second device in the list of display devices;

wherein the media content is displayed on the second display in response to receiving the selection of the second device.

20. The method of claim 15 , further comprising:
determining alternative media content for display based on one or more of a user preference, a show popularity, and a status of a live sporting event; and
displaying a notification comprising the determined alternative media content.

21. An electronic device comprising:
a first display;
one or more processors; and
memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for:
receiving speech input from a user;
determining a user intent of the speech input based on content displayed on the first display;
determining media content based on the user intent; and
playing the media content on a second device associated with a second display.

22. The device of claim 21 , wherein the one or more programs further include instructions for:
identifying the user b

Assignees

• Apple Inc

Inventors

• Os Marcel Van
• Saddler Harry J
• Napolitano Lia T
• Russell Jonathan H
• Lister Patrick M
• Dasari Rohit

Classifications

• H04N21/4828

  for searching programme descriptors (retrieval of video data G06F16/739) · CPC title

• H04N21/4316

  for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window · CPC title

• H04N21/42222

  Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device · CPC title

• H04N21/4312

  involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title