Method and device for presenting synthesized reality companion content

Patent metadata

Field	Value
Publication number	US-11727606-B2
Application number	US-202217707774-A
Country	US
Kind code	B2
Filing date	Mar 29, 2022
Priority date	Jan 22, 2018
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

In one implementation, a method includes: obtaining a user input to view SR content associated with video content; if the video content includes a first scene when the user input was detected: obtaining first SR content for a first time period of the video content associated with the first scene; obtaining a task associated with the first scene; and causing presentation of the first SR content and a first indication of the task associated with the first scene; and if the video content includes a second scene when the user input was detected: obtaining second SR content for a second time period of the video content associated with the second scene; obtaining a task associated with the second scene; and causing presentation of the second SR content and a second indication of the task associated with the second scene.

First claim

What is claimed is:

1. A method comprising:
at a computing system including non-transitory memory and one or more processors, wherein the computing system is communicatively coupled to a display device and one or more input devices:
while causing presentation of video content, obtaining, via the one or more input devices, a user input indicating a request to view synthesized reality (SR) content associated with the video content;
in accordance with a determination that the video content includes a first scene associated with to a first time period of the video content when the user input was detected:
obtaining first SR content for the first time period of the video content, wherein the first SR content is associated with the first scene within the video content;
obtaining one or more missions or tasks associated with the first scene within the video content; and
causing presentation of the first SR content associated with the first scene within the video content and a first indication of the one or more missions or tasks associated with the first scene within the video content via the display device; and

in accordance with a determination that the video content includes a second scene associated with to a second time period of the video content when the user input was detected:
obtaining second SR content for the first time period of the video content, wherein the second SR content is associated with the second scene within the video content;
obtaining one or more missions or tasks associated with the second scene within the video content; and
causing presentation of the second SR content associated with the second scene within the video content and a second indication of the one or more missions or tasks associated with the second scene within the video content via the display device.

2. The method of claim 1 , wherein the first SR content and second SR content are obtained from a library associated with the video content.

3. The method of claim 1 , wherein the first SR content is generated on-the-fly based at least in part on the first scene within the video content, and wherein the second SR content is generated on-the-fly based at least in part on the second scene within the video content.

4. The method of claim 1 , wherein the first SR content continues concurrently while causing presentation of the first SR content, and wherein the second SR content continues concurrently while causing presentation of the second SR content.

5. The method of claim 4 , wherein the first SR content is paused in response to the user breaching a first threshold region associated with the first SR content, and wherein the second SR content is paused in response to the user breaching a second threshold region associated with the second SR content.

6. The method of claim 1 , wherein the first SR content is paused while causing presentation of the first SR content, and wherein the second SR content is paused while causing presentation of the second SR content.

7. The method of claim 1 , further comprising:
while presenting the first SR content and in response to receiving a first command from the user, ceasing to present the first SR content and continuing causing presentation of the first scene associated with to the first time period of the video content; and
while presenting the second SR content and in response to receiving a second command from the user, ceasing to present the second SR content and continuing causing presentation of the second scene associated with to the second time period of the video content.

8. The method of claim 1 , further comprising:
while causing presentation of the video content that includes the first scene, causing display of a first affordance indicating availability of the first SR content associated with the first scene within the video content, wherein the user input corresponds to selection of the first affordance; and
while causing presentation of the video content that includes the second scene, causing display of a second affordance indicating availability of the second SR content associated with the second scene within the video content, wherein the user input corresponds to selection of the second affordance.

9. The method of claim 1 , further comprising:
while causing presentation of the first SR content
obtaining, via the one or more input devices, one or more user interactions with the first SR content; and
adjusting the first SR content in response to receiving the one or more user interactions with the first SR content; and

while causing presentation of the second SR content
obtaining, via the one or more input devices, one or more user interactions with the second SR content; and
adjusting the first SR content in response to receiving the one or more user interactions with the second SR content.

10. The method of claim 9 , wherein the one or more user interactions with the first SR content include at least one of picking up an object associated with the first SR content, physically engaging with an SR character associated with the first SR content, or speaking with an SR character associated with the first SR content.

11. The method of claim 9 , further comprising:
in response to determining that the one or more user interactions with the first SR content satisfy an exit condition for the first scene, ceasing to cause presentation of the first SR content and causing presentation of the first scene associated with to the first time period of the video content, wherein the exit condition for the first scene is satisfied when the one or more user interactions with the first SR content complete a mission or task associated with the first scene within the video content; and
in response to determining that the one or more user interactions with the second SR content satisfy an exit condition for the second scene, ceasing to cause presentation of the second SR content and causing presentation of the second scene associated with to the second time period of the video content, wherein the exit condition for the second scene is satisfied when the one or more user interactions with the second SR content complete a mission or task associated with the second scene within the video content.

12. The method of claim 1 , further comprising:
while causing presentation of the first SR content
obtaining, via the one or more input devices, one or more commands associated with driving a first SR character within the first SR content; and
adjusting the first SR content in response to obtaining the one or more commands associated with driving the first SR character within the first SR content; and

while causing presentation of the second SR content
obtaining, via the one or more input devices, one or more commands associated with driving a second SR character within the second SR content; and
adjusting the second SR content in response to obtaining the one or more commands associated with driving the second SR character within the second SR content.

13. The method of claim 1 , further comprising:
presenting, via the display device, the one or more missions or tasks associated with the first scene within the video content; and
detecting, via the one or more input devices, a user input selecting a respective mission or task among the one or more missions or tasks associated with the first scene within the video content, wherein the first indication corresponds to the respective mission or task selected by the user input.

14. A computing system comprising:
one or more processors;
a non-transitory memory;
an interface for communicating with a display device and one or more input devices;

Assignees

• Apple Inc

Inventors

• Richter Ian M
• Ulbricht Daniel
• Nahmias Jean-Daniel E
• Elafifi Omar
• Meier Peter

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/04815 Primary

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G02B27/017

  Head mounted · CPC title

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title