Interactive Element in a Replay

Patent metadata

Field	Value
Publication number	US-2024054746-A1
Application number	US-202318345661-A
Country	US
Kind code	A1
Filing date	Jun 30, 2023
Priority date	Aug 11, 2022
Publication date	Feb 15, 2024
Grant date	—

Abstract

An electronic device such as a head-mounted device may present extended reality content such as a representation of a three-dimensional environment. The representation of the three-dimensional environment may be changed between different viewing modes having different immersion levels in response to user input. The three-dimensional environment may represent a multiuser communication session. A multiuser communication session may be saved and subsequently viewed as a replay. There may be an interactive virtual object within the replay of the multiuser communication session. The pose of the interactive virtual object may be manipulated by a user while the replay is paused. Some multiuser communication sessions may be hierarchical multiuser communication sessions with a presenter and audience members. The presenter and audience members may receive generalized feedback based on the audience members during the presentation. A participant may have their role changed between audience member and presenter during the presentation.

First claim

What is claimed is:

1 . An electronic device comprising:
one or more sensors; one or more displays; one or more processors; and memory storing instructions configured to be executed by the one or more processors, the instructions for:
presenting, using at least the one or more displays, a replay of an extended reality session that includes an interactive virtual object;
obtaining, via the one or more sensors, a first user input;
in response to the first user input, pausing the replay of the extended reality session;
obtaining, via the one or more sensors, a second user input; and
in response to the second user input and while the replay of the extended reality session is paused, modifying a pose of the interactive virtual object within the replay of the extended reality session.

2 . The electronic device defined in claim 1 , wherein the instructions further comprise instructions for:
while presenting, using at least the one or more displays, the replay of the extended reality session that includes the interactive virtual object, modifying the pose of the interactive virtual object without user input based on the replay.

3 . The electronic device defined in claim 1 , wherein, when the replay is paused, the pose of the interactive virtual object is static without user input.

4 . The electronic device defined in claim 1 , wherein the replay of the extended reality session includes an additional virtual object, wherein a pose of the additional virtual object changes while the replay is presented, wherein the pose of the additional virtual object is static when the replay of the extended reality session is paused, and wherein the pose of the additional virtual object is not responsive to user input when the replay of the extended reality session is paused.

5 . The electronic device defined in claim 1 , wherein the instructions further comprise instructions for:
obtaining, via the one or more sensors, a third user input; and after modifying the pose of the interactive virtual object within the replay of the extended reality session in response to the second user input and while the replay of the extended reality session is paused:
in response to the third user input, resuming the replay of the extended reality session.

6 . The electronic device defined in claim 5 , wherein the pose of the interactive virtual object has a given state before the replay of the extended reality session is resumed and wherein the pose of the interactive virtual object remains in the given state when the replay of the extended reality session is resumed.

7 . The electronic device defined in claim 5 , wherein the pose of the interactive virtual object has a given state before the replay of the extended reality session is resumed and wherein the pose of the interactive virtual object changes from the given state to a state that is synchronized with the replay when the replay of the extended reality session is resumed.

8 . The electronic device defined in claim 1 , wherein the replay of the extended reality session includes an avatar, wherein pausing the replay of the extended reality session causes movement of the avatar and audio from the avatar to be paused, and wherein a pose of the avatar is not responsive to user input when the replay of the extended reality session is paused.

9 . The electronic device defined in claim 1 , wherein the instructions further comprise instructions for:
obtaining, via the one or more sensors, a fourth user input directed to a portion of the paused replay of the extended reality session; and starting an application based on the fourth user input, wherein starting the application comprises displaying a video or application window overlaid on the paused replay of the extended reality session.

10 . A method of operating an electronic device that comprises one or more sensors and one or more displays, the method comprising:
presenting, using at least the one or more displays, a replay of an extended reality session that includes an interactive virtual object; obtaining, via the one or more sensors, a first user input; in response to the first user input, pausing the replay of the extended reality session; obtaining, via the one or more sensors, a second user input; and in response to the second user input and while the replay of the extended reality session is paused, modifying a pose of the interactive virtual object within the replay of the extended reality session.

11 . The method defined in claim 10 , further comprising:
while presenting, using at least the one or more displays, the replay of the extended reality session that includes the interactive virtual object, modifying the pose of the interactive virtual object without user input based on the replay.

12 . The method defined in claim 10 , wherein, when the replay is paused, the pose of the interactive virtual object is static without user input.

13 . The method defined in claim 10 , wherein the replay of the extended reality session includes an additional virtual object, wherein a pose of the additional virtual object changes while the replay is presented, wherein the pose of the additional virtual object is static when the replay of the extended reality session is paused, and wherein the pose of the additional virtual object is not responsive to user input when the replay of the extended reality session is paused.

14 . The method defined in claim 10 , further comprising:
obtaining, via the one or more sensors, a third user input; and after modifying the pose of the interactive virtual object within the replay of the extended reality session in response to the second user input and while the replay of the extended reality session is paused:
in response to the third user input, resuming the replay of the extended reality session.

15 . The method defined in claim 14 , wherein the pose of the interactive virtual object has a given state before the replay of the extended reality session is resumed and wherein the pose of the interactive virtual object remains in the given state when the replay of the extended reality session is resumed.

16 . The method defined in claim 14 , wherein the pose of the interactive virtual object has a given state before the replay of the extended reality session is resumed and wherein the pose of the interactive virtual object changes from the given state to a state that is synchronized with the replay when the replay of the extended reality session is resumed.

17 . The method defined in claim 10 , wherein the replay of the extended reality session includes an avatar, wherein pausing the replay of the extended reality session causes movement of the avatar and audio from the avatar to be paused, and wherein a pose of the avatar is not responsive to user input when the replay of the extended reality session is paused.

18 . The method defined in claim 10 , further comprising:
obtaining, via the one or more sensors, a fourth user input directed to a portion of the paused replay of the extended reality session; and starting an application based on the fourth user input, wherein starting the application comprises displaying a video or application window overlaid on the paused replay of the extended reality session.

19 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device that comprises one or more sensors and one or more displays, the one or more programs including instructions for:
prese

Assignees

• Apple Inc

Inventors

• Burns Aaron M
• Poulos Adam G
• Palangie Alexis H
• Blachnitzky Benjamin R
• Papadopoulos Charilaos
• Schattel David M
• Demirayak Ezgi
• Wang Jia
• Abbasian Reza
• Carlin Ryan S

Classifications

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title

• H04N21/4316

  for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window · CPC title