System and method of three-dimensional immersive applications in multi-user communication sessions

Patent metadata

Field	Value
Publication number	US-12272005-B2
Application number	US-202318174337-A
Country	US
Kind code	B2
Filing date	Feb 24, 2023
Priority date	Feb 28, 2022
Publication date	Apr 8, 2025
Grant date	Apr 8, 2025

Abstract

Some examples of the disclosure are directed to selective display of avatars corresponding to users of electronic devices in a multi-user communication session. In some examples, when immersive content is shared in the communication session, the avatars remain displayed when presenting the content in the three-dimensional environment. In some examples, when perspective-limited immersive content is shared in the communication session, the avatars cease being displayed when presenting the content in the three-dimensional environment. In some examples, when content presented in a full-screen mode is shared in the communication session, the avatars remain displayed when presenting the content in the full-screen mode in the three-dimensional environment. In some examples, when object-bounded content is shared in the communication session, the avatars remain displayed when presenting the object-bounded content in the three-dimensional environment.

First claim

What is claimed is:

1. A method comprising:
at a first electronic device in communication with a display, one or more input devices, and a second electronic device:
while in a communication session with the second electronic device, presenting, via the display, a first computer-generated environment including an avatar corresponding to a user of the second electronic device;
while displaying the first computer-generated environment including the avatar corresponding to the user of the second electronic device, receiving, via the one or more input devices, a first indication corresponding to a request from the second electronic device to share content with the first electronic device; and
in response to receiving the first indication:
in accordance with a determination that the request is accepted:
 in accordance with a determination that the content shared with the first electronic device is a first type of content:
 replacing display of the first computer-generated environment with a second computer-generated environment corresponding to the content; and
 displaying the avatar corresponding to the user of the second electronic device in the second computer-generated environment; and
 in accordance with a determination that the content shared with the first electronic device is a second type of content, different from the first type of content:
 updating display of the first computer-generated environment to include a first object corresponding to the content; and
 ceasing display of the avatar corresponding to the user of the second electronic device in the first computer-generated environment.

2. The method of claim 1 , wherein displaying the avatar corresponding to the user of the second electronic device includes presenting audio corresponding to a voice of the user of the second electronic device, the method further comprising:
in accordance with the determination that the content shared with the first electronic device is the second type of content, different from the first type of content, maintaining presentation of the audio corresponding to the voice of the user of the second electronic device in the first computer-generated environment after ceasing display of the avatar corresponding to the user of the second electronic device;
wherein:
presenting the audio corresponding to the voice of the user of the second electronic device includes presenting spatial audio corresponding to the voice of the user of the second electronic device in the first computer-generated environment; and
in accordance with the determination that the content shared with the first electronic device is the second type of content, the audio corresponding to the voice of the user of the second electronic device presented in the first computer-generated environment is non-spatial audio.

3. The method of claim 1 , wherein the first type of content is content that includes a three-dimensional immersive environment, and the second computer-generated environment corresponding to the content is a representation of the three-dimensional immersive environment, the method further comprising:
while displaying the second computer-generated environment, detecting, via the one or more input devices, movement of the first electronic device in a physical environment surrounding the first electronic device from a first location to a second location; and
in response to detecting the movement of the first electronic device:
changing a location of a viewpoint of the user of the first electronic device in the second computer-generated environment from a first respective location to a second respective location, wherein the second respective location in the second computer-generated environment is based on the second location in the physical environment; and
maintaining display of the avatar corresponding to the user of the second electronic device in the second computer-generated environment.

4. The method of claim 1 , wherein the second type of content is content that includes a viewpoint-limited three-dimensional immersive video, scene, or environment, and the first object corresponding to the content is a representation of the viewpoint-limited three-dimensional immersive video, scene, or environment, the method further comprising:
while displaying the first computer-generated environment including the first object, detecting, via the one or more input devices, movement of a respective portion of the user of the first electronic device from a first pose to a second pose; and
in response to detecting the movement of the respective portion of the user:
changing a viewpoint of the first object in the first computer-generated environment from a first viewpoint to a second viewpoint, different from the first viewpoint, wherein the second viewpoint is based on the second pose of the respective portion of the user; and
restricting changing a location of the viewpoint of the user of the first electronic device in the first computer-generated environment.

5. The method of claim 1 , further comprising:
in response to receiving the first indication:
in accordance with a determination that the request is accepted:
in accordance with a determination that the content shared with the first electronic device is a third type of content, different from the first type of content and the second type of content:
updating display of the first computer-generated environment to include a second object, different from the first object, corresponding to the content; and
maintaining display of the avatar corresponding to the user of the second electronic device in the first computer-generated environment.

6. The method of claim 5 , wherein:
the third type of content is content that includes two-dimensional content configured to be displayed in the second object in the first computer-generated environment; or
the third type of content is an application object associated with an application running on the second electronic device, the application object configured to display second content.

7. The method of claim 1 , further comprising:
after replacing display of the first computer-generated environment with the second computer-generated environment corresponding to the content and displaying the avatar corresponding to the user of the second electronic device in the second computer-generated environment in accordance with the determination that the content shared with the first electronic device is the first type of content, receiving, via the one or more input devices, a second input corresponding to a request to navigate away from the second computer-generated environment corresponding to the content; and
in response to receiving the second input:
replacing display of the second computer-generated environment with the first computer-generated environment; and
forgoing display of the avatar corresponding to the user of the second electronic device in the first computer-generated environment.

8. The method of claim 1 , further comprising:
after updating display of the first computer-generated environment to include the first object corresponding to the content and ceasing display of the avatar corresponding to the user of the second electronic device in the first computer-generated environment in accordance with the determination that the content shared with the first electronic device is the second type of content, receiving, via the one or more input devices, a second input corresponding to a request to cease display of the first object corresponding to the content; and
in response to receiving the second input:
ceasing display of the first object corresponding to the content in the first computer-generated environment; and
forgoing redisplay of the avatar corres

Assignees

• Apple Inc

Inventors

• Ren Miao
• Smith Connor A
• Lee Hayden J
• Sommer Bruno M

Classifications

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• H04N13/351

  for displaying simultaneously · CPC title

• A63F13/60

  Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor · CPC title