Devices, methods, and graphical user interfaces for interacting with three-dimensional environments

Patent metadata

Field	Value
Publication number	US-11810244-B2
Application number	US-202218091154-A
Country	US
Kind code	B2
Filing date	Dec 29, 2022
Priority date	Sep 25, 2020
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

A computer system displays a first view of a three-dimensional environment, including a first user interface object, while a first user is at a first location in a first physical environment. A respective position of the first user interface object in the three-dimensional environment corresponds to a respective location of the first object in a second physical environment. The computer system detects movement of the first user in the first physical environment or movement of the first object in the second physical environment, and in response, displays a second view corresponding to a second viewpoint, and the first user interface object in the second view. The first user interface object is displayed at a first or second display position, in accordance with a determination that the respective position of the first user interface object is more or less than a threshold distance from the second viewpoint, respectively.

First claim

What is claimed is:

1. A method, comprising:
at a computer system that is in communication with a first display generation component and one or more first input devices:
while a first user is at a first location in a first physical environment, displaying a first view of a three-dimensional environment corresponding to a first viewpoint that is associated with the first location in the first physical environment, wherein the first view of the three-dimensional environment includes a first user interface object that represents a first object in a second physical environment different from the first physical environment, wherein a respective position of the first user interface object in the three-dimensional environment corresponds to a respective location of the first object in the second physical environment in a first manner;
detecting at least one of movement of the first user in the first physical environment and movement of the first object in the second physical environment; and
in response to detecting the at least one of movement of the first user in the first physical environment and movement of the first object in the second physical environment:
displaying a second view of the three-dimensional environment corresponding to a second viewpoint; and
displaying the first user interface object in the second view of the three-dimensional environment, including:
in accordance with a determination that the respective position of the first user interface object in the three-dimensional environment that corresponds to the respective location of the first object in the second physical environment in the first manner is more than a threshold distance from a respective position in the three-dimensional environment that corresponds to the second viewpoint associated with the second view of the three-dimensional environment, displaying the first user interface object at a first display position in the second view of in the three-dimensional environment, wherein the first display position is the respective position of the first user interface object in the three-dimensional environment; and
in accordance with a determination that the respective position of the first user interface object in the three-dimensional environment that corresponds to the respective location of the first object in the second physical environment in the first manner is less than the threshold distance from the respective position in the three-dimensional environment that corresponds to the second viewpoint associated with the second view of the three-dimensional environment, displaying the first user interface object at a second display position in the second view of the three-dimensional environment, wherein the second display position is offset from the respective position of the first user interface object in the three-dimensional environment.

2. The method of claim 1 , wherein:
detecting the at least one of movement of the first user in the first physical environment and movement of the first object in the second physical environment includes detecting first movement of the first user in the first physical environment while the first object remains stationary in the second physical environment; and
during the first movement of the first user in the first physical environment:
in accordance with the determination that the respective position of the first user interface object in the three-dimensional environment that corresponds to the respective location of the first object in the second physical environment in the first manner is more than the threshold distance from the respective position in the three-dimensional environment that corresponds to a viewpoint associated with a currently displayed view of the three-dimensional environment, the first user interface object is displayed at the respective position of the first user interface object in the three-dimensional environment that corresponds to the respective location of the first object in the second physical environment in the first manner; and
in accordance with the determination that the respective position of the first user interface object in the three-dimensional environment that corresponds to the respective location of the first object in the second physical environment in the first manner is not more than the threshold distance from the respective position in the three-dimensional environment that corresponds to the viewpoint associated with the currently displayed view of the three-dimensional environment, the first user interface object is displayed at an adjusted position in the three-dimensional environment while the first object remains stationary in the second physical environment, wherein the adjusted position in the three-dimensional environment corresponds to the respective location of the first object in the second physical environment in a second manner different from the first manner.

3. The method of claim 1 , wherein:
detecting the at least one of movement of the first user in the first physical environment and movement of the first object in the second physical environment includes detecting second movement of the first object in the second physical environment while the first user remains stationary in the first physical environment; and
during the second movement of the first object in the second physical environment:
in accordance with the determination that the respective position of the first user interface object in the three-dimensional environment that corresponds to the respective location of the first object in the second physical environment in the first manner is more than the threshold distance from the respective position in the three-dimensional environment that corresponds to a viewpoint associated with a currently displayed view of the three-dimensional environment, the first user interface object is displayed at an updated position in the three-dimensional environment, wherein the updated position in the three-dimensional environment corresponds to an updated location of the first object in the second physical environment as a result of the second movement in the first manner; and
in accordance with the determination that the respective position of the first user interface object in the three-dimensional environment that corresponds to the respective location of the first object in the second physical environment in the first manner is not more than the threshold distance from the respective position in the three-dimensional environment that corresponds to the viewpoint associated with the currently displayed view of the three-dimensional environment, the first user interface object is displayed at an adjusted updated position in the three-dimensional environment while the first object remains stationary in the second physical environment, wherein the adjusted updated position in the three-dimensional environment corresponds to the updated location of the first object in the second physical environment in a second manner different from the first manner.

4. The method of claim 1 , wherein:
detecting the at least one of movement of the first user in the first physical environment and movement of the first object in the second physical environment includes concurrently detecting third movement of the first user in the first physical environment and fourth movement of the first object in the second physical environment; and
during the third movement of the first user in the first physical environment and the fourth movement of the first object in the second physical environment:
in accordance with the determination that the respective position of the first user interface object in the three-dimensional environment that corresponds to the location of the first object in the second physical environment in the first manner is more than the threshold distance from th

Assignees

• Apple Inc

Inventors

• Rockel Philipp
• Henderson Nicholas W
• Bauerly Kristi E S

Classifications

• G06T15/10 Primary

  Geometric effects · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• A63F13/426

  involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun · CPC title