Devices, methods, and graphical user interfaces for interacting with three-dimensional environments

Patent metadata

Field	Value
Publication number	US-12288301-B2
Application number	US-202418584911-A
Country	US
Kind code	B2
Filing date	Feb 22, 2024
Priority date	Sep 25, 2020
Publication date	Apr 29, 2025
Grant date	Apr 29, 2025

Abstract

A computer system displays a first view of a three-dimensional environment, including a first user interface object, that when activated by a user input meeting first criteria, causes performance of a first operation. While displaying the first view, the computer system detects first movement of a hand in a physical environment, and in response, changes an appearance of the first user interface object in the first view based on the first movement, including: in accordance with a determination that the first movement meets the first criteria requiring the hand move in a first manner, performing and indicating performance of the first operation; and in accordance with a determination that the first movement does not meet the first criteria, moving the first user interface object away from a position in the three-dimensional environment corresponding to a location of the hand in the physical environment without performing the first operation.

First claim

What is claimed is:

1. A method, comprising:
at a computer system that is in communication with a first display generation component and one or more first input devices:
displaying, via the first display generation component, a first view of a three-dimensional environment, including displaying a first user interface object at a first position in the three-dimensional environment, wherein the first user interface object, when activated by a user input that meets first criteria, causes performance of a first operation;
while displaying the first view of the three-dimensional environment including displaying the first user interface object at the first position in the three-dimensional environment, detecting first movement of a hand in a physical environment; and
in response to detecting the first movement of the hand in the physical environment:
changing an appearance of the first user interface object in the first view of the three-dimensional environment based on the first movement of the hand in the physical environment, including:
in accordance with a determination that the first movement of the hand meets the first criteria, wherein the first criteria require that the hand moves in a first manner during the first movement of the hand in order for the first criteria to be met, performing the first operation associated with the first user interface object and changing the appearance of the first user interface object to indicate that the first operation has been performed; and
in accordance with a determination that the first movement of the hand does not meet the first criteria, moving the first user interface object away from a respective position in the three-dimensional environment that corresponds to a location of the hand in the physical environment without performing the first operation.

2. The method of claim 1 , wherein the first movement of the hand in the physical environment corresponds to a movement of a representation of the hand within the three-dimensional environment toward the first position in the three-dimensional environment, and the method includes:
in response to detecting the first movement of the hand in the physical environment, displaying, via the first display generation component, the representation of the hand moving in the three-dimensional environment toward the first position in the three-dimensional environment.

3. The method of claim 1 , wherein the first criteria require that the first movement of the hand includes movement of a first finger of the hand relative to a portion of the hand connected to the first finger.

4. The method of claim 1 , including:
in response to detecting the first movement of the hand in the physical environment:
in accordance with a determination that the first movement of the hand meets the first criteria, generating a first audio output in conjunction with performing the first operation.

5. The method of claim 1 , including:
in response to detecting the first movement of the hand in the physical environment:
in accordance with a determination that the first movement of the hand does not meet the first criteria, generating a second audio output in conjunction with moving the first user interface object away from the respective position in the three-dimensional environment that corresponds to the location of the hand in the physical environment.

6. The method of claim 1 , including:
in response to detecting the first movement of the hand in the physical environment:
generating a respective audio output with a respective audio output profile that is selected from a plurality of audio output profiles in accordance with whether the first criteria are met.

7. The method of claim 1 , wherein moving the first user interface object away from the respective position in the three-dimensional environment that corresponds to the location of the hand in the physical environment includes dynamically selecting a current position of the first user interface object in accordance with a current location of the hand in the physical environment.

8. A computer system, comprising:
a first display generation component;
one or more first input devices;
one or more processors; and
memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for:
displaying, via the first display generation component, a first view of a three-dimensional environment, including displaying a first user interface object at a first position in the three-dimensional environment, wherein the first user interface object, when activated by a user input that meets first criteria, causes performance of a first operation;
while displaying the first view of the three-dimensional environment including displaying the first user interface object at the first position in the three-dimensional environment, detecting first movement of a hand in a physical environment; and
in response to detecting the first movement of the hand in the physical environment:
changing an appearance of the first user interface object in the first view of the three-dimensional environment based on the first movement of the hand in the physical environment, including:
in accordance with a determination that the first movement of the hand meets the first criteria, wherein the first criteria require that the hand moves in a first manner during the first movement of the hand in order for the first criteria to be met, performing the first operation associated with the first user interface object and changing the appearance of the first user interface object to indicate that the first operation has been performed; and
in accordance with a determination that the first movement of the hand does not meet the first criteria, moving the first user interface object away from a respective position in the three-dimensional environment that corresponds to a location of the hand in the physical environment without performing the first operation.

9. The computer system of claim 8 , wherein the first movement of the hand in the physical environment corresponds to a movement of a representation of the hand within the three-dimensional environment toward the first position in the three-dimensional environment, and the one or more programs include instructions for:
in response to detecting the first movement of the hand in the physical environment, displaying, via the first display generation component, the representation of the hand moving in the three-dimensional environment toward the first position in the three-dimensional environment.

10. The computer system of claim 8 , wherein the first criteria require that the first movement of the hand includes movement of a first finger of the hand relative to a portion of the hand connected to the first finger.

11. The computer system of claim 8 , wherein the one or more programs include instructions for:
in response to detecting the first movement of the hand in the physical environment:
in accordance with a determination that the first movement of the hand meets the first criteria, generating a first audio output in conjunction with performing the first operation.

12. The computer system of claim 8 , wherein the one or more programs include instructions for:
in response to detecting the first movement of the hand in the physical environment:
in accordance with a determination that the first movement of the hand does not meet the first criteria, generating a second audio output in conjunction with moving the first user interface object away from the respective position in the three-dimensional environment that corr

Assignees

• Apple Inc

Inventors

• Rockel Philipp
• Hoyt Charles C

Classifications

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T2219/2021

  Shape modification · CPC title

• G06T19/003

  Navigation within 3D models or images · CPC title