Devices, methods, and graphical user interfaces for interacting with three-dimensional environments

Patent metadata

Field	Value
Publication number	US-11615597-B2
Application number	US-202117483761-A
Country	US
Kind code	B2
Filing date	Sep 23, 2021
Priority date	Sep 25, 2020
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

A computer system in communication with one or more input devices displays a first view of a three-dimensional environment, including a representation of a physical environment and a first user interface object having a first surface at a first position in the three-dimensional environment corresponding to a first location in the physical environment. While displaying the first view, the computer system detects movement, in the physical environment, of a first person not using the one or more input devices, and in response, in accordance with a determination that the movement of the first person in the physical environment has a first spatial relationship to the first location in the physical environment corresponding to the first user interface object, the computer system moves the first surface of the first user interface object in the first view in accordance with the movement of the first person in the physical environment.

First claim

What is claimed is:

1. A method, comprising:
at a computer system that is in communication with a first display generation component and one or more first input devices:
displaying, via the first display generation component, a first view of a three-dimensional environment, including displaying a representation of a physical environment surrounding the first display generation component and displaying a first user interface object comprising a first surface, wherein the first surface is displayed at a first position in the three-dimensional environment that corresponds to a first location in the physical environment;
while displaying the first view of the three-dimensional environment, detecting movement of a first person in the physical environment, wherein the first person is not a user of the one or more first input devices in communication with the computer system; and
in response to detecting the movement of the first person in the physical environment:
in accordance with a determination that the movement of the first person in the physical environment has a first spatial relationship to the first location in the physical environment that corresponds to the first surface, moving the first surface of the first user interface object in the first view of the three-dimensional environment in accordance with the movement of the first person in the physical environment.

2. The method of claim 1 , including:
in response to detecting the movement of the first person in the physical environment:
in accordance with a determination that the movement of the first person in the physical environment does not have the first spatial relationship to the first location in the physical environment that corresponds to the first user interface object, forgoing movement of the first surface of the first user interface object in the first view of the three-dimensional environment in accordance with the movement of the first person in the physical environment.

3. The method of claim 1 , including:
obtaining values of one or more characteristics of the movement of the first person in the physical environment; and
determining values of one or more characteristics of the movement of the first surface of the first user interface object in accordance with the values of the one or more characteristics of the movement of the first person in the physical environment, while a location of the first person is spatially separated from the first location by a distance.

4. The method of claim 1 , including:
in response to detecting movement of the first person in the physical environment relative to the first location in the physical environment:
in accordance with a determination that at least a portion of the first person in the physical environment has a second spatial relationship to the first location in the physical environment, generating a first audio output in conjunction with moving the first surface in the three-dimensional environment.

5. The method of claim 4 , including:
while displaying the first view of the three-dimensional environment, detecting movement of at least a portion of a second person in the physical environment, wherein the second person is a user of the one or more first input devices in communication with the computer system; and
in response to detecting the movement of at least the portion of the second person in the physical environment:
in accordance with a determination that at least the portion of the second person in the physical environment has the second spatial relationship to the first location in the physical environment, generating a second audio output in conjunction with moving the first surface in the three-dimensional environment, wherein the second audio output is different from the first audio output in at least one of one or more characteristics of the first audio output and the second audio output.

6. The method of claim 1 , including:
in response to detecting the movement of the first person in the physical environment:
in accordance with the determination that the movement of the first person in the physical environment has a third spatial relationship to the first location in the physical environment that corresponds to the first position of the first surface of the first user interface object, changing a first display property of the first surface of the first user interface object.

7. The method of claim 6 , wherein changing the first display property of the first surface of the first user interface object includes changing the first display property of a first portion of the first surface relative to a second portion of the first surface.

8. The method of claim 7 , wherein changing the first display property of the first portion of the first surface relative to the second portion of the first surface includes changing the first display property of the first portion of the first surface relative to the second portion of the first surface by a first amount, and changing the first display property of a third portion of the first surface that is between the first portion and the second portion of the first surface by a second amount that is smaller than the first amount.

9. The method of claim 6 , including:
in response to detecting the movement of the first person in the physical environment relative to the first location in the physical environment:
in accordance with a determination that the movement of the first person in the physical environment does not have the third spatial relationship to the first location in the physical environment that corresponds to the first position of the first surface of the first user interface object, restoring the first display property of the first surface of the first user interface object.

10. The method of claim 6 , including:
in response to detecting the movement of the first person in the physical environment relative to the first location in the physical environment:
in accordance with a determination that a location of a first portion of the first person has the third spatial relationship to the first location while a location of a second portion of the first person does not have the third spatial relationship to the first location, changing the first display property of the first surface of the first user interface object includes changing the first display property of the first portion of the first surface without changing the first display property of the second portion of the first surface; and
in accordance with a determination that an updated location of the first portion of the first person does not have the third spatial relationship to the first location while an updated location of the second portion of the first person has the third spatial relationship to the first location, changing the first display property of the first surface of the first user interface object includes restoring changes made to the first display property of the first portion of the first surface; and changing the first display property of the second portion of the first surface.

11. The method of claim 1 , including:
in response to detecting the movement of the first person in the physical environment:
in accordance with the determination that the movement of the first person in the physical environment has the first spatial relationship to the first location in the physical environment that corresponds to the first user interface object, modifying one or more display properties of the first surface in accordance with an appearance of the first person while the first person is at a location that corresponds to a position behind the first surface of the first user interface object relative to a viewpoint of the first view

Assignees

• Apple Inc

Inventors

• Rockel Philipp
• Dargan Dorian D

Classifications

• G06F3/04815 Primary

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T19/003

  Navigation within 3D models or images · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title