Devices, methods, and graphical user interfaces for interacting with three-dimensional environments

Patent metadata

Field	Value
Publication number	US-12578835-B2
Application number	US-202318097136-A
Country	US
Kind code	B2
Filing date	Jan 13, 2023
Priority date	Sep 24, 2020
Publication date	Mar 17, 2026
Grant date	Mar 17, 2026

Abstract

A system displays, via the first display generation component: a first view of a computer-generated three-dimensional environment, including a representation of a physical environment, and a representation of one or more projections of light. The representation of the one or more projections of light has an appearance that corresponds to a spatial relationship between first virtual content present in the computer-generated three-dimensional environment and the representation of the physical environment. The system displays animated changes of the representation of the one or more projections of light and displays alterations of an appearance of a representation of a first surface in the physical environment in accordance with the animated changes of the representation of the one or more projections of light.

First claim

What is claimed is:

1 . A method, comprising:
at a computing system including a first display generation component and one or more input devices:
displaying, via the first display generation component:
a first view of a first computer-generated augmented reality three-dimensional environment, wherein the first view of the first computer-generated augmented reality three-dimensional environment includes:
a representation of a respective portion of a physical environment, including an augmented reality representation of a first physical surface; and
a first representation of virtual illumination in a first portion of the first computer-generated augmented reality three-dimensional environment, the first representation of the virtual illumination including a first representation of one or more projections of light in the first portion of the first computer-generated augmented reality three-dimensional environment, wherein the first representation of the one or more projections of light has an appearance that corresponds to a spatial relationship between first virtual content present in the first computer-generated augmented reality three-dimensional environment and the representation of the respective portion of the physical environment, including the first physical surface, in the first computer-generated augmented reality three-dimensional environment;

while displaying the first view of the first computer-generated augmented reality three-dimensional environment, displaying animated changes of the virtual illumination, including:
displaying changes of the first representation of the one or more projections of light in the first portion of the first computer-generated augmented reality three-dimensional environment; and
displaying one or more changes within the first representation of the virtual illumination in the respective portion of the physical environment, including changing an appearance of a first portion of the augmented reality representation of the first physical surface within the first representation of the virtual illumination relative to a second portion of the augmented reality representation of the first physical surface within the first representation of the virtual illumination.

2 . The method of claim 1 , including:
moving the first virtual content in the first computer-generated augmented reality three-dimensional environment relative to the augmented reality representation of the first physical surface in the first computer-generated augmented reality three-dimensional environment, wherein the animated changes of the virtual illumination, including changes of the first representation of the one or more projections of light is generated in accordance with movement of the first virtual content in the first computer-generated augmented reality three-dimensional environment.

3 . The method of claim 2 , wherein moving the first virtual content relative to the augmented reality representation of the first physical surface in the first computer-generated augmented reality three-dimensional environment includes:
at a first time, displaying the first representation of the one or more projections of light without displaying the first virtual content within the first view of the first computer-generated augmented reality three-dimensional environment including the representation of the respective portion of the physical environment; and at a second time after the first time, displaying the first virtual content within the representation of the respective portion of the physical environment and displaying the first representation of the one or more projections of light extending outward from the first virtual content within the first view of the first computer-generated augmented reality three-dimensional environment.

4 . The method of claim 1 , including:
changing an appearance of the first virtual content in the first computer-generated augmented reality three-dimensional environment, wherein the animated changes of the virtual illumination, including changes of the first representation of the one or more projections of light in the first portion of the first computer-generated augmented reality three-dimensional environment is generated at least in part based on changes in the appearance of the first virtual content.

5 . The method of claim 1 , wherein, for at least a first period of time that the first virtual content is visible via the first display generation component, the first virtual content moves relative to the first computer-generated augmented reality three-dimensional environment in accordance with movement of a viewpoint of a user that is in a position to view content displayed via the first display generation component.

6 . The method of claim 1 , wherein, for at least a first period of time that the first virtual content is visible via the first display generation component, the first virtual content maintains its position in the first computer-generated augmented reality three-dimensional environment while a viewpoint of a user that is in a position to view content displayed via the first display generation component changes relative to the first computer-generated augmented reality three-dimensional environment.

7 . The method of claim 1 , including:
detecting a first gaze input directed to a respective portion of a current field of view provided by the first display generation component; and in response to detecting the first gaze input, and in accordance with a determination that the first gaze input meets first criteria, displaying the first virtual content in the current field of view provided by the first display generation component.

8 . The method of claim 1 , wherein changing the appearance of the augmented reality representation of the first physical surface in the respective portion of the physical environment is further based on one or more surface properties of the first physical surface.

9 . An electronic device, comprising:
a first display generation component; one or more input devices; one or more processors; and memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for:
displaying, via the first display generation component:
a first view of a first computer-generated augmented reality three-dimensional environment, wherein the first view of the first computer-generated augmented reality three-dimensional environment includes:
a representation of a respective portion of a physical environment, including an augmented reality representation of a first physical surface; and
a first representation of virtual illumination in a first portion of the first computer-generated augmented reality three-dimensional environment, the first representation of the virtual illumination including a first representation of one or more projections of light in the first portion of the first computer-generated augmented reality three-dimensional environment, wherein the first representation of the one or more projections of light has an appearance that corresponds to a spatial relationship between first virtual content present in the first computer-generated augmented reality three-dimensional environment and the representation of the respective portion of the physical environment, including the first physical surface, in the first computer-generated augmented reality three-dimensional environment;

while displaying the first view of the first computer-generated augmented reality three-dimensional environment, displaying animated changes of the virtual illumination, including:
displaying changes of the first representation of the one or more projections of

Assignees

• Apple Inc

Inventors

• Rockel Philipp
• Faulkner Jeffrey M
• Pastrana Vicente Israel
• Holder Wesley M
• Henderson Nicholas W
• Tilton Robert T
• Dellinger Richard R
• Lemay Stephen O
• Sorrentino Iii William A
• Yerkes Giancarlo
• Mari Pedro
• Anton Peter D

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06T2219/2021

  Shape modification · CPC title