Devices, methods, and graphical user interfaces for interacting with three-dimensional environments

Patent metadata

Field	Value
Publication number	US-12175010-B2
Application number	US-202017030219-A
Country	US
Kind code	B2
Filing date	Sep 23, 2020
Priority date	Sep 28, 2019
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

While displaying a three-dimensional scene including at least a first virtual object displayed with a first value corresponding to a first portion and a second value corresponding to a second portion of the first virtual object at a first location and a first physical surface at a second location, a computer system generates a first visual effect at the second location of the three-dimensional scene, including modifying a visual appearance of a first portion of the first physical surface in the three-dimensional scene in accordance with the first value for the first display property that corresponds to the first portion of the first virtual object; and modifying a visual appearance of a second portion of the first physical surface in the three-dimensional scene in accordance with the second value for the first display property that corresponds to the second portion of the first virtual object.

First claim

What is claimed is:

1. A method, comprising:
at a computer system including a display generation component and one or more input devices:
displaying, via the display generation component, a three-dimensional scene that includes a view of a physical environment and one or more virtual objects, wherein the three-dimensional scene includes at least a first virtual object at a first location and a first physical surface at a second location that is separate from the first location by a first portion of the physical environment, wherein the first virtual object is displayed with a first value for a first display property that corresponds to a first portion of the first virtual object and a second value for the first display property that corresponds to a second portion of the first virtual object, the second value of the first display property being distinct from the first value of the first display property; and
while displaying the three-dimensional scene including the first virtual object and the first physical surface, generating, via the display generation component, a first visual effect of the three-dimensional scene, wherein generating the first visual effect includes:
modifying a visual appearance of the first portion of the physical environment between the first virtual object and the first physical surface in the three-dimensional scene in accordance with at least one of the first value and the second value for the first display property of the first virtual object, including displaying a simulated glow of the first virtual object that extends across a portion of a visual field that is between the first virtual object and the first physical surface, wherein the simulated glow of the first virtual object is displayed concurrently with the first virtual object and a simulated shadow associated with the first physical surface in the three-dimensional scene;
modifying a visual appearance of a first portion of the first physical surface in the three-dimensional scene in accordance with the first value for the first display property that corresponds to the first portion of the first virtual object; and
modifying a visual appearance of a second portion of the first physical surface in the three-dimensional scene in accordance with the second value for the first display property that corresponds to the second portion of the first virtual object, wherein the visual appearance of the first portion of the first physical surface and the visual appearance for the second portion of the first physical surface are modified differently due to differences in the first value and the second value of the first display property in the first and second portions of the first virtual object;

while displaying, via the display generation component, the three-dimensional scene that includes the view of the physical environment and the one or more virtual objects, including the first virtual object at the first location and the first physical surface at the second location, detecting a change in a viewpoint of a user relative to the physical environment; and
in response to detecting the change in the viewpoint of the user relative to the physical environment, updating an appearance of the simulated glow of the first virtual object that extends across the portion of the visual field that is between the first virtual object at the first location and the first physical surface at the second location, and updating an appearance of the simulated shadow associated with the first physical surface, that is concurrently displayed with the first virtual object and the simulated glow of the first virtual object in the three dimensional scene.

2. The method of claim 1 , including:
detecting changes in appearance of the first virtual object, including value changes of the first display property in the first and second portions of the first virtual object; and
in response to detecting the changes in appearance of the first virtual object, modifying a visual appearance of the first physical surface at different portions of the first physical surface in accordance with the changes in appearance of the first virtual object, including:
modifying the visual appearance of the first portion of the first physical surface in accordance with a first relationship between the first display property and the visual appearance of the first portion of the first physical surface; and
modifying the visual appearance of the second portion of the first physical surface in accordance with a second relationship between the first display property and the visual appearance of the second portion of the first physical surface, wherein the first relationship and the second relationship correspond to different physical characteristics of the first and second portions of the first physical surface.

3. The method of claim 1 , wherein the first virtual object includes a virtual overlay on a second physical surface at a location that corresponds to the first location in the three-dimensional scene, and wherein the computer system changes an appearance of the virtual overlay in accordance with a change in respective values of one or more parameters including at least one of a time of day, a location, and a size of the virtual overlay.

4. The method of claim 3 , wherein generating the first visual effect includes:
modifying the visual appearance of the first portion of the first physical surface in accordance with changes in content shown in a first portion of the virtual overlay; and
modifying the visual appearance of the second portion of the first physical surface in accordance with changes in content shown in a second portion of the virtual overlay.

5. The method of claim 1 , wherein the first virtual object includes a virtual screen that displays media content at a location that corresponds to the first location in the three-dimensional scene, and wherein the computer system changes the media content shown on the virtual screen in accordance with playback progress of a media item associated with the media content.

6. The method of claim 5 , wherein generating the first visual effect includes:
modifying the visual appearance of the first portion of the first physical surface in accordance with changes in content shown in a first portion of the virtual screen; and
modifying the visual appearance of the second portion of the first physical surface in accordance with changes in content shown in a second portion of the virtual screen.

7. The method of claim 1 , wherein the first virtual object is a virtual assistant that interacts with a user via speech, and wherein the computer system changes an appearance of the virtual assistant in accordance with a mode of operation of the virtual assistant.

8. The method of claim 1 , wherein generating the first visual effect includes:
modifying the visual appearance of the first portion of the first physical surface in accordance with simulated reflection of the first virtual object on the first portion of the first physical surface; and
modifying the visual appearance of the second portion of the first physical surface in accordance with simulated reflection of the first virtual object on the second portion of the first physical surface.

9. The method of claim 1 , wherein generating the first visual effect includes:
modifying the visual appearance of the first portion of the first physical surface in accordance with simulated shadows cast by the first virtual object on the first portion of the first physical surface; and
modifying the visual appearance of the second portion of the first physical surface in accordance with simulated shadows of the first virtual object on the second portion of the first physical surface.

10. The meth

Assignees

• Apple Inc

Inventors

• Lemay Stephen O
• Faulkner Jeffrey M
• Pastrana Vicente Israel
• Rockel Philipp
• Holder Wesley M
• Conesa Pol Pla I
• Henderson Nicholas W
• Tilton Robert T
• Sorrentino Iii William A
• Anton Peter D
• Dye Alan C

Classifications

• G06F3/04845 Primary

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06V40/113

  Recognition of static hand signs · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/143

  Sensing or illuminating at different wavelengths · CPC title