Devices, Methods, and Graphical User Interfaces for System-Wide Behavior for 3D Models

Patent metadata

Field	Value
Publication number	US-2024272766-A1
Application number	US-202418631905-A
Country	US
Kind code	A1
Filing date	Apr 10, 2024
Priority date	Jan 24, 2018
Publication date	Aug 15, 2024
Grant date	—

Abstract

A device displays a representation of a virtual object in a user interface region that includes a representation of a field of view of one or more cameras. The displaying includes maintaining a first spatial relationship between the representation of the virtual object and a plane detected within a physical environment that is captured in the field of view of the one or more cameras. In response to detecting movement of the computer system that adjusts the field of view of the one or more cameras, the computer system adjusts display of the representation of the virtual object in accordance with the first spatial relationship as the field of view is adjusted, and, if the movement of the computer system causes the virtual object to move outside of a displayed portion of the field of view of the one or more cameras, the computer system generates an alert.

First claim

What is claimed is:

1 . A method, comprising:
at a computer system having a display generation component, one or more input devices, and one or more cameras:
displaying, via the display generation component, a representation of a virtual object in a first user interface region that includes a representation of a field of view of one or more cameras, wherein the displaying includes maintaining a first spatial relationship between the representation of the virtual object and a plane detected within a physical environment that is captured in the field of view of the one or more cameras;
detecting movement of the computer system that adjusts the field of view of the one or more cameras; and
in response to detecting movement of the computer system that adjusts the field of view of the one or more cameras:
adjusting display of the representation of the virtual object in the first user interface region in accordance with the first spatial relationship between the virtual object and the plane detected within the field of view of the one or more cameras as the field of view of the one or more cameras is adjusted, and,
in accordance with a determination that the movement of the computer system causes more than a threshold amount of the virtual object to move outside of a displayed portion of the field of view of the one or more cameras, generating a first alert.

2 . The method of claim 1 , wherein the computer system includes one or more audio output generators, and generating the first alert includes generating, via the one or more audio output generators, a first audio alert.

3 . The method of claim 1 , including, after the movement of the computer system causes more than a threshold amount of the virtual object to move outside of the displayed portion of the field of view of the one or more cameras, generating audio associated with the virtual object.

4 . The method of claim 1 , wherein outputting the first alert includes generating an audio output that indicates an amount of the virtual object that remains visible on the displayed portion of the field of view of the one or more cameras.

5 . The method of claim 1 , wherein outputting the first alert includes generating an audio output that indicates an amount of the displayed portion of the field of view that is occluded by the virtual object.

6 . The method of claim 1 , wherein the one or more input devices include a touch-sensitive surface, and the method includes:
detecting an input by a contact at a location on the touch-sensitive surface that corresponds to the representation of the field of view of the one or more cameras; and in response to detecting the input, and in accordance with a determination that the input is detected at a first location on the touch-sensitive surface that corresponds to a first portion of the field of view of the one or more cameras that is not occupied by the virtual object, generating a second audio alert.

7 . The method of claim 1 , wherein outputting the first alert includes generating an audio output that indicates an operation that is performed with respect to the virtual object and a resulting state of the virtual object after the performance of the operation.

8 . The method of claim 7 , wherein the resulting state of the virtual object after performance of the operation is described in the audio output in the first alert in relation to a reference frame corresponding to the physical environment captured in the field of view of the one or more cameras.

9 . The method of claim 1 , including:
detecting additional movement of the computer system that further adjusts the field of view of the one or more cameras after generation of the first alert; and in response to detecting the additional movement of the computer system that further adjusts the field of view of the one or more cameras:
adjusting display of the representation of the virtual object in the first user interface region in accordance with the first spatial relationship between the virtual object and the plane detected within the field of view of the one or more cameras as the field of view of the one or more cameras is further adjusted, and,
in accordance with a determination that the additional movement of the computer system causes more than a second threshold amount of the virtual object to move into a displayed portion of the field of view of the one or more cameras, generating a second alert.

10 . The method of claim 9 , wherein the computer system includes one or more audio output generators, and generating the second alert includes generating, via the one or more audio output generators, a third audio alert.

11 . The method of claim 1 , including:
while displaying the representation of the virtual object in the first user interface region and a first object manipulation type of a plurality of object manipulation types applicable to the virtual object is currently selected for the virtual object, detecting a request to switch to another object manipulation type applicable to the virtual object; and in response to detecting the request to switch to another object manipulation type applicable to the virtual object, generating an audio output that names a second object manipulation type among a plurality of object manipulation types applicable to the virtual object, wherein the second object manipulation type is distinct from the first object manipulation type.

12 . The method of claim 11 , including:
after generating the audio output that names the second object manipulation type among the plurality of object manipulation types applicable to the virtual object, detecting a request to execute an object manipulation behavior corresponding to a currently selected object manipulation type; and in response to detecting the request to perform the object manipulation behavior corresponding to the currently selected object manipulation type, executing an object manipulation behavior that corresponds to the second object manipulation type.

13 . The method of claim 11 , wherein the one or more input devices include a touch-sensitive surface, and the method includes:
in response to detecting the request to switch to another object manipulation type applicable to the virtual object:
in accordance with a determination that the second object manipulation type is a continuously adjustable manipulation type, generating an audio alert in conjunction with the audio output naming the second object manipulation type, to indicate that the second object manipulation type is a continuously adjustable manipulation type;
detecting a request to execute an object manipulation behavior that corresponds to the second object manipulation type, including detecting a swipe input at a location on the touch-sensitive surface that corresponds to a portion of the first user interface region that displays the representation of the field of view of the one or more cameras; and
in response to detecting the request to execute the object manipulation behavior corresponding to the second object manipulation type, executing the object manipulation behavior corresponding to the second object manipulation type by an amount that corresponds to a magnitude of the swipe input.

14 . The method of claim 1 , including:
prior to displaying the representation of the virtual object in the first user interface region, displaying the representation of the virtual object in a second user interface region, wherein the second user interface region does not include a representation of the field of view of one or more cameras; while displaying the representation of the virtual object in the second user

Assignees

• Apple Inc

Inventors

• Kasemset Clare T
• Lemay Stephen O
• Cranfill Elizabeth C

Classifications

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T19/003

  Navigation within 3D models or images · CPC title