Devices, methods, and graphical user interfaces for system-wide behavior for 3D models

Patent metadata

Field	Value
Publication number	US-11099707-B2
Application number	US-201816145035-A
Country	US
Kind code	B2
Filing date	Sep 27, 2018
Priority date	Jan 24, 2018
Publication date	Aug 24, 2021
Grant date	Aug 24, 2021

Abstract

A computer system with a display, a touch-sensitive surface, and one or more cameras displays a virtual object in a first user interface region. In response to detecting a first input at a location that corresponds to the virtual object and in accordance with a determination that the first input meets first criteria, the virtual object is displayed in a second user interface region. In response to detecting a second input, in accordance with a determination that the second input corresponds to a request to manipulate the virtual object in the second user interface region, a display property of the virtual object is changed, and in accordance with a determination that the second input corresponds to a request to display the virtual object in an augmented reality environment, the virtual object is displayed with a representation of a field of view of the one or more cameras.

First claim

What is claimed is:

1. A method, comprising:
at a device having a display, a touch-sensitive surface, and one or more cameras:
displaying a first representation of a virtual object in a first user interface on the display, wherein the first representation of the virtual object is a two-dimensional representation of the virtual object;
while displaying the first representation of the virtual object in the first user interface on the display, detecting a first input by a first contact at a location on the touch-sensitive surface that corresponds to the first representation of the virtual object on the display;
in response to detecting the first input by the first contact and in accordance with a determination that the first input by the first contact meets first criteria, displaying a second representation of the virtual object in a second user interface that is different from the first user interface, wherein the second representation of the virtual object is a three-dimensional representation of the virtual object;
while displaying the second representation of the virtual object in the second user interface, detecting a sequence of one or more second inputs directed to the second user interface;
in response to detecting the sequence of one or more second inputs directed to the second user interface:
in accordance with a determination that the sequence of one or more second inputs include an input that corresponds to a first request to manipulate the virtual object in the second user interface, changing a first display property of the second representation of the virtual object within the second user interface based on the sequence of one or more second inputs; and
in accordance with a determination that the sequence of one or more second inputs include an input that corresponds to a second request to manipulate the virtual object in the second user interface, changing a second display property of the second representation of the virtual object within the second user interface based on the sequence of one or more second inputs;

while displaying the second representation of the virtual object in the second user interface with an appearance based on the change in the first display property and the change in the second display property, detecting a third input directed to the second user interface; and
in response to detecting the third input directed to the second user interface:
in accordance with a determination that the third input corresponds to a request to display the virtual object in an augmented reality environment, displaying a third representation of the virtual object with a representation of a field of view of the one or more cameras, wherein:
the third representation of the virtual object is displayed with a value for the first display property that is selected based on the change in the first display property that occurred in response to the sequence of one or more second inputs;
the third representation of the virtual object is displayed with a value for the second display property that is selected independent of the change in the second display property that occurred in response to the sequence of one or more second inputs; and
the third representation of the virtual object is displayed with an appearance that concurrently has both the value for the first display property that is selected based on the change in the first display property that occurred in response to the sequence of one or more second inputs and the value for the second display property that is selected independent of the change in the second display property that occurred in response to the sequence of one or more second inputs.

2. The method of claim 1 , wherein the first criteria include criteria that are satisfied when the first input includes a tap input by the first contact at a location on the touch-sensitive surface that corresponds to a virtual object indicator.

3. The method of claim 1 , wherein the first criteria include criteria that are satisfied when the first contact is maintained at the location on the touch-sensitive surface that corresponds to the first representation of the virtual object with less than a threshold amount of movement for at least a predefined threshold amount of time.

4. The method of claim 1 , wherein:
the device includes one or more sensors to detect intensities of contacts with the touch-sensitive surface; and
the first criteria include criteria that are satisfied when a characteristic intensity of the first contact increases above a first intensity threshold.

5. The method of claim 1 , including:
in response to detecting the first input by the first contact and in accordance with a determination that the first input by the first contact meets second criteria, wherein the second criteria require that the first input includes movement of the first contact in a direction across the touch-sensitive surface for more than a threshold distance, scrolling the first user interface in a direction that corresponds to the direction of movement of the first contact.

6. The method of claim 1 , including:
in response to detecting the first input by the first contact and in accordance with a determination that the first input by the first contact meets third criteria, displaying the third representation of the virtual object with the representation of the field of view of the one or more cameras.

7. The method of claim 6 , wherein:
the device includes one or more device orientation sensors;
the method includes, in response to detecting the first input by the first contact, determining, by the one or more device orientation sensors, a current device orientation of the device; and
the third criteria require that the current device orientation be within a first range of orientations in order for the third criteria to be met.

8. The method of claim 1 , including:
in response to detecting at least an initial portion of the first input by the first contact:
activating the one or more cameras; and
analyzing the field of view of the one or more cameras to detect one or more planes in the field of view of the one or more cameras.

9. The method of claim 8 , wherein:
the device includes one or more tactile output generators; and
the method includes, in response to detecting a respective plane in the field of view of the one or more cameras, outputting, with the one or more tactile output generators, a tactile output to indicate the detection of a respective plane in the field of view of the one or more cameras.

10. The method of claim 1 , wherein a size of the third representation of the virtual object on the display is determined based on a simulated real-world size of the virtual object and a distance between the one or more cameras and a location in the field of view of the one or more cameras with which the third representation of the virtual object has a fixed spatial relationship.

11. The method of claim 1 , wherein the third input that corresponds to the request to display the virtual object in an augmented reality environment includes an input that drags the second representation of the virtual object.

12. The method of claim 1 , including, while displaying the second representation of the virtual object in the second user interface, detecting a fourth input that meets respective criteria for redisplaying the first user interface; and,
in response to detecting the fourth input:
ceasing to display the second representation of the virtual object in the second user interface; and
redisplaying the first representation of the virtual object in the first user interface.

13. The method of claim 1 , including:
while display

Assignees

• Apple Inc

Inventors

• Rockel Philipp
• Dascola Jonathan R
• Lemay Stephen O
• Faulkner Jeffrey M
• Addey David J
• Lui David
• Yerkes Giancarlo

Classifications

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06F2203/014

  Force feedback applied to GUI · CPC title

• G06F2203/04806

  Zoom, i.e. interaction techniques or interactors for controlling the zooming operation · CPC title

• G06F3/048

  Interaction techniques based on graphical user interfaces [GUI] · CPC title