Displaying Representations of Environments

Patent metadata

Field	Value
Publication number	US-2024345697-A1
Application number	US-202418750769-A
Country	US
Kind code	A1
Filing date	Jun 21, 2024
Priority date	Sep 25, 2019
Publication date	Oct 17, 2024
Grant date	—

Abstract

A method includes displaying diorama-view representations from corresponding viewing vectors. The diorama-view representations respectively correspond to enhanced reality (ER) environments. Each of the diorama-view representations is associated with a respective set of ER world coordinates that characterizes a respective ER environment. The diorama-view representations include a first diorama-view representation displayed from a first viewing vector. The first diorama-view representation includes a first one or more of ER objects arranged according to a first set of ER world coordinates. The method includes detecting an input associated with the first diorama-view representation, and in response thereto, changing display of the first diorama-view representation from the first viewing vector to a second viewing vector while maintaining the first one or more ER objects arranged according to the first set of ER world coordinates.

First claim

What is claimed is:

1 . A method comprising:
at an electronic device including one or more processors, a non-transitory memory, one or more input devices, and a display device:
displaying, via the display device, a plurality of diorama-view representations from a corresponding plurality of viewing vectors, wherein the plurality of
diorama-view representations corresponds to a plurality of enhanced reality (ER) environments, wherein each of the plurality of diorama-view representations is associated with a respective set of ER world coordinates that characterizes a respective ER environment, wherein the plurality of diorama-view representations includes a first one of the plurality of diorama-view representations displayed from a first viewing vector, and wherein the first one of the plurality of diorama-view representations includes a first one or more of ER objects arranged according to a first set of ER world coordinates;
detecting, via the one or more input devices, an input associated with the first one of the plurality of diorama-view representations; and
in response to detecting the input, changing display of the first one of the plurality of diorama-view representations from the first viewing vector to a second viewing vector while maintaining the first one or more ER objects arranged according to the first set of ER world coordinates.

2 . The method of claim 1 , wherein the input is directed to the first one of the plurality of diorama-view representations.

3 . The method of claim 2 , wherein the one or more input devices includes a hand tracking sensor, the method further comprising:
detecting the input via the hand tracking sensor; obtaining hand tracking data from the hand tracking sensor based on the input; and determining, from the hand tracking data, that the input is directed to the first one of the plurality of diorama-view representations.

4 . The method of claim 2 , wherein the one or more input devices includes an eye tracking sensor, the method further comprising:
detecting the input via the eye tracking sensor; obtaining eye tracking data from the eye tracking sensor based on the input; and determining, from the eye tracking data, that the input is directed to the first one of the plurality of diorama-view representations.

5 . The method of claim 1 , wherein the input corresponds to a change in position of the electronic device from a first pose to a second pose relative to the first one of the plurality of diorama-view representations.

6 . The method of claim 5 , wherein the one or more input devices includes a positional-change sensor, the method further comprising:
detecting the input via the positional-change sensor; obtaining positional-change data from the positional-change sensor based on the input; and determining the change in position of the electronic device based on the positional-change data.

7 . The method of claim 1 , wherein the input is also associated with a second one of the plurality of diorama-view representations, and wherein the second one of the plurality of diorama-view representations is displayed from a third viewing vector, and wherein the second one of the plurality of diorama-view representations includes a second one or more of ER objects arranged according to a second set of ER world coordinates, the method further comprising, in response to detecting the input, changing display of the second one of the plurality of diorama-view representations from the third viewing vector to a fourth viewing vector while maintaining the second one or more ER objects arranged according to the second set of ER world coordinates.

8 . The method of claim 1 , wherein the plurality of diorama-view representations corresponds to reduced-sized representations of the corresponding plurality of ER environments.

9 . The method of claim 1 , wherein the first one of the plurality of diorama-view representations corresponds to a virtual reality (VR) representation of a first ER environment.

10 . The method of claim 1 , wherein the first one of the plurality of diorama-view representations corresponds to an augmented reality (AR) representation of a first ER environment, and wherein the first one of the plurality of diorama-view representations includes AR content overlaid on environmental data that is associated with physical features of the first ER environment.

11 . The method of claim 1 , wherein displaying the plurality of diorama-view representations includes animating a portion of the plurality of diorama-view representations.

12 . The method of claim 1 , further comprising:
obtaining a plurality of characterization vectors that respectively provide a plurality of spatial characterizations of the corresponding plurality of ER environments, wherein each of the plurality of characterization vectors includes a plurality of object label values that respectively identify one or more ER objects, and wherein each of the plurality of characterization vectors also includes a plurality of relative position values providing respective positions of the one or more ER objects relative to each other; and generating, from the corresponding plurality of ER environments and the plurality of characterization vectors, the plurality of diorama-view representations of the corresponding plurality of ER environments according to the plurality of relative position values.

13 . The method of claim 12 , wherein generating the plurality of diorama-view representations includes scaling down the corresponding plurality of ER environments by a scaling factor.

14 . The method of claim 13 , further comprising obtaining, via the one or more input devices, a scaling request input that specifies the scaling factor.

15 . The method of claim 13 , wherein the electronic device includes an image sensor, the method further comprising:
obtaining, via the image sensor, pass-through image data bounded by a field-of-view of a physical environment associated with the image sensor; identifying, within the pass-through image data, one or more physical objects within the physical environment; and determining the scaling factor based on the one or more physical objects.

16 . The method of claim 1 , wherein at least a subset of the corresponding plurality of ER environments is respectively associated with a plurality of ER environments corresponding to a plurality of ER sessions that are distinct from each other, and wherein each of the plurality of ER environments enables graphical representations of individuals to be concurrently within the ER environment.

17 . The method of claim 16 , wherein each of at least a subset of the plurality of diorama-view representations corresponding to at least the subset of the corresponding plurality of ER environments includes one or more ER representations of one or more corresponding individuals that are associated with a respective ER session.

18 . The method of claim 17 , wherein each of the one or more corresponding individuals is associated with a respective access level that satisfies an access level criterion that is associated with the respective ER session.

19 . The method of claim 16 , further comprising:
playing, via a speaker of the electronic device, a first set of speech data while displaying the plurality of diorama-view representations, wherein the first set of speech data is associated with one or more corresponding individuals that are associated with a particular ER session associated with a respective ER environment; obtaining, via an audio s

Assignees

• Apple Inc

Inventors

• Sommer Bruno M
• Morgan Tucker Bull
• Chiu Shih Sang
• Smith Connor Alexander

Classifications

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title