Systems and methods for virtual and augmented reality

Patent metadata

Field	Value
Publication number	US-2021304509-A1
Application number	US-202117211502-A
Country	US
Kind code	A1
Filing date	Mar 24, 2021
Priority date	Mar 24, 2020
Publication date	Sep 30, 2021
Grant date	—

Abstract

An apparatus for providing a virtual or augmented reality experience, includes: a screen, wherein the screen is at least partially transparent for allowing a user of the apparatus to view an object in an environment surrounding the user; a surface detector configured to detect a surface of the object; an object identifier configured to obtain an orientation and/or an elevation of the surface of the object, and to make an identification for the object based on the orientation and/or the elevation of the surface of the object; and a graphic generator configured to generate an identifier indicating the identification for the object for display by the screen, wherein the screen is configured to display the identifier.

First claim

1 . An apparatus for providing a virtual or augmented reality experience, comprising:
a screen, wherein the screen is at least partially transparent for allowing a user of the apparatus to view an object in an environment surrounding the user; a surface detector configured to detect a surface of the object; an object identifier configured to obtain an orientation and/or an elevation of the surface of the object, and to make an identification for the object based on the orientation and/or the elevation of the surface of the object; and a graphic generator configured to generate an identifier indicating the identification for the object for display by the screen, wherein the screen is configured to display the identifier.

2 . The apparatus of claim 1 , wherein the object identifier is configured to identify the object as a wall if the orientation of the surface of the object is substantially vertical, and if the elevation of the surface of the object is above an elevation threshold.

3 . The apparatus of claim 1 , wherein the object identifier is configured to identify the object as a floor if the orientation of the surface of the object is substantially horizontal, and if the elevation of the surface of the object is below an elevation threshold.

4 . The apparatus of claim 1 , wherein the object is configured to identify the object as a furniture if the orientation of the surface of the object is substantially horizontal, and if the elevation of the surface of the object is above a first elevation threshold and is below a second elevation threshold.

5 . The apparatus of claim 1 , wherein the screen is configured to display the identifier at a location in the screen, such that when the user views the object in the environment through the screen, the identifier will be in a spatial relationship with respect to the object.

6 . The apparatus of claim 1 , wherein the object identifier is configured to obtain an input indicating a selection of the object for which the identification of the object is to be determined.

7 . The apparatus of claim 6 , wherein the input comprises a user input generated via a controller component, the user input indicating the selection of the object.

8 . The apparatus of claim 7 , wherein the user input indicates a cursor position in the screen, and wherein the object identifier is configured to determine the object in the environment being selected based on the cursor position.

9 . The apparatus of claim 7 , wherein the user input indicates an orientation of the controller component, and wherein the object identifier is configured to determine the object in the environment being selected based on a direction of pointing by the controller component towards the object in the environment.

10 . The apparatus of claim 1 , further comprising a camera, wherein the object identifier is configured to select the object for identification based on a presence of an image of the object in a camera image provided by the camera.

11 . The apparatus of claim 10 , wherein the object identifier is configured to select the object automatically.

12 . The apparatus of claim 10 , wherein the object identifier is configured to select the object in response to the object being presence in a sequence of camera images that comprise the camera image within a duration exceeding a time threshold.

13 . The apparatus of claim 1 , further comprising a space definer configured to define a virtual space.

14 . The apparatus of claim 13 , wherein the space definer is configured to define a virtual wall for the virtual space.

15 . The apparatus of claim 14 , wherein the virtual wall is offset from a real physical wall in the environment surrounding the user.

16 . The apparatus of claim 14 , wherein the virtual wall is aligned with, or intersects, a real physical wall in the environment surrounding the user.

17 . The apparatus of claim 14 , wherein the screen is configured to display a wall identifier at a location in the screen, such that when the user views the virtual wall, the wall identifier will be in a spatial relationship with respect to the virtual wall.

18 . The apparatus of claim 13 , wherein the space definer is configured to define a plurality of virtual walls for the virtual space; and
wherein the screen is configured to display wall identifiers for the respective virtual walls.

19 . The apparatus of claim 13 , wherein the space definer is configured to define a corner for the virtual space.

20 . The apparatus of claim 13 , wherein the space definer is configured to define a wall edge for the virtual space.

21 . The apparatus of claim 13 , wherein the space definer is configured to obtain a user input generated via a controller component, the user input indicating a selection of a feature in the environment for defining at least a part of the virtual space.

22 . The apparatus of claim 21 , wherein the feature in the environment comprises a wall, a wall corner, an edge, or any combination of the foregoing.

23 . The apparatus of claim 21 , wherein the user input indicates a cursor position in the screen.

24 . The apparatus of claim 21 , wherein the user input indicates an orientation of the controller component, and wherein the selection of the feature in the environment is based on a direction of pointing by the controller component towards the feature in the environment.

25 . The apparatus of claim 13 , further comprising a camera, wherein the apparatus is configured to select a feature in the environment, for defining at least a part of the virtual space, based on a presence of an image of the feature in a camera image provided by the camera.

26 . The apparatus of claim 25 , wherein the apparatus is configured to select the feature in the environment automatically.

27 . The apparatus of claim 25 , wherein the apparatus is configured to select the feature in response to the feature being presence in a sequence of camera images that includes the camera image within a duration exceeding a time threshold.

28 . The apparatus of claim 1 , wherein the screen is configured to display a pointer based on an orientation of a controller component, and wherein the apparatus further comprises a pointer configurator, wherein the pointer configurator is configured to adjust a configuration of the pointer displayed in the screen, and/or to present an interaction graphic in association with the pointer.

29 . The apparatus of claim 28 , wherein the pointer configurator is configured to adjust the configuration of the pointer and/or to present the interaction graphic in response to an interaction between the pointer and a virtual object.

30 . The apparatus of claim 1 , wherein the graphic generator is also configured to provide a virtual content for interaction by the user and an additional user, and wherein the screen is configured to display the virtual content.

31 . The apparatus of claim 30 , wherein the apparatus is configured to connect the user and the additional user to a same virtual space so that the user and the additional user can interact with the virtual content at the same virtual space.

32 . The apparatus of claim 30 , wherein the graphic generator is configured to provide the virtual content for interaction by the user and the additional user

Assignees

• Magic Leap Inc

Inventors

• Berkebile Robert David

Classifications

• G06V40/20 Primary

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title