Navigation in augmented reality via a transient user interface control

Patent metadata

Field	Value
Publication number	US-10725720-B2
Application number	US-201615276734-A
Country	US
Kind code	B2
Filing date	Sep 26, 2016
Priority date	Sep 26, 2016
Publication date	Jul 28, 2020
Grant date	Jul 28, 2020

Abstract

Examples provide for intelligent navigation of informational items within an augmented reality environment. A hierarchical set of informational items within the augmented reality environment is automatically generated. A current level in the hierarchy includes items currently within a field of view (FOV), a current topic of conversation, items associated with user movements, or items selected by the user. A transient user interface control directed to the current group of informational items and a contextual breadcrumb list is generated and displayed within the FOV. An action is performed on a selected group of informational items based on input received from the user via the transient user interface control. The results of the performed action are presented to the user within the FOV.

First claim

What is claimed is:

1. An augmented reality system for generating an augmented reality environment comprising a partial real-world reality and a partial virtual reality, the augmented reality system comprising:
a memory storing a navigation engine; and
one or more processors programmed to:
identify a first item in the real-world reality within the field of view (FOV) of a user;
based at least on the identified first item, automatically infer a hierarchical set of informational items;
identify a second item in the real-world reality that is currently within the FOV of the user:
based at least on the identified first item and the identified second item, generate a contextual breadcrumb list representing a path from at least one parent node in the hierarchical set of informational items to a level within the hierarchical set of informational items comprising the second item;
present the contextual breadcrumb list and a transient user interface control for display within the FOV of a user via the augmented reality generation system, a scope of the transient user interface control being adjusted based on the level comprising the second item:
perform an action on at least one group of informational items within the hierarchical set of informational items based on input received from the user via the presented transient user interface control; and

present, for display, results of the performed action.

2. The augmented reality system of claim 1 , wherein the action is a scoped sort action, wherein the one or more processors are programmed to perform the scoped sort action on the at least one group of informational items based on at least one sort criteria.

3. The augmented reality system of claim 1 , wherein the hierarchical list includes real world items previously seen in the FOV.

4. The augmented reality system of claim 1 , wherein the action is a scoped search action performed on the at least one group of informational items, wherein the one or more processors are programmed to perform a search within the at least one group of informational items based on a t least one search criteria.

5. The augmented reality system of claim 1 , wherein the processor is further programmed to automatically detect the level based on one or more items visible within the FOV of the user.

6. The augmented reality system of claim 1 , wherein the processor is further programmed to:
automatically detect a new level in the hierarchical set of informational items in response to the user moving through the augmented reality environment;
update the contextual breadcrumb list, wherein the updated contextual breadcrumb list represents a path from at least one parent node in the hierarchical set of informational items to the new level; and
display the updated contextual breadcrumb list and the transient user interface control within the FOV of the user.

7. The augmented reality system of claim 1 , further comprising:
a remote computing device and a wearable augmented reality device, wherein the remote computing device generates the contextual breadcrumb list based on the input received from the wearable augmented reality device, and wherein the remote computing device sends the contextual breadcrumb list to the wearable augmented reality device via a network for display to the user.

8. A method for generating an augmented reality environment comprising a partial real-world reality and a partial virtual reality, the method comprising:
automatically identifying, by at least one processor, a first item among a current group of informational items within a current field of view (FOV) of a user and associated with the augmented reality environment, the augmented reality environment comprising a plurality of groups of informational items, wherein the identified first item is in the real-world reality that is currently within the FOV of the user;
based at least on the identified first item, automatically inferring a hierarchical set of informational items;
identifying a second item in the real-world reality that is currently within the FOV of the user:
generating a contextual breadcrumb list representing a path from at least one parent node in the hierarchical set of informational items to a level detected within the hierarchical set of informational items comprising the second item;
presenting the contextual breadcrumb list and a transient user interface control for display within the FOV of the user by the augmented reality device;
performing an action, by a task component, on the second item based on input received from the user via the transient user interface control; and
presenting a result of the performed action to the user.

9. The method of claim 8 , wherein the action is a transient sort and further comprising:
receiving the input indicating the user selection of a sort control associated with the transient user interface control; and
sorting items within the detected current group of informational items based on at least one sort criteria, wherein items within the current group of informational items are sorted, and wherein items within the plurality of groups of informational items that are excluded from the current group of informational items remain unsorted.

10. The method of claim 9 , wherein the action is a transient filter and further comprising:
receiving the input indicating the user selection of a filter control associated with the transient user interface control; and
filtering the detected current group of informational items based on at least one criteria, wherein items within the plurality of groups of informational items that are excluded from the current group of informational items are excluded from the filtering.

11. The method of claim 8 , wherein the action is a transient search and further comprising:
receiving the input indicating the user selection of a search control associated with the transient user interface control; and
performing a search on the detected current group of informational items based on at least one search criteria, wherein items within the plurality of groups of informational items that are excluded from the current group of informational items are excluded from the search.

12. The method of claim 8 , wherein the plurality of groups of informational items are items in the hierarchical set of informational items, and further comprising:
automatically detecting a change in a context of the plurality of groups of informational items, wherein the change in context comprises a user walking through the augmented reality environment.

13. The method of claim 8 , further comprising:
retrieving a contextual map associated with at least a portion of the augmented reality environment associated with a current location of the user, wherein the contextual map comprises an indicator identifying the current location of the user relative to at least one feature of interest within the portion of the augmented reality environment.

14. The method of claim 8 , further comprising:
automatically detect a new current level in the hierarchical set of informational items in response to a user movement changing the FOV of the user to include a different portion of the hierarchical set of informational items; and
present an updated transient user interface control directed to the new current level within the FOV to the user.

15. One or more computer storage media embodying computer-executable components generating an augmented reality environment comprising a partial real-world reality and a partial virtual reality, said components comprising:
a navigation engine component that when executed,

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Cheung Chairy Chiu Ying
• Mallya Raghunath

Classifications

• G06F3/147 Primary

  using display panels · CPC title

• G09G2370/16

  Use of wireless transmission of display information · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06F3/14 Primary

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title