Generating augmented reality exemplars

Patent metadata

Field	Value
Publication number	US-9607436-B2
Application number	US-201213879594-A
Country	US
Kind code	B2
Filing date	Aug 27, 2012
Priority date	Aug 27, 2012
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

Technologies are generally described for automatic clustering and rendering of augmentations into one or more operational exemplars in an augmented reality environment. In some examples, based on a user's context, augmentations can be retrieved, analyzed, and grouped into clusters. Exemplars can be used to render the clusters as conceptual representations of the grouped augmentations. An exemplar's rendering format can be derived from the grouped augmentations, the user's context, or formats of other exemplars. Techniques for grouping the augmentations into clusters and rendering these clusters as exemplars to a user can enhance the richness and meaning of an augmented reality environment along contextually or user-determined axes while reducing the sensorial and cognitive load on the user.

First claim

The invention claimed is:

1. A system to display augmented data, comprising:
a processor; and
a memory communicatively coupled to the processor, the memory bearing processor instructions that, when executed by the processor, cause the system to at least:
determine augmentation data based on a context associated with a user device, the augmentation data comprising a plurality of augmentations, each of the plurality of augmentations associated with one of a plurality of physical venues including a certain physical venue, and wherein the context includes information regarding the user's environment;
generate one or more clusters, each cluster of the one or more clusters comprising a subset of the plurality of augmentations, wherein each cluster of the one or more clusters represents one of the plurality of physical venues;
grouping each of the plurality of augmentations into one of the one or more clusters based on a respective location of the plurality of physical venues;
determine rendering formats for each of the one or more clusters, wherein the rendering formats determine at least a look of conceptual representations of the one or more clusters, wherein the look of the conceptual representation of a cluster is based on at least a first property of the subset of the plurality of augmentations grouped into the cluster, and wherein the conceptual representation is different from individual augmentations within the cluster, and
wherein a respective rendering format for a certain cluster of the one or more clusters associated with the certain physical venue is based on the subset of the plurality of augmentations associated with the certain physical venue, and
wherein the certain cluster of the one or more clusters comprises at least one lower-level cluster, wherein the rendering formats for the at least one lower-level cluster is based on an at least one second property of a second subset of the subset of the plurality of augmentations.

2. The system of claim 1 further comprising instructions that, when executed, cause the system to remove portions of previously transmitted augmentation data to define a subset of the previously transmitted augmentation data, and transmit the subset of the previously transmitted augmentation data, rendering formats for the subset, the generated clusters, and the rendering formats for the generated clusters.

3. The system of claim 1 , wherein the instructions to determine augmentation data comprises retrieving augmentation data from a repository, and wherein the context comprises information about physical and virtual environments associated with the user device.

4. The system of claim 1 , wherein the instructions to generate clusters comprises:
analyzing properties of the augmentation data;
generating classes of clusters based on the analyzed properties;
associating the generated classes with descriptions; and
grouping the augmentation data into the one or more clusters based on the descriptions associated with the classes of clusters.

5. The system of claim 4 , wherein the analyzing properties of the augmentation data comprises determining one or more clustering algorithms from a library of algorithms and applying the one or more clustering algorithms to the augmentation data to derive the properties and to generate the descriptions to be associated with the classes, and wherein determining the one or more clustering algorithms is based on the context and the augmentation data.

6. The system of claim 1 , wherein the instructions to determine rendering formats for the generated clusters comprises deriving properties determined for the one or more clusters based on the grouped augmentation data and using the derived properties to create the conceptual representations of the one or more clusters.

7. The system of claim 6 , wherein the conceptual representations of the generated clusters comprise visual, auditory, and haptic representations of the properties exhibited by the one or more clusters.

8. The system of claim 6 , wherein the deriving the properties comprises analyzing the grouped augmentation data to determine appearance, behavior, and interactivity properties of the grouped augmentation data.

9. The system of claim 8 , wherein the appearance properties comprise look, sound, and feel characteristics common to the augmentation data.

10. The system of claim 8 , wherein the behavior properties comprise a behavior common to the augmentation data.

11. The system of claim 8 , wherein the interactivity properties comprise an activity common to the augmentation data.

12. The system of claim 1 , wherein the rendering formats comprise a simplified representation of grouped augmentation data in a given cluster.

13. The system of claim 1 , further comprising processor instructions that, when executed, cause the system to render the certain cluster of the one or more clusters as an exemplar based on the respective rendering format; and merge the exemplar of the certain cluster with an image of a scene to generate a virtual image, wherein the exemplar includes avatars based on the subset of the plurality of augmentations.

14. The system of claim 1 , further comprising an image capture device and processor instructions that, when executed, cause the image capture device to capture data for displaying the image of the scene.

15. The system of claim 1 , wherein the scene is associated with a first set of coordinates and wherein the rendering formats are associated with a second set of coordinates, the system further comprising processor instructions that, when executed, cause the system to align the rendering formats with the image of the scene based on the first and second sets of coordinates.

16. A method to render augmented reality data on a computing device, comprising:
sending, by the computing device to an augmented reality service, a context of the computing device, wherein the context includes information regarding the user's environment;
receiving, from the augmented reality service, data representing clusters comprising one or more groupings of augmented reality data, the data generated based on the context and comprising formats for the clusters, wherein the formats are indicative of augmented reality data grouped in the clusters, and wherein each of the formats comprises sensory representations reflecting visual properties of grouped augmented reality data exhibited by each corresponding generated cluster in a form of avatars in a virtual image, wherein the sensory representations are different from any individual augmented reality data of the grouped augmented reality data in a respective cluster;
generating, via the augmented reality service, at least one lower-level cluster for at least one of the clusters based on a respective format of the formats associated with the at least one lower-level cluster, wherein the respective format comprises sensory representations reflecting visual properties of a subset of the grouped augmented reality data corresponding to the at least one of the clusters of the at least one lower-level cluster; and
rendering the clusters based on the received data.

17. The method of claim 16 further comprising:
receiving, by the computing device, additional augmented reality data omitted from the received clusters; and
rendering the received clusters and the additional augmented reality data.

18. The method of claim 16 , wherein the clusters are generated by determining a plurality of clustering algorithms based on the sent context and the augmented reality data and applying the pluralit

Assignees

• Malamud Mark
• Levien Royce
• Empire Technology Dev Llc

Inventors

• Malamud Mark
• Levien Royce

Classifications

• G06F17/3071

  Physics · mapped topic

• G06F17/3005

  Physics · mapped topic

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06K9/00671

  Physics · mapped topic

• G06V20/20

  in augmented reality scenes · CPC title