User controlled real object disappearance in a mixed reality display

Patent metadata

Field	Value
Publication number	US-9255813-B2
Application number	US-201113274136-A
Country	US
Kind code	B2
Filing date	Oct 14, 2011
Priority date	Oct 14, 2011
Publication date	Feb 9, 2016
Grant date	Feb 9, 2016

Abstract

The technology causes disappearance of a real object in a field of view of a see-through, mixed reality display device system based on user disappearance criteria. Image data is tracked to the real object in the field of view of the see-through display for implementing an alteration technique on the real object causing its disappearance from the display. A real object may satisfy user disappearance criteria by being associated with subject matter that the user does not wish to see or by not satisfying relevance criteria for a current subject matter of interest to the user. In some embodiments, based on a 3D model of a location of the display device system, an alteration technique may be selected for a real object based on a visibility level associated with the position within the location. Image data for alteration may be prefetched based on a location of the display device system.

First claim

What is claimed is:

1. One or more processor-readable storage devices having instructions encoded thereon for causing one or more processors to execute a method for causing a real object to be obscured in a see-through display of a see-through, mixed reality display device system, the method comprising:
receiving metadata identifying one or more real objects in a field of view of the see-through display;
determining whether any of the one or more real objects satisfies user disappearance criteria, the user disappearance criteria determinative of whether the one or more objects should be obscured and based on a current subject matter of interest, wherein the determining comprises:
identifying any real object type associated with the current subject matter of interest based on notification of the current subject matter of interest to a user, while the user is viewing the one or more real objects through the see-through display; and

responsive to determining a first real object of the one or more real objects satisfies the user disappearance criteria, tracking image data to the first real object in the see-through display for causing the first real object to be obscured in the field of view of the see-through display by selectively controlling natural light transmissivity of each pixel of a dense grid of pixels of an opacity filter of the see-through display.

2. The one or more processor-readable storage devices of claim 1 further comprising:
detecting another see-through display device system within a predetermined distance of the see-through, mixed reality display device system;
based on being within the predetermined distance of the see-through, mixed reality display device system, receiving an identifier of a second real object which satisfies user disappearance criteria of the user wearing the other see-through display device system;
receiving image data for tracking to the second real object for an alteration technique from the other see-through display device system; and
displaying the image data tracking the second real object from a perspective of the field of view of the see-through display.

3. The one or more processor-readable storage devices of claim 1 wherein determining whether any of the one or more real objects satisfies user disappearance criteria further comprises:
identifying any real object in the field of view of the see-through display matching any identified real object type; and
determining whether any identified real object does not satisfy relevance criteria for the current subject matter of interest.

4. The one or more processor-readable storage devices of claim 3 wherein determining whether any identified real object does not satisfy relevance criteria for the current subject matter of interest further comprises:
applying a keyword search technique to the metadata of any real object identified as matching any identified real object type for determining a respective keyword relevancy score; and
identifying any real object not satisfying relevance criteria for the current subject matter of interest based on the respective keyword relevancy score.

5. The one or more processor-readable storage devices of claim 1 wherein the method further comprises:
determining a position of each real object identified as satisfying user disappearance criteria within the field of view of the see-through display;
identifying a visibility level for each identified real object based on its respective determined position, and a predetermined visibility distance for a location of the display device; and
prioritizing each real object for disappearance based on its identified visibility level in the field of view.

6. The one or more processor-readable storage devices of claim 5 wherein the method further comprises:
determining a trajectory of each real object identified as satisfying user disappearance criteria within the field of view of the see-through display; and
wherein identifying the visibility level for each identified real object is also based on its respective determined trajectory.

7. The one or more processor-readable storage devices of claim 5 wherein identifying the visibility level for each identified real object is also based on at least one appearance characteristic of each real object identified as satisfying user disappearance criteria within the field of view of the see-through display.

8. The one or more processor-readable storage devices of claim 5 wherein tracking image data to the first real object in the see-through display for causing disappearance of the first real object in the field of view of the see-through display further comprises:
selecting an alteration technique for each real object to be obscured by alteration techniques indicated for the respective real object based on a priority of the visibility level identified for each of the real objects; and
applying the selected alteration technique for each real object for disappearance.

9. The one or more processor-readable storage devices of claim 1 further comprising:
checking for identification of any real object satisfying user disappearance criteria which is outside the field of view but within a predetermined visibility distance for a location of the display device system; and
prefetching any applicable disappearance image data for any identified real object satisfying user disappearance criteria which is outside the field of view but within the predetermined visibility distance for the location of the display device system.

10. The one or more processor-readable storage devices of claim 9 further comprising:
applying a location prediction method for identifying one or more subsequent locations for entry by the display device system;
responsive to identifying a subsequent location, checking for identification of any real object satisfying user disappearance criteria in the subsequent location based on stored real object metadata associated with the subsequent location; and
prefetching any applicable disappearance image data for any identified real object satisfying user disappearance criteria in the subsequent location.

11. A see-through, head mounted, mixed reality display device system for causing a real object to be obscured in a field of view of a see-through display of the display device system comprising:
an opacity filter selectively controlling natural light transmissivity of each pixel of a dense grid of pixels and blocking natural light from passing through a light guide optical element to obscure elements in the field of view;
one or more location detection sensors;
a memory for storing user disappearance criteria including at least one subject matter item to be selectively obscured from view through the see-through display;
one or more processors having access to the memory and configured to receive a notification of the at least one subject matter item while a user is viewing the real object through the see-through display, the one or more processors communicatively coupled to the one or more location detection sensors for receiving location identifier data for the display device system and for identifying one or more real objects in the field of view of the see-through display which are related to the at least one subject matter item and within a predetermined visibility distance for a location determined from the location identifier data; and
at least one image generation unit communicatively coupled to the one or more processors and optically coupled to the see-through display for tracking image data to the identified one or more real objects in the field of view of the see-through display for causing the one or more real objects to be selectively obscured

Assignees

• Liu James C
• Latta Stephen G
• Vaught Benjamin I
• Novak Christopher M
• Bennett Darren
• Microsoft Technology Licensing Llc

Inventors

• Liu James C
• Latta Stephen G
• Vaught Benjamin I
• Novak Christopher M
• Bennett Darren

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title