Method and apparatus for selectively integrating sensory content

Patent metadata

Field	Value
Publication number	US-10068369-B2
Application number	US-201514929322-A
Country	US
Kind code	B2
Filing date	Oct 31, 2015
Priority date	Nov 4, 2014
Publication date	Sep 4, 2018
Grant date	Sep 4, 2018

Abstract

To integrate a sensory property such as occlusion, shadowing, reflection, etc. among physical and notional (e.g. virtual/augment) visual or other sensory content, providing an appearance of similar occlusion, shadowing, etc. in both models. A reference position, a physical data model representing physical entities, and a notional data model are created or accessed. A first sensory property from either data model is selected. A second sensory property is determined corresponding with the first sensory property, and notional sensory content is generated from the notional data model with the second sensory property applied thereto. The notional sensory content is outputted to the reference position with a see-through display. Consequently, notional entities may appear occluded by physical entities, physical entities may appear to cast shadows from notional light sources, etc.

First claim

We claim:

1. A machine-implemented method, comprising in a processor:
establishing a reference position;
establishing a physical data model representing at least one physical entity, said physical data model being spatially dynamic in time;
establishing a notional data model representing at least one notional entity, said notional data model being dynamic in time and non-exclusive of spatial coincidence with said physical data model;
establishing a first sensory property from said notional data model;
wherein said first sensory phenomenon represents a physical environmental phenomenon;
wherein said physical environmental phenomenon comprises at least one of physical illumination, physical shadowing, a physical volumetric effect, and an optical phenomenon; and
wherein said physical volumetric effect comprises at least one of a group consisting of at least one falling element, at least one flying element, and at least one suspended element;

determining a second sensory property, said second sensory property corresponding with said first sensory property for another of said physical and notional data models;
generating notional sensory content representing at least a portion of said notional data model with at least a portion of said second sensory property applied thereto; and
outputting to said reference position with a perceive-through display said notional sensory content, registered with said physical data model.

2. The method of claim 1 , wherein:
said first sensory property comprises at least one of a visual property and an auditory property.

3. The method of claim 1 , wherein:
said second sensory property comprises at least one of a visual property and an auditory property.

4. The method of claim 1 , wherein:
said at least one physical entity comprises at least a portion of a viewer.

5. The method of claim 1 , wherein:
said at least one notional entity comprises at least one of a virtual reality entity and an augmented reality entity.

6. The method of claim 1 , wherein:
said reference point corresponds with an eye of a viewer.

7. The method of claim 1 , wherein:
said perceive-through display is a head mounted display.

8. The method of claim 1 , wherein:
said perceive-through display is at least one of an optical see-through display such that optical physical content passes therethrough and a virtual see-through display adapted to output at least a portion of said physical data model therefrom.

9. The method of claim 1 , wherein:
said physical volumetric effect is a function of a distance between said reference point and said physical data model.

10. A machine-implemented method, comprising in a processor:
establishing a reference position;
establishing a physical data model representing at least one physical entity, said physical data model being spatially dynamic in time;
establishing a notional data model representing at least one notional entity, said notional data model being dynamic in time and non-exclusive of spatial coincidence with said physical data model;
establishing a first sensory property from said notional data model;
wherein said first sensory phenomenon represents a physical environmental phenomenon;
wherein said physical environmental phenomenon comprises at least one of physical illumination, physical shadowing, a physical volumetric effect, and an optical phenomenon; and
wherein said physical volumetric effect comprises at least one of a group consisting of ash, debris, dust, fog, gas, hail, heat distortion, insects, leaves, mist, rain, sleet, smoke, snow, spray, and steam;

determining a second sensory property, said second sensory property corresponding with said first sensory property for another of said physical and notional data models;
generating notional sensory content representing at least a portion of said notional data model with at least a portion of said second sensory property applied thereto; and

outputting to said reference position with a perceive-through display said notional sensory content, registered with said physical data model.

11. The method of claim 10 , wherein:
said physical volumetric effect is a function of a distance between said reference point and said physical data model.

12. The method of claim 10 , wherein:
said first sensory property comprises at least one of a visual property and an auditory property.

13. The method of claim 10 , wherein:
said second sensory property comprises at least one of a visual property and an auditory property.

14. The method of claim 10 , wherein:
said at least one physical entity comprises at least a portion of a viewer.

15. The method of claim 10 , wherein:
said at least one notional entity comprises at least one of a virtual reality entity and an augmented reality entity.

16. The method of claim 10 , wherein:
said reference point corresponds with an eye of a viewer.

17. The method of claim 10 , wherein:
said perceive-through display is a head mounted display.

18. The method of claim 10 , wherein:
said perceive-through display is at least one of an optical see-through display such that optical physical content passes therethrough and a virtual see-through display adapted to output at least a portion of said physical data model therefrom.

19. A machine-implemented method, comprising in a processor:
establishing a reference position;
establishing a physical data model representing at least one physical entity, said physical data model being spatially dynamic in time;
establishing a notional data model representing at least one notional entity, said notional data model being dynamic in time and non-exclusive of spatial coincidence with said physical data model;
establishing a first sensory property from said notional data model;
wherein said first sensory phenomenon represents a physical environmental phenomenon;
wherein said physical environmental phenomenon comprises at least one of physical illumination, physical shadowing, a physical volumetric effect, and an optical phenomenon; and
wherein said physical environmental phenomenon comprises at least one of a group consisting of diffraction, diffusion, glory, haloing, lens flare, and reflection;

determining a second sensory property, said second sensory property corresponding with said first sensory property for another of said physical and notional data models;
generating notional sensory content representing at least a portion of said notional data model with at least a portion of said second sensory property applied thereto; and

outputting to said reference position with a perceive-through display said notional sensory content, registered with said physical data model.

20. The method of claim 19 , wherein:
said physical volumetric effect is a function of a distance between said reference point and said physical data model.

21. The method of claim 19 , wherein:
said first sensory property comprises at least one of a visual property and an auditory property.

22. The method of claim 19 , wherein:
said second sensory property comprises at least one of a visual property and an auditory property.

23. The method of claim 19 , wherein:
said at least one physical entity comprises at least a portion of a viewer.

24. The method of claim 19 , wherein:
said at least one notional entity comprises at least one of a virtual reality entity and an augmented reality entity.

25. The m

Assignees

• Atheer Inc

Inventors

• James Greg
• Yang Allen Yang
• Itani Sleiman

Classifications

• H04N23/00

  Cameras or camera modules comprising electronic image sensors; Control thereof · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06T15/60 Primary

  Shadow generation · CPC title