System and method for generating a progressive representation associated with surjectively mapped virtual and physical reality image data

Patent metadata

Field	Value
Publication number	US-10403043-B2
Application number	US-201715564145-A
Country	US
Kind code	B2
Filing date	Apr 14, 2017
Priority date	Apr 14, 2016
Publication date	Sep 3, 2019
Grant date	Sep 3, 2019

Abstract

A corresponding method and computer-readable device are also disclosed.

First claim

What is claimed is:

1. A system for generating a progressive representation associated with virtual and physical reality image data, the system comprising:
a wearable communication device;
an image capturing device; and
a computing device, the computing device:
receiving virtual image data associated with a virtual map of a virtual scene;
receiving physical image data from the image capturing device, the physical image data associated with a physical environment of a real scene;
determining perimeter information of the physical environment;
determining boundary information of an obstacle associated with the physical environment;
using the perimeter information and boundary information to generate a real scene map associated with the physical environment including an obstacle and one or more free space areas;
generating a corresponding virtual barrier in the real scene map, the virtual barrier associated with the boundary information of the obstacle in the real scene map;
generating a folding of the virtual scene map into the one or more free space areas of the real scene map, the free space areas surrounding or adjacent the generated virtual barrier;
performing a stratified sampling of delineated sample points of the virtual map; and
generating a progressive representation to the wearable communication device, the representation associated with virtual scene pixels of the virtual map corresponding to real scene points of the real scene map in a time interval.

2. The system as recited in claim 1 , which further comprises the computing device adding optimization constraints to the delineated sample points of the virtual map as folded in the real scene map.

3. The system as recited in claim 2 , which further comprises the computing device determining if fold-over is detected in a sampling of the virtual map.

4. The system as recited in claim 1 , in which the stratified sampling further includes a stratum.

5. The system as recited in claim 4 , in which the stratified sampling further includes a stratum comprising 0.025% pixels of the virtual map.

6. The system as recited in claim 5 , in which the stratified sampling further includes a set of stratified samples associated in which the stratified samples are located at a distance of at least 5× an average stratified sampling distance.

7. The system as recited in claim 1 , which further comprises the computing device generating a Gaussian based barrier for each obstacle associated with the physical environment.

8. The system as recited in claim 1 , which further comprises the computing device generating a progressive representation that is progressively updated in a predetermined time interval.

9. The system as recited in claim 1 , which further comprises the computing device adding a local bijective constraint to sample points of the virtual map.

10. The system as recited in claim 1 , which further comprises the computing device determining an updated physical position of a user associated with navigation of the physical environment.

11. The system as recited in claim 10 , which further comprises the computing device determining a corresponding virtual position of the user associated with the updated physical position of the user.

12. The system as recited in claim 11 , which further comprises the computing device determining a next virtual position of the user based on an estimated virtual direction of the user, the estimated virtual direction of the user being based on a Jacobian relation of respective mappings of the physical position and the virtual position of the user.

13. The system as recited in claim 11 , which further comprise the computing device progressively updating the virtual map based on a dynamic inverse mapping associated with the updated physical position of the user in the physical environment.

14. The system as recited in claim 13 , which further comprises the computing device generating progressive static representations of the virtual map based on the updated physical position of the user in the physical environment.

15. The system as recited in claim 1 , which further comprises the computing device surjectively mapping each virtual scene pixel of the virtual scene map to each real scene point of the real scene map.

16. The system as recited in claim 1 , which further comprises the computing device generating the virtual map associated with a pathway of navigation of a user, the pathway of navigation overlaying the virtual map to form a virtual navigation map.

17. The system as recited in claim 16 , which further comprises the computing device generating a folding of the virtual navigation map into the one or more free space areas of the real scene map.

18. The system as recited in claim 17 , which further comprises the computing device generating a representation associated with the virtual navigation map to the wearable communication device.

19. A method of generating a progressive representation associated with virtual and physical reality image data using a computing device, the computing device:
receiving virtual image data associated with a virtual map of a virtual scene;
receiving physical image data associated with a physical environment of a real scene;
determining perimeter information of the physical environment;
determining boundary information of an obstacle associated with the physical environment;
using the perimeter information and boundary information to generate a real scene map associated with the physical environment including an obstacle and one or more free space areas;
generating a corresponding virtual barrier in the real scene map, the virtual barrier associated with the boundary information of the obstacle in the real scene map;
generating a folding of the virtual scene map into the one or more free space areas of the real scene map, the free space areas surrounding or adjacent the generated virtual barrier;
performing a stratified sampling of delineated sample points of the virtual map; and
generating a progressive representation to the wearable communication device, the representation associated with virtual scene pixels of the virtual map corresponding to real scene points of the real scene map in a time interval.

20. The method as recited in claim 19 , further comprising adding optimization constraints to the delineated sample points of the virtual map as folded in the real scene map.

21. The method as recited in claim 20 , which further comprises determining if fold-over is detected in a sampling of the virtual map.

22. The method as recited in claim 19 , in which the stratified sampling further includes a stratum.

23. The method as recited in claim 22 , in which the stratified sampling further includes a stratum comprising 0.025% pixels of the virtual map.

24. The method as recited in claim 23 , in which the stratified sampling further includes a set of stratified samples associated in which the stratified samples are located at a distance of at least 5× an average stratified sampling distance.

25. The method as recited in claim 19 , which further comprises generating a Gaussian based barrier for each obstacle associated with the physical environment.

26. The method as recited in claim 19 , which further comprises generating a progressive representation that is progressively updated in a predetermined time interval.

27. The method as recited in claim 19 , w

Assignees

• Univ New York State Res Found

Inventors

• Kaufman Arie
• Sun Qi
• Wei Li-Yi

Classifications

• G09B9/06

  for teaching control of ships, boats, or other waterborne vehicles · CPC title

• H04N13/117

  the virtual viewpoint locations being selected by the viewers or determined by viewer tracking · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title