Virtual object pathing

Patent metadata

Field	Value
Publication number	US-9824500-B2
Application number	US-201615072192-A
Country	US
Kind code	B2
Filing date	Mar 16, 2016
Priority date	Mar 16, 2016
Publication date	Nov 21, 2017
Grant date	Nov 21, 2017

Abstract

Examples are disclosed for controlling virtual object placement and movement in a physical space as viewed on or through a display. An example method includes receiving a count indicating a number of occupancy transitions over time for a plurality of regions in the physical space, displaying a virtual object in an initial location selected based on the counts for each of the plurality of regions in the physical space, and determining an updated location of the virtual object in the physical space as viewed on or through the display based at least on the initial location of the virtual object and the counts for each of the plurality regions in the physical space. The example method further includes displaying the virtual object in the updated location, the virtual object being controlled to move in the physical space based at least on movements of physical objects through the physical space.

First claim

The invention claimed is:

1. A method of controlling virtual object placement in a physical space as viewed on or through a display, the method comprising:
receiving a count indicating a number of occupancy transitions over a period of time for each of a plurality of regions in the physical space;
displaying the virtual object in an initial location, the initial location selected based on the counts for each of the plurality of regions in the physical space;
determining an updated location of the virtual object in the physical space as viewed on or through the display, the updated location of the virtual object being based at least on the initial location of the virtual object in the physical space and the counts for each of the plurality regions in the physical space; and
displaying the virtual object in the updated location of the physical space as viewed on or through the display, the virtual object being controlled to move in the physical space based at least on movements of physical objects through the physical space.

2. The method of claim 1 , wherein the count is generated based on imaging the physical space with one or more depth cameras over at least the period of time.

3. The method of claim 2 , further comprising generating a point cloud, wherein the count for each of the plurality of regions is stored in association with a different three-dimensional point in the physical space.

4. The method of claim 2 , wherein the plurality of regions in the physical space are mapped to a plurality of voxels, and wherein the count for each of the plurality of regions is associated with a different voxel of the plurality of voxels.

5. The method of claim 1 , wherein the display comprises a see-through display of a head-mounted display device.

6. The method of claim 1 , further comprising determining a rate of change of occupancy transitions for each of the plurality of regions in the physical space and determining a velocity of movement through the regions in the physical space based on the rate of change.

7. The method of claim 6 , wherein the updated location of the virtual object is further based at least on one or more of a velocity of movement associated with the virtual object and a type of the virtual object, one or more of the velocity of movement associated with the virtual object and the type of the virtual object being mapped to an associated rate of change of occupancy transitions for the plurality of regions in the physical space.

8. The method of claim 1 , further comprising determining a path through the physical space, the path comprising a subset of the plurality of regions in the physical space, where each region in the subset is adjacent to at least one other region in the subset, and where each region in the subset is associated with a substantially equal number of occupancy transitions.

9. The method of claim 8 , wherein the updated location of the virtual object is positioned on the path through the physical space, the method further comprising determining one or more subsequent updated locations of the virtual object positioned along the path through the physical space.

10. The method of claim 1 , further comprising, for each of the plurality of regions, determining a score for that region based at least on the count associated with that region, the score indicating whether the count associated with that region is above a predetermined threshold.

11. A head-mounted display device comprising:
a near-eye display;
a logic device; and
a storage device holding instructions executable by the logic device to:
receive a count indicating a number of occupancy transitions over a period of time for each of a plurality of regions in a physical space;
for each of the plurality of regions, determine a score for that region based at least on the count associated with that region and a predetermined threshold;
display a virtual object in an initial location, the initial location selected based on the scores for each of the plurality of regions in the physical space;
determine an updated location of the virtual object in the physical space as viewed on or through the display, the updated location of the virtual object being based at least on the initial location of the virtual object in the physical space and the scores of each of the plurality regions in the physical space; and
display the virtual object in the updated location of the physical space as viewed on or through the display.

12. The head-mounted display device of claim 11 , the head-mounted display device further comprising a depth camera, wherein receiving the count comprises capturing a plurality of depth images of the physical space using the depth camera over the period of time and tracking the number of occupancy transitions for each of a plurality of voxels of the plurality of depth images.

13. The head-mounted display device of claim 12 , wherein the instructions are further executable to associate each of the plurality of voxels of each of the plurality of depth images with an associated three-dimensional point in the physical space.

14. The head-mounted display device of claim 11 , the instructions further executable to determine a rate of change of occupancy transitions for each of the plurality of regions in the physical space and determine a velocity of movement through the regions in the physical space based on the rate of change.

15. The head-mounted display device of claim 14 , wherein the updated location of the virtual object is further based at least on one or more of a velocity of movement associated with the virtual object and a type of the virtual object, one or more of the velocity of movement associated with the virtual object and the type of the virtual object being mapped to an associated rate of change of occupancy transitions for the plurality of regions in the physical space.

16. The head-mounted display device of claim 11 , wherein the instructions are further executable to determine a path through the physical space, the path comprising a subset of the plurality of regions in the physical space, where each region in the subset is adjacent to at least one other region in the subset, and where each region in the subset is associated with a substantially equal number of occupancy transitions.

17. The head-mounted display device of claim 16 , wherein the updated location of the virtual object is positioned on the path through the physical space, the method further comprising determining one or more subsequent updated locations of the virtual object positioned along the path through the physical space.

18. The head-mounted display device of claim 11 , wherein receiving the count comprises one or more of 1) receiving the count from an external computing device and 2) receiving a plurality of depth images from an external depth camera and determining the count based at least on the plurality of depth images.

19. A method of controlling virtual object placement in a physical space as viewed on or through a display, the method comprising:
monitoring the physical space over a period of time to determine a count indicating a number of occupancy transitions over a period of time for each of a plurality of three-dimensional points in the physical space;
determining one or more paths of movement through the physical space, each path of movement comprising a subset of the plurality of three-dimensional points in the physical space, each three-dimensional point included in the subset being adjacent to at least one other three-dimensional point of the subset,

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Schoenberg Michael John

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06T7/004

  Physics · mapped topic

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06T2200/04

  involving 3D image data · CPC title