2D obstacle boundary detection

Patent metadata

Field	Value
Publication number	US-11087471-B2
Application number	US-202016917255-A
Country	US
Kind code	B2
Filing date	Jun 30, 2020
Priority date	Apr 19, 2019
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

Techniques are provided to dynamically generate and render an object bounding fence in a mixed-reality scene. Initially, a sparse spatial mapping is accessed. The sparse spatial mapping beneficially includes perimeter edge data describing an object's edge perimeters. A gravity vector is also generated. Based on the perimeter edge data and the gravity vector, two-dimensional (2D) boundaries of the object are determined and a bounding fence mesh of the environment is generated. A virtual object is then rendered, where the virtual object is representative of at least a portion of the bounding fence mesh and visually illustrates a bounding fence around the object.

First claim

What is claimed is:

1. A method for using a low-resolution spatial mapping of an environment to facilitate obstacle avoidance, said method comprising:
accessing a low-resolution spatial mapping of an environment, said low-resolution spatial mapping comprising incomplete three-dimensional (3D) data describing an object located in the environment such that the object is partially, but not completely, defined by the low-resolution spatial mapping;
identifying perimeter edge data for the object in the low-resolution spatial mapping, said perimeter edge data defining outer boundaries of the object;
generating, based on the identified perimeter edge data, two-dimensional (2D) boundaries for the object; and
visually displaying the 2D boundaries as a collision warning for the object.

2. The method of claim 1 , wherein the low-resolution spatial mapping is distinct from a dense spatial mapping that comprises complete 3D data describing the object such that the object is completely defined by the dense spatial mapping, and wherein a level of detail of the object provided by the incomplete 3D data is less than 70% of a level of detail of the object provided by the complete 3D data.

3. The method of claim 2 , wherein the level of detail of the object provided by the incomplete 3D data is less than 50% of the level of detail of the object provided by the complete 3D data.

4. The method of claim 3 , wherein the level of detail of the object provided by the incomplete 3D data is less than 30% of the level of detail of the object provided by the complete 3D data.

5. The method of claim 1 , wherein the low-resolution spatial mapping is generated based on low-resolution images generated by one or more low-resolution cameras or, alternatively, generated by one or more cameras of a mobile phone.

6. The method of claim 1 , wherein the low-resolution spatial mapping is generated based on low-resolution images generated by a passive stereo camera pair.

7. The method of claim 1 , wherein the low-resolution spatial mapping is generated based on low-resolution images generated by an active stereo camera system operating at a reduced power level relative to a default power level.

8. The method of claim 1 , wherein the low-resolution spatial mapping is generated based on low-resolution images generated by an active stereo camera system operating using a reduced scanning duration relative to a default scanning duration.

9. The method of claim 1 , wherein generating the 2D boundaries is performed by removing a height dimension along a gravity vector from the incomplete 3D data.

10. The method of claim 1 , wherein the low-resolution spatial mapping is generated based on low-resolution images generated by a motion stereo camera system.

11. A computer system configured to use a low-resolution spatial mapping of an environment to facilitate obstacle avoidance, said computer system comprising:
one or more processors; and
one or more computer-readable hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to at least:
access a low-resolution spatial mapping of an environment, said low-resolution spatial mapping comprising incomplete three-dimensional (3D) data describing an object located in the environment such that the object is partially, but not completely, defined by the low-resolution spatial mapping;
identify perimeter edge data for the object in the low-resolution spatial mapping, said perimeter edge data defining outer boundaries of the object;
generate, based on the identified perimeter edge data, two-dimensional (2D) boundaries for the object; and
visually display the 2D boundaries as a collision warning for the object.

12. The computer system of claim 11 , wherein a gravity vector is computed using inertial measurement unit (IMU) data and/or gyroscope data, and wherein the 2D boundaries are generated relative to the gravity vector.

13. The computer system of claim 11 , wherein generating the 2D boundaries is performed by translating the incomplete 3D data of the object into 2D data.

14. The computer system of claim 11 , wherein generating the 2D boundaries is performed by generating a 2D ground plane of the environment and translating the incomplete 3D data onto the 2D ground plane.

15. The computer system of claim 14 , wherein execution of the instructions further causes the computer system to:
for each pixel defining the 2D ground plane, determine whether said each pixel corresponds to a free space that is not occupied by any objects in the environment or an occupied space that is occupied by a corresponding object in the environment.

16. The computer system of claim 15 , wherein a binary value is stored for said each pixel to indicate whether said each pixel corresponds to the free space or the occupied space.

17. The computer system of claim 11 , wherein the low-resolution spatial mapping is generated based on images generated by a head tracking camera system.

18. The computer system of claim 11 , wherein visually displaying the 2D boundaries as the collision warning is triggered in response to determining the computer system is within a threshold distance of the object in the environment.

19. The computer system of claim 11 , wherein a group of particular 2D boundaries included in said 2D boundaries are merged together to form a single merged 2D boundary, and wherein visually displaying the 2D boundaries includes visually displaying the single merged 2D boundary instead of visually displaying each particular 2D boundary included in the group of particular 2D boundaries.

20. A head-mounted device (HMD) configured to use a low-resolution spatial mapping of an environment to facilitate obstacle avoidance, said HMD comprising:
a display;
one or more processors; and
one or more computer-readable hardware storage devices that store instructions that are executable by the one or more processors to cause the HMD to at least:
access a low-resolution spatial mapping of an environment, said low-resolution spatial mapping comprising incomplete three-dimensional (3D) data describing an object located in the environment such that the object is partially, but not completely, defined by the low-resolution spatial mapping;
identify perimeter edge data for the object in the low-resolution spatial mapping, said perimeter edge data defining outer boundaries of the object;
generate, based on the identified perimeter edge data, two-dimensional (2D) boundaries for the object relative to a gravity vector; and
visually display, on the HMD's display, the 2D boundaries as a collision warning for the object.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Bleyer Michael
• Pekelny Yuri
• Price Raymond Kirk

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06T2210/12

  Bounding box · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06T2207/10012

  Stereo images · CPC title