Spatial construction using guided surface detection

Patent metadata

Field	Value
Publication number	US-10679372-B2
Application number	US-201815988891-A
Country	US
Kind code	B2
Filing date	May 24, 2018
Priority date	May 24, 2018
Publication date	Jun 9, 2020
Grant date	Jun 9, 2020

Abstract

Described herein are a system and methods for efficiently using depth and image information for a space to generate a 3D representation of that space. In some embodiments, an indication of one or more points is received with respect to image information, which is then mapped to corresponding points within depth information. A boundary may then be calculated to be associated with each of the points based on the depth information at, and surrounding, each point. Each of the boundaries are extended outward until junctions are identified as bounding the boundaries in a direction. The system may determine whether the process is complete or not based on whether any of the calculated boundaries are currently unlimited in extent in any direction. Once the system determines that each of the boundaries is limited in extent, a 3D representation of the space may be generated based on the identified junctions and/or boundaries.

First claim

What is claimed is:

1. A method of generating a 3D representation of a space comprising:
displaying a representation of at least a portion of the space on a graphical user interface, features depicted within the representation corresponding to features within the space;
receiving, via input received from the graphical user interface, a user selection of a number of points within the representation;
corresponding each of the points within the representation to a location upon a surface of a structural feature within the space, the location being some distance inside of the boundaries of the surface;
determining, for each of the number of points, a number of corresponding boundaries that match the surface of the corresponding structural feature for at least some amount of area;
identifying, from the determined number of corresponding boundaries, multiple pairs of intersecting boundaries;
generating a set of junctions, wherein each junction is generated as an intersection of a pair of intersecting boundaries of the multiple pairs of intersecting boundaries; and
after determining that each of the number of corresponding boundaries are completely limited in extent by junctions within the set of junctions, generating the 3D representation of the space using the set of junctions.

2. The method of claim 1 , wherein each of the points are determined to correspond to the location upon the surface of the structural feature based on depth information received in relation to the point.

3. The method of claim 2 , wherein the depth information received in relation to the point corresponds to image information.

4. The method of claim 2 , wherein the depth information is obtained via a depth sensor installed upon a user device.

5. The method of claim 1 , wherein the 3D representation is a wireframe model of the space.

6. The method of claim 1 , wherein the 3D representation is generated by a computer aided drafting application using the set of junctions.

7. The method of claim 1 , wherein the indication of the number of points are received at a server from a user device.

8. A system comprising:
one or more camera devices;
a display;
a processor; and
a memory including instructions that, when executed with the processor, cause the system to, at least:
obtain, from the one or more camera devices, a depth information associated with a scene;
display, via the display, a representation of at least a portion of the scene, features depicted within the representation corresponding to features within the scene;
receive, via a user selection of a portion of the display, an indication of a number of points within the representation;
determine depth information for the number of points based on locations of the number of points and the depth information associated with the scene;
calculate, using the depth information for the number of points, a first boundary associated with the indicated point and that matches a surface boundary of a structural feature, the indicated point corresponding to a location upon the surface that is some distance inside of the boundaries of the surface;
determine one or more bounds for the first boundary based on at least one second boundary obtained in relation to the scene; and
generate a 3D representation of the scene based at least in part on the one or more bounds.

9. The system of claim 8 , wherein the one or more camera devices are in a user device and wherein the processor and memory are in a mobile application server in communication with the user device.

10. The system of claim 9 , wherein the atypical structural feature is one of a beam, protrusion, or depression.

11. The system of claim 8 , wherein the 3D representation of the scene includes at least a representation of an atypical structural feature.

12. The system of claim 8 , wherein the second boundary matches a second surface boundary of the structural feature for at least some portion of distance.

13. An apparatus comprising:
a display;
a camera device configured to capture image information depicting at least a portion of a scene;
a depth sensor device configured to capture depth information for the portion of the scene;
a mobile application stored in a computer-readable medium that, when executed, causes the apparatus to, at least:
display the image information representing at least the portion of the scene on the display;
receive depth information from the depth sensor which corresponds to the image information captured using the camera device;
receive an indication of a first point and a second point within the image information as a selection of points on the display by a user of the apparatus;
determine depth information for the first point and the second point by identifying data within the depth information corresponding to locations of the first point and the second point;
identify, using the depth information for the first point and the second point, a first boundary associated with the first point that matches a first surface boundary for some first portion of distance and a second boundary associated with the second point that matches a second surface boundary for some second portion of distance within the depth information, the first point corresponding to a location some distance inside of the first surface boundary and the second point corresponding to a location some distance inside the second surface boundary;
determine a junction as a line on which the first boundary and second boundary intersect; and
cause a 3D model to be generated that includes at least the determined junction.

14. The apparatus of claim 13 , wherein the 3D model is of a bounded space that the apparatus is within.

15. The apparatus of claim 13 , wherein the first point and the second point are on different surfaces of one or more structural features.

16. The apparatus of claim 13 , wherein a position of the first point and a position of the second point are indicated in relation to a single point of origin.

17. The apparatus of claim 13 , wherein the single point of origin is an initial position of the apparatus.

18. The apparatus of claim 13 , wherein the 3D model is caused to be generated by a mobile application server.

19. The apparatus of claim 13 , wherein the camera device is a motion tracking camera which tracks a relative position of the image information with respect to the apparatus.

Assignees

• Lowes Companies Inc

Inventors

• Sheffield Mason E

Classifications

• G06F30/17

  Mechanical parametric or variational design · CPC title

• G06T7/593 Primary

  from stereo images · CPC title

• G06F30/20

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T2210/04

  Architectural design, interior design · CPC title