Spatial construction using guided surface detection

Patent metadata

Field	Value
Publication number	US-11580658-B2
Application number	US-202016866801-A
Country	US
Kind code	B2
Filing date	May 5, 2020
Priority date	May 24, 2018
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

Described herein are a system and methods for efficiently using depth and image information for a space to generate a 3D representation of that space. In some embodiments, an indication of one or more points is received with respect to image information, which is then mapped to corresponding points within depth information. A boundary may then be calculated to be associated with each of the points based on the depth information at, and surrounding, each point. Each of the boundaries are extended outward until junctions are identified as bounding the boundaries in a direction. The system may determine whether the process is complete or not based on whether any of the calculated boundaries are currently unlimited in extent in any direction. Once the system determines that each of the boundaries is limited in extent, a 3D representation of the space may be generated based on the identified junctions and/or boundaries.

First claim

What is claimed is:

1. A method of generating a three-dimensional (3D) representation of a space comprising:
presenting, via a user interface, an image that shows a surface of a structural feature within the space;
receiving, based on input via the user interface, an indication of a number of user-selected points, each of the user-selected points corresponding to a location upon the surface, wherein the input is received after the image is presented and indicates a user selection of the location shown in the image;
determining, for each of the number of user-selected points, a number of corresponding boundaries that match the surface for at least some amount of area, wherein the boundaries are determined after the input is received and prior to generating the 3D representation;
identifying, from the determined number of corresponding boundaries, multiple pairs of intersecting boundaries;
generating a set of junctions, wherein each junction is generated as an intersection of a pair of intersecting boundaries of the multiple pairs of intersecting boundaries; and
after determining that each of the number of corresponding boundaries are completely limited in extent by junctions within the set of junctions, generating the 3D representation of the space using the set of junctions.

2. The method of claim 1 , wherein each of the user-selected points is determined to correspond to the location upon the surface of the structural feature based on depth information received in relation to the user-selected point.

3. The method of claim 2 , wherein the depth information received in relation to the user-selected point corresponds to image information.

4. The method of claim 2 , wherein the depth information is obtained via a depth sensor installed upon a user device.

5. The method of claim 1 , wherein the 3D representation is a wireframe model of the space.

6. The method of claim 1 , wherein the 3D representation is generated by a computer aided drafting application using the set of junctions.

7. The method of claim 1 , wherein the indication of the number of user-selected points are received at a server from a user device.

8. The method of claim 1 , wherein the 3D representation generated by a mobile application server.

9. A system comprising:
one or more camera devices;
a processor; and
a memory including instructions that, when executed with the processor, cause the system to, at least:
obtain, from the one or more camera devices, depth information associated with a scene;
present, via a user interface, an image that shows a surface of a structural feature in the scene;
receive, based on input via the user interface, an indication of a user-selected point within the depth information, wherein the input is received after the image is presented and indicates a user selection of a location shown in the image and being upon the surface;
calculate, using the depth information, a first boundary associated with the user-selected point, wherein the first boundary is calculated after the input is received and prior to generating a three-dimensional (3D) representation of the scene;
determine one or more bounds for the first boundary based on at least one second boundary that is obtained in relation to the scene and that intersect with the first boundary; and
generate a 3D representation of the scene based at least in part on the one or more bounds.

10. The system of claim 9 , wherein the one or more camera devices are in a user device and wherein the processor and the memory are in a mobile application server in communication with the user device.

11. The system of claim 9 , wherein the 3D representation of the scene includes at least a representation of an atypical structural feature.

12. The system of claim 11 , wherein the atypical structural feature is one of a beam, protrusion, or depression.

13. The system of claim 9 , wherein the first boundary matches the surface of the structural feature for at least some portion of distance.

14. The system of claim 13 , wherein the second boundary snatches a second surface of the structural feature for at least some portion of distance.

15. An apparatus comprising:
a camera device configured to capture image information;
a depth sensor device configured to capture depth information;
a mobile application stored in a computer-readable medium that, when executed, causes the apparatus to, at least:
receive the depth information from the depth sensor device which corresponds to the image information captured using the camera device;
receive an indication, based on input via a user interface, of a first user-selected point and a second user-selected point within the depth information, wherein each of the first user-selected point and the second user-selected point are on one or more atypical surfaces of one or more structural features;
identify, using the depth information, a first boundary associated with the first user-selected point and a second boundary associated with the second user-selected point within the depth information;
determine a junction as a line on which the first boundary and second boundary intersect; and
cause a three-dimensional (3D) model to be generated that includes at least the determined junction.

16. The apparatus of claim 15 , wherein the apparatus further comprises a display and the indication of the first user-selected point and the second user-selected point are received as a selection of corresponding user-selected points on the display.

17. The apparatus of claim 15 , wherein the 3D model is of a bounded space that the apparatus is within.

18. The apparatus of claim 15 , wherein a position of each of the first user-selected point and the second user-selected point are indicated in relation to a single point of origin.

19. The apparatus of claim 18 , wherein the single point of origin is an initial position of the apparatus.

20. The apparatus of claim 15 , wherein the 3D model is caused to be generated by a mobile application server.

Assignees

• Lowes Companies Inc

Inventors

• Sheffield Mason E

Classifications

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T2210/04

  Architectural design, interior design · CPC title

• G06T2200/04

  involving 3D image data · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06F30/13

  Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads · CPC title