Method and apparatus for constructing a 3D geometry

Patent metadata

Field	Value
Publication number	US-11670045-B2
Application number	US-202117502973-A
Country	US
Kind code	B2
Filing date	Oct 15, 2021
Priority date	May 7, 2021
Publication date	Jun 6, 2023
Grant date	Jun 6, 2023

Abstract

Aspects of the disclosure include methods, apparatuses, and non-transitory computer-readable storage mediums for generating a three-dimensional (3D) geometry of a room from a panorama image of the room. An apparatus includes processing circuitry that determine two-dimensional (2D) positions of wall corner points of the room in the panorama image based on a user input. Each wall corner point is in one of a floor plane or a ceiling plane of the room. The processing circuitry calculates 3D positions of the wall corner points based on the 2D positions of the wall corner points, a size of the panorama image, and a distance between the floor plane and a capture position of a device capturing the panorama image, determines a room layout based on an order of the wall corner points, and generates the 3D geometry based on the room layout and the 3D positions of the wall corner points.

First claim

What is claimed is:

1. A method of generating a three-dimensional (3D) representation of a room from a panorama image of the room, the method comprising:
determining two-dimensional (2D) positions of wall corner points of the room in the panorama image of the room based on a user selection of the wall corner points captured in the panorama image, each of the wall corner points being in one of a floor plane or a ceiling plane of the room;
calculating 3D positions of the wall corner points based on the 2D positions of the wall corner points, a size of the panorama image, and a distance between the floor plane of the room and a capture position of a device configured to capture the panorama image of the room;
determining a layout of the room based on an order of the wall corner points; and
generating the 3D representation of the room based on the layout of the room and the 3D positions of the wall corner points.

2. The method of claim 1 , wherein the determining the 2D positions of the wall corner points comprises:
determining the 2D positions of the wall corner points of the room in the panorama image of the room based on an order in which the wall corner points are selected.

3. The method of claim 1 , wherein at least one of the wall corner points is a first type of wall corner point, the first type of wall corner point indicating a wall plane of the 3D representation.

4. The method of claim 1 , wherein at least one of the wall corner points is a second type of wall corner point, the second type of wall corner point indicating an open area plane of the 3D representation.

5. The method of claim 1 , wherein the generating comprises:
generating a plane of the 3D representation based on a type of a predetermined one of two adjacent wall corner points.

6. The method of claim 1 , further comprising:
determining, for each 3D position in a plane of the 3D representation, color information of the respective 3D position based on color information at a 2D position in the panorama image of the room corresponding to the respective 3D position.

7. The method of claim 1 , wherein
each wall plane of the 3D representation is parallel or orthogonal to at least one other wall plane of the 3D representation, and
the method further includes generating a guide line that assists a user to select one of the wall corner points.

8. The method of claim 1 , wherein
each wall plane of the 3D representation is parallel or orthogonal to at least one other wall plane of the 3D representation, and
the method further includes adjusting one of the wall corner points that is selected by the user.

9. The method of claim 1 , further comprising:
determining 2D positions of two points in the panorama image of the room;
calculating 3D positions of the two points based on the 2D positions of the two points, the size of the panorama image, and the distance between the floor plane of the room and the capture position of the device; and
calculating a distance between the 3D positions of the two points.

10. An apparatus, comprising:
processing circuitry configured to:
determine two-dimensional (2D) positions of wall corner points of a room in a panorama image of the room based on a user selection of the wall corner points captured in the panorama image, each of the wall corner points being in one of a floor plane or a ceiling plane of the room;
calculate three-dimensional (3D) positions of the wall corner points based on the 2D positions of the wall corner points, a size of the panorama image, and a distance between the floor plane of the room and a capture position of a device configured to capture the panorama image of the room;
determine a layout of the room based on an order of the wall corner points; and
generate a 3D representation of the room based on the layout of the room and the 3D positions of the wall corner points.

11. The apparatus of claim 10 , wherein the processing circuitry is configured to:
determine the 2D positions of the wall corner points of the room in the panorama image of the room based on an order in which the wall corner points are selected.

12. The apparatus of claim 10 , wherein at least one of the wall corner points is a first type of wall corner point, the first type of wall corner point indicating a wall plane of the 3D representation.

13. The apparatus of claim 10 , wherein at least one of the wall corner points is a second type of wall corner point, the second type of wall corner point indicating an open area plane of the 3D representation.

14. The apparatus of claim 10 , wherein the processing circuitry is further configured to:
generate a plane of the 3D representation based on a type of a predetermined one of two adjacent wall corner points.

15. The apparatus of claim 10 , wherein the processing circuitry is further configured to:
determine, for each 3D position in a plane of the 3D representation, color information of the respective 3D position based on color information at a 2D position in the panorama image of the room corresponding to the respective 3D position.

16. The apparatus of claim 10 , wherein
each wall plane of the 3D representation is parallel or orthogonal to at least one other wall plane of the 3D representation, and
the processing circuitry is further configured to:
generate a guide line that assists a user to select one of the wall corner points.

17. The apparatus of claim 10 , wherein
each wall plane of the 3D representation is parallel or orthogonal to at least one other wall plane of the 3D representation, and
the processing circuitry is further configured to:
adjust one of the wall corner points that is selected by the user.

18. The apparatus of claim 10 , wherein the processing circuitry is further configured to:
determine 2D positions of two points in the panorama image of the room;
calculate 3D positions of the two points based on the 2D positions of the two points, the size of the panorama image, and the distance between the floor plane of the room and the capture position of the device; and
calculate a distance between the 3D positions of the two points.

19. A non-transitory computer-readable storage medium storing instructions which, when executed by at least one processor, cause the at least one processor to perform:
determining two-dimensional (2D) positions of wall corner points of a room in a panorama image of the room based on a user selection of the wall corner points captured in the panorama image, each of the wall corner points being in one of a floor plane or a ceiling plane of the room;
calculating three-dimensional (3D) positions of the wall corner points based on the 2D positions of the wall corner points, a size of the panorama image, and a distance between the floor plane of the room and a capture position of a device configured to capture the panorama image of the room;
determining a layout of the room based on an order of the wall corner points; and
generating a 3D representation of the room based on the layout of the room and the 3D positions of the wall corner points.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the determining the 2D positions of the wall corner points comprises:
determining the 2D positions of the wall corner points of the room in the panorama image of the room based on an order in which the wall corner points are selected.

Assignees

• Tencent America LLC

Inventors

• Zhang Xiang
• Jian Bing
• He Lu
• Zhu Haichao
• Liu Shan
• Liu Kelin
• Feng Weiwei

Classifications

• G06T7/50

  Depth or shape recovery · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06T2207/20101

  Interactive definition of point of interest, landmark or seed · CPC title

• G06T2210/04

  Architectural design, interior design · CPC title