Automated Generation Of Building Floor Plans Having Associated Absolute Locations Using Coordination Of Multiple Data Sets

Patent metadata

Field	Value
Publication number	US-2025190640-A1
Application number	US-202318243652-A
Country	US
Kind code	A1
Filing date	Sep 7, 2023
Priority date	Sep 7, 2023
Publication date	Jun 12, 2025
Grant date	—

Abstract

Techniques are described for using data capture devices at a building to automatically generate a building floor plan and to determine associated absolute location data for the generated floor plan, such as by associating separately captured GPS data or other absolute location data with the floor plan. In some situations, a building floor plan is automatically generated by analyzing visual data of images captured at multiple image acquisition locations by a camera device to determine room shapes of surrounding rooms, and GPS absolute location data is associated with the generated floor plan using additional data captured at other data capture locations at the building by a separate mobile device that moves independently from the camera device, such as by extending the absolute location data from the mobile device to the camera device's image acquisition location and its surrounding room shape.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
obtaining, by one or more computing devices during an acquisition session for a house that has multiple rooms, and for each of the multiple rooms:
a first panorama image captured by a camera device that is moved by a first user and that lacks any GPS (global positioning system) receivers, the first panorama image captured at a first acquisition location in that room and having first visual data with 360 degrees of horizontal visual coverage showing walls of that room, and
second data concurrently captured by a mobile capture device that is moved independently from the camera device by a second user, the second data including multiple GPS location data points for multiple second capture locations in that room;
determining, by the one or more computing devices, a floor plan for the house with associated GPS location data based at least in part on combining the second data captured at the multiple second capture locations in each of the multiple rooms with information about that room determined from analysis of the first visual data of the first panorama image captured in that room, including:
analyzing, by the one or more computing devices and for each of the multiple rooms, the first visual data of the first panorama image captured in that room to determine a three-dimensional (“3D”) room shape of that room that models at least some of the walls of that room as planar surfaces, and a position within that determined 3D room shape of the first acquisition location in that room;
determining, by the one or more computing devices and for each of the multiple rooms, a representative GPS location data point associated with the first acquisition location in that room, including identifying two or more candidate GPS location data points from the multiple GPS location data points captured in that room using capture times of the two or more candidate GPS location data points, and using the two or more candidate GPS location data points to produce the representative GPS location data point for the first acquisition location in that room;
generating, by the one or more computing devices, the floor plan for the house, including positioning the determined 3D room shapes of the multiple rooms relative to each other, and determining a position in the floor plan, using a local coordinate system for the floor plan, of the first acquisition location in each of the multiple rooms;
determining, by the one or more computing devices, a global transformation that maps data in a first data set to data in a second data set, the first data set including the positions in the floor plan of the first acquisition locations of the multiple rooms using the local coordinate system for the floor plan, and the second data set including the representative GPS location data points associated with the first acquisition locations;
determining, by the one or more computing devices and for each of the multiple rooms, GPS location data for the at least some walls of the determined 3D room shape of that room by using the global transformation to combine the determined GPS location data for the first acquisition location in that room with the determined position within that determined 3D room shape of the first acquisition location in that room; and
identifying, by the one or more computing devices, at least one exterior wall of the house represented on the floor plan, and using the GPS location data for the at least some walls of the determined 3D room shapes of the multiple rooms to determine associated GPS location data for the at least one exterior wall; and
displaying, by the one or more computing devices and using the associated GPS location for the at least one exterior wall of the floor plan, the generated floor plan for the house positioned on a map of an area that surrounds a property on which the house is located.

2 . The computer-implemented method of claim 1 wherein the two or more candidate GPS location data points associated with the first acquisition location in one of the multiple rooms are captured at multiple times and at multiple capture locations and are a subset of the multiple GPS location data points captured in the one room after performing smoothing operations that remove outlier data points, wherein the one or more computing devices include the mobile capture device, and wherein the determining of the representative GPS location data point associated with the first acquisition location in the one room includes:
retrieving, by the one or more computing devices, an acquisition time of the first panorama image acquired at the first acquisition location in the one room;
determining, by the one or more computing devices, the two or more candidate GPS location data points associated with the first acquisition location in the one room based at least in part on the two or more candidate GPS location data points having capture times that are within a defined time window around the retrieved acquisition time; and
generating, by the one or more computing devices, the representative GPS location data point associated with the first acquisition location in the one room based at least in part on averaging location data for the multiple capture locations of the two or more candidate GPS location data points associated with the first acquisition location in the one room.

3 . The computer-implemented method of claim 1 wherein the determining of the GPS location data for the at least some walls of the determined 3D room shape for each of the multiple rooms includes, by the one or more computing devices and for each of at least some of the first acquisition locations, determining a revised representative GPS location data point for that first acquisition location based on the global transformation, and further includes extending GPS location data from each of one or more of the first acquisition locations to the at least some walls of the determined 3D room shape for the room in which that first acquisition location is positioned, wherein walls to which the GPS location data is extended include the at least one exterior wall.

4 . The computer-implemented method of claim 1 wherein the determining of the global transformation includes at least one of:
determining, by the one or more computing devices, a rigid transformation between the first and second data sets using Kabsch algorithm; or
determining, by the one or more computing devices, a rigid transformation between the first and second data sets using Quaternion estimation algorithm; or
determining, by the one or more computing devices, a rigid transformation between the first and second data sets using an algorithm that can solve Wahba's problem; or
determining, by the one or more computing devices, the global transformation using point-set registration techniques; or
determining, by the one or more computing devices, the global transformation using a trained machine learning model that takes as input at least the first and second data sets and that produces the global transformation as an output.

5 . A computer-implemented method comprising:
obtaining, by one or more computing devices, a plurality of images and a plurality of GPS (global positioning system) location data points in multiple rooms of a building, including, for each of the multiple rooms, one or more images of the plurality of images that are acquired by a camera device at a first acquisition location in that room and have visual coverage of walls of that room, and multiple GPS location data points of the plurality of GPS location data points that are concurrently captured at multiple second capture locations in that room by a mobile capture device moved independently from the camera device; obtaining, by the one or more computing devices and based on

Assignees

• Mftb Holdco Inc

Inventors

• Narayana Manjunath
• Kim Yongjoon
• Cier Sean P
• Kang Sing Bing

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title