Estimating dimensions for an enclosed space using a multi-directional camera

Patent metadata

Field	Value
Publication number	US-11276191-B2
Application number	US-201916252426-A
Country	US
Kind code	B2
Filing date	Jan 18, 2019
Priority date	Jul 22, 2016
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

Certain examples described herein relate to estimating dimensions of an enclosed space such as a room using a monocular multi-directional camera device. In examples, a movement of the camera device around a point in a plane of movement is performed, such as by a robotic device. Using the monocular multi-directional camera device, a sequence of images are obtained at a plurality of different angular positions during the movement. Pose data is determined from the sequence of images. The pose data is determined using a set of features detected within the sequence of images. Depth values are then estimated by evaluating a volumetric function of the sequence of images and the pose data. A three dimensional volume is defined around a reference position of the camera device, wherein the three-dimensional volume has a two-dimensional polygonal cross-section within the plane of movement. The three dimensional volume is then fitted to the depth values to determine dimensions for the polygonal cross-section. These dimensions then provide an estimate for the shape of the enclosed space.

First claim

What is claimed is:

1. An image processing method for estimating dimensions of an enclosed space comprising:
obtaining image data from a monocular multi-directional camera device located within the enclosed space, the monocular multi-directional camera device being arranged to capture image data from a plurality of angular positions, the image data comprising a sequence of images having disparity within a plane of movement of the camera device;
determining pose data corresponding to the image data, the pose data indicating the location and orientation of the monocular multi-directional camera device, the pose data being determined using a set of features detected within the image data;
estimating depth values by evaluating a volumetric function of the image data and the pose data, each depth value representing a distance from a reference position of the monocular multi-directional camera device to a surface in the enclosed space;
defining a three-dimensional volume around the reference position of the monocular multi-directional camera device, the three-dimensional volume having a two-dimensional polygonal cross-section within the plane of movement of the camera device; and
fitting the three-dimensional volume to the depth values to determine dimensions for the polygonal cross-section,
wherein the determined dimensions provide an estimate for the dimensions of the enclosed space.

2. The method of claim 1 , wherein fitting the three-dimensional volume to the depth values comprises:
optimizing, with regard to the dimensions for the polygonal cross-section, a function of an error between:
a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and
a second set of depth values estimated from the reference position to the walls of the three-dimensional volume.

3. The method of claim 2 , comprising:
using ray tracing to determine the second set of depth values.

4. The method of claim 2 , wherein the function of the error is evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values.

5. The method of claim 2 , wherein the function comprises an asymmetric function.

6. The method of claim 5 , wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.

7. The method of claim 2 , comprising:
applying automatic differentiation with forward accumulation to compute Jacobians, wherein said Jacobians are used to optimise the function of the error between the first and second sets of depth values.

8. The method of claim 1 , wherein the polygonal cross-section comprises a rectangle and said dimensions comprise distances from the reference position to respective sides of the rectangle.

9. The method of claim 8 , wherein fitting the three-dimensional volume comprises determining an angle of rotation of the rectangle with respect to the reference position.

10. The method of claim 9 , wherein the three-dimensional volume is fitted using a coordinate descent approach that evaluates the distances from the reference position to respective sides of the rectangle before the angle of rotation of the rectangle with respect to the reference position.

11. The method of claim 8 , wherein:
the method is repeated for multiple spaced movements of the monocular multi-directional camera device to determine dimensions for a plurality of rectangles, the rectangles representing an extent of the enclosed space.

12. The method of claim 11 , comprising:
determining an overlap of the rectangles; and
using the overlap to determine room demarcation within the enclosed space,
wherein, if the overlap is below a predefined threshold, the plurality of rectangles are determined to be associated with a respective plurality of rooms within the space, and
wherein, if the overlap is above a predefined threshold, the plurality of rectangles are determined to be associated with a complex shape of the enclosed space.

13. The method of claim 11 , comprising:
computing a Boolean union of the plurality of rectangles to provide an estimate for a shape of the enclosed space.

14. The method of claim 1 , comprising:
inputting the dimensions for the polygonal cross-section into a room classifier; and
determining a room class using the room classifier.

15. The method of claim 14 , comprising:
determining an activity pattern for a robotic device based on the room class.

16. A system for estimating dimensions of an enclosed space comprising:
a monocular multi-directional camera device to capture a sequence of images from a plurality of angular positions within the enclosed space;
a pose estimator to determine pose data from the sequence of images, the pose data indicating the location and orientation of the monocular multi-directional camera device at a plurality of positions during the instructed movement, the pose data being determined using a set of features detected within the sequence of images;
a depth estimator to estimate depth values by evaluating a volumetric function of the sequence of images and the pose data, each depth value representing a distance from a reference position of the multi-directional camera device to a surface in the enclosed space; and
a dimension estimator to:
fit a three-dimensional volume to the depth values from the depth estimator by optimising dimensions of a two-dimensional polygonal cross-section of the three-dimensional volume, and
output an estimate for the dimensions of the enclosed space based on the optimised dimensions of the two-dimensional polygonal cross-section.

17. The system of claim 16 , wherein at least one of the monocular multi-directional camera device, the depth estimator, the pose estimator and the dimension estimator are embedded within a robotic device.

18. The system of claim 17 , comprising:
a room database comprising estimates from the dimension estimator for a plurality of enclosed spaces within a building.

19. The system of claim 18 , wherein data from the room database is accessible from a mobile computing device over a network.

20. A non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed by a processor, cause a computing device to map a space, wherein the instructions cause the computing device to:
receive a sequence of frames from a monocular multi-directional camera, the multi-directional camera being arranged to capture image data for each of the frames from a plurality of angular positions, the sequence of frames being captured at different angular positions within a plane of movement for the space;
determine location and orientation estimates for the camera for each frame by matching detected features across the sequence of frames;
bundle adjust the location and orientation estimates for the camera and the detected features across the sequence of frames to generate an optimised set of location and orientation estimates for the camera;
determine a reference frame from the sequence of frames, the reference frame having an associated reference location and orientation;
evaluate a photometric error function between pixel values for the reference frame and projected pixel values from a set of comparison images that overlap the reference frame, said projected pixel values being a functio

Assignees

• Imperial College Sci Tech & Medicine
• Imperial College Innovations Ltd

Inventors

• Lukierski Robert
• Leutenegger Stefan
• Davison Andrew

Classifications

• G05D1/0251

  extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593) · CPC title

• G05D1/0044

  by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps (maps used for automatic navigation G05D1/0274; flight directors G01C23/005) · CPC title

• G05D2105/87

  for exploration, e.g. mapping of an area · CPC title

• G05D1/243

  Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals (using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249) · CPC title

• G05D2111/10

  Optical signals · CPC title