Reality capture device

Patent metadata

Field	Value
Publication number	US-12008783-B2
Application number	US-202217827527-A
Country	US
Kind code	B2
Filing date	May 27, 2022
Priority date	May 27, 2021
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

A reality capture device configured to perform a measuring process for generating a digital representation of an environment comprising a body defining a first axis, and an imaging unit with one or more 2D cameras configured to provide 2D image data of the environment. The device comprises a ToF camera arrangement configured for capturing 3D point-cloud data of the environment and comprising at least two time-of-flight cameras, wherein each time-of-flight camera comprises a sensor array and one or more laser emitters, the sensor array of each of the time-of-flight cameras having an optical axis and being configured to receive reflections of light pulses emitted by the one or more laser emitters of the respective time-of-flight camera, the time-of-flight cameras being arranged around the first axis so that each sensor array has one or two other sensor arrays as a neighbouring sensor array.

First claim

The invention claimed is:

1. A reality capture device configured to perform a measuring process for generating a digital representation of an environment, the device comprising:
a body defining a first axis,
an imaging unit with one or more 2D cameras configured to provide two-dimensional image data of the environment,
a ToF camera arrangement configured for capturing three-dimensional point-cloud data of the environment and comprising at least two time-of-flight cameras, wherein:
each time-of-flight camera comprises a sensor array and one or more laser emitters,
the sensor array of each of the time-of-flight cameras has an optical axis and is configured to receive reflections of light pulses emitted by the one or more laser emitters of the respective time-of-flight camera, and
the at least two time-of-flight cameras are arranged around the first axis so that each sensor array has one or two other sensor arrays as a neighbouring sensor array, wherein no angle about the first axis between the optical axes of a sensor array and one of its neighbouring sensor arrays is larger than

360
⁢
°

n

+

20
⁢
°

,

where n is the number of time-of-flight cameras arranged around the first axis,
wherein the device is a mobile reality capture device configured to be carried and moved by a mobile carrier, and to be moved during the measuring process, the mobile reality capture device comprising a localization unit configured for continuously determining a pose of the mobile reality capture device and for generating localization data, particularly wherein:
the localization unit comprises an inertial measurement unit, and/or is configured for determining a trajectory of the mobile reality capture device;
the mobile reality capture device is designed to be oriented during the measuring process so that the first axis is upright, and/or
the mobile reality capture device comprises a handle portion and is configured to be carried by a person.

2. The reality capture device according to claim 1 , the ToF camera arrangement comprising at least three time-of-flight cameras that are around the first axis so that each sensor array has two other sensor arrays as neighbouring sensor arrays, wherein no angle about the first axis between the optical axes of a sensor array and one of its neighbouring sensor arrays is larger than 140°.

3. The reality capture device according to claim 1 , wherein:
each sensor array has a resolution of at least 0.3 Megapixels, particularly wherein each sensor array has a resolution of at least 640×480 pixels;
the sensor arrays are arranged and configured for jointly covering at least 75% of a hemispherical field of view, particularly at least 90%; and/or
the laser emitters comprise laser diodes provided on a printed circuit board of the respective time-of-flight camera, and/or are configured to emit infrared light.

4. The reality capture device according to claim 1 , wherein the time-of-flight cameras and the at least one 2D camera are integrated into the body, wherein the body has a housing with a lateral surface defining the first axis, wherein:
the lateral surface is circumferentially arranged around the first axis, and/or
the housing comprises an area that is permeable for infrared radiation, and the laser emitters are integrated into the housing and configured to emit infrared laser beams through the permeable area.

5. The reality capture device according to claim 1 , configured for executing the measuring process while the device is moved along a path through the environment, in the course of which measuring process:
three-dimensional point-cloud data of the environment is captured continuously with the time-of-flight cameras,
two-dimensional image data of the environment is captured continuously by the one or more 2D cameras,
the localization unit is configured to continuously generate localization data while the device is moved along the path and to track a pose of the device based on the localization data, particularly in 6 degrees of freedom, and
the device is configured to link the captured point-cloud data and image data to the pose at which they are captured,
wherein the time-of-flight cameras and the one or more 2D cameras each are configured to capture and provide the three-dimensional point-cloud data and the two-dimensional image data with a rate of at least 5 operations per second, particularly at least 25 operations per second.

6. The reality capture device according to claim 5 , wherein the localization unit is configured to execute:
a ToF-SLAM functionality using the three-dimensional point-cloud data for simultaneous localization and mapping, particularly also using two-dimensional image data of the imaging unit and/or localization data of the localization unit; and/or
a pose and trajectory determination functionality for continuously determining a pose and a trajectory of the device based on two-dimensional image data of the imaging unit and/or on three-dimensional point-cloud data.

7. The reality capture device according to claim 1 , wherein the localization unit is configured to execute:
a ToF-SLAM functionality using the three-dimensional point-cloud data for simultaneous localization and mapping; and/or
a pose and trajectory determination functionality for continuously determining a pose and a trajectory of the device based on two-dimensional image data of the imaging unit and/or on three-dimensional point-cloud data.

8. The reality capture device according to claim 7 , wherein:
at least a subset of the laser emitters is configured to emit light pulses in the form of a pattern to generate a pattern of reflections of the light pulses, and
the ToF camera arrangement is configured for capturing three-dimensional point-cloud data using the pattern of reflections,
the subset of the laser emitters comprises an optical lens, grating or mesh to produce the pattern, and/or
a localization unit of the device is configured to use the three-dimensional point-cloud data of the pattern of reflections to perform a ToF SLAM functionality for simultaneous localization and mapping.

9. The reality capture device according to claim 1 , wherein:
at least a subset of the laser emitters is configured to emit light pulses in the form of a pattern to generate a pattern of reflections of the light pulses, and
the ToF camera arrangement is configured for capturing three-dimensional point-cloud data using the pattern of reflections,
the subset of the laser emitters comprises an optical lens, grating or mesh to produce the pattern, and/or
a localization unit of the device is configured to use the three-dimensional point-cloud data of the pattern of reflections to perform a ToF SLAM functionality for simultaneous localization and mapping.

10. The reality capture device according to claim 9 , wherein:
the laser emitters are configured emit diffused infrared lighting,
the sensor array of each of the time-of-flight cameras is configured to receive reflections of the diffused infrared lighting emitted by the one or more laser emitters of the respective time-of-flight camera,
the time-of-flight cameras are configured to generate intensity images based on the received reflections of the diffused infrared lighting, and
the localization unit is configured to execute a Visual-S

Assignees

• Leica Geosystems Ag

Inventors

• Wohlgenannt Rainer
• Heinzle Lukas
• Bruneau Garance
• Harti Ralph
• Graf Roland
• Steffen Roman
• Mark Simon

Classifications

• H04N7/015

  High-definition television systems · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G06T2207/10048

  Infrared image · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title