Mobile imaging platform calibration

Patent metadata

Field	Value
Publication number	US-9922422-B2
Application number	US-201615250069-A
Country	US
Kind code	B2
Filing date	Aug 29, 2016
Priority date	Jun 14, 2013
Publication date	Mar 20, 2018
Grant date	Mar 20, 2018

Abstract

Mobile platforms are used to capture an area using a variety of sensors (e.g., cameras and laser scanners) while traveling through the area, in order to create a representation (e.g., a navigable set of panoramic images, or a three-dimensional reconstruction). However, such sensors are often precisely calibrated in a controlled setting, and miscalibration during travel (e.g., due to a physical jolt) may result in a corruption of data and/or a recalibration that leaves the platform out of service for an extended duration. Presented herein are techniques for verifying sensor calibration during travel. Such techniques involve the identification of a sensor path for each sensor over time (e.g., a laser scanner path, a camera path, and a location sensor path) and a comparison of the paths, optionally after registration with a static coordinate system, to verify that the continued calibration of the sensors during the mobile operation of the platform.

First claim

What is claimed is:

1. A computer implemented method comprising:
receiving first information from a first sensor;
translating the first information into a first plurality of vectors representing a path of the first sensor, the first plurality of vectors comprising a first vector having a first location value, a first orientation value, a first translation value, the first vector associated with a time;
receiving second information from a second sensor;
translating the second information into a second plurality of vectors representing a path of the second sensor, the second plurality of vectors comprising a second vector having a second location value, a second orientation value, and a second translation value, the second vector associated with the time;
receiving third information from a third sensor;
translating the third information into a third plurality of vectors representing a path of the third sensor, the third plurality of vectors comprising a third vector having a third location value, a third orientation value, and a third translation value, the third vector associated with the time;
performing a three-way comparison by:
comparing the first plurality of vectors with the second plurality of vectors;
comparing the second plurality of vectors with the third plurality of vectors;
comparing the third plurality of vectors with the first plurality of vectors;

determining, based on the comparison, that the first sensor is mis-calibrated based on a determining that:
a first difference exists between the first vector and the second vector;
a second difference exists between the first vector and the third vector; and
a similarity exists between the third vector and the second vector.

2. The method of claim 1 , wherein the first sensor is a laser, the second sensor is a camera, and the third sensor is a GPS sensor.

3. The method of claim 1 , wherein the first difference results from the first location value being different from the second location value; wherein the second difference results from the first location value being different from the third location value; and wherein the similarity results from the second location value being similar to the third location value.

4. The method of claim 1 , wherein the first sensor, the second sensor, and the third sensor are located on a single platform.

5. The method of claim 4 , further comprising calibrating the first sensor by
positioning the platform relative to a fixed position of at least one target in a fixed area;
comparing a first detected position of the target by the first sensor with the fixed position of the target relative to the single platform; and
comparing a second detected position of the target by the second sensor with the fixed position of the target relative to the single platform.

6. The method of claim 4 , wherein the single platform is fixed to a vehicle.

7. The method of claim 6 , wherein the first plurality of vectors, the second plurality of vectors, and the third plurality of vectors is captured during motion of the vehicle.

8. A computer-readable hardware device storing instructions that, when executed, perform a method, the method comprising:
receiving first information from a first sensor;
translating the first information into a first plurality of vectors representing a path of the first sensor, the first plurality of vectors comprising a first vector having a first location value, a first orientation value, a first translation value, the first vector associated with a time;
receiving second information from a second sensor;
translating the second information into a second plurality of vectors representing a path of the second sensor, the second plurality of vectors comprising a second vector having a second location value, a second orientation value, and a second translation value, the second vector associated with the time;
receiving third information from a third sensor;
translating the third information into a third plurality of vectors representing a path of the third sensor, the third plurality of vectors comprising a third vector having a third location value, a third orientation value, and a third translation value, the third vector associated with the time;
performing a three-way comparison by:
comparing the first plurality of vectors with the second plurality of vectors;
comparing the second plurality of vectors with the third plurality of vectors;
comparing the third plurality of vectors with the first plurality of vectors;

determining, based on the comparison, that the first sensor is mis-calibrated based on a determining that:
a first difference exists between the first vector and the second vector;
a second difference exists between the first vector and the third vector; and
a similarity exists between the third vector and the second vector.

9. The computer-readable hardware device of claim 8 , wherein the first sensor is a laser, the second sensor is a camera, and the third sensor is a GPS sensor.

10. The computer-readable hardware device of claim 8 , wherein the first difference results from the first location value being different from the second location value; wherein the second difference results from the first location value being different from the third location value; and wherein the similarity results from the second location value being similar to the third location value.

11. The computer-readable hardware device of claim 8 , wherein the first sensor, the second sensor, and the third sensor are located on a single platform.

12. The computer-readable hardware device of claim 11 , further comprising calibrating the first sensor by
positioning the platform relative to a fixed position of at least one target in a fixed area;
comparing a first detected position of the target by the first sensor with the fixed position of the target relative to the single platform; and
comparing a second detected position of the target by the second sensor with the fixed position of the target relative to the single platform.

13. The computer-readable hardware device of claim 11 , wherein the single platform is fixed to a vehicle.

14. The computer-readable hardware device of claim 11 , wherein the first plurality of vectors, the second plurality of vectors, and the third plurality of vectors is captured during motion of the vehicle.

15. A system comprising a processor in communication with computer-readable media, the computer-storage media storing instructions that, during execution, cause the processor to perform the following method:
receiving first information from a first sensor;
translating the first information into a first plurality of vectors representing a path of the first sensor, the first plurality of vectors comprising a first vector having a first location value, a first orientation value, a first translation value, the first vector associated with a time;
receiving second information from a second sensor;
translating the second information into a second plurality of vectors representing a path of the second sensor, the second plurality of vectors comprising a second vector having a second location value, a second orientation value, and a second translation value, the second vector associated with the time;
receiving third information from a third sensor;
translating the third information into a third plurality of vectors representing a path of the third sensor, the third plurality of vectors comprising a third vector having a third location value, a third orientation value, and a third translation value, the third vector associated with the time;
performing a three

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Irschara Arnold
• Schweighofer Gerald
• Karner Konrad
• Ladstädter Richard
• Ponticelli Martin

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G01S17/42

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) · CPC title

• G01S17/86 Primary

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title

• G06T7/00

  Image analysis · CPC title

• G06T3/00

  Geometric image transformations in the plane of the image · CPC title