Imaging-based sensor calibration

Patent metadata

Field	Value
Publication number	US-9940535-B1
Application number	US-201715698316-A
Country	US
Kind code	B1
Filing date	Sep 7, 2017
Priority date	Sep 7, 2017
Publication date	Apr 10, 2018
Grant date	Apr 10, 2018

Abstract

Sensor calibration relative to common coordinates with depth, height and shift dimensions includes obtaining, via a mobile apparatus camera, an image of a calibration target. The calibration target includes first and second surfaces at first and second predefined depths, bearing first and second sets of indicia at heights encoded by the indicia and having predefined shifts. The method includes decoding the heights; generating first and second transforms between image coordinates and first and second planes at predefined common coordinate depths; applying the transforms to each of a plurality of calibration pixels to generate position pairs including calibration positions on each of the first and second planes; determining a common coordinate camera position from an intersection of calibration lines defined by the position pairs; and storing the camera position in association with a location of the mobile apparatus, for common coordinate mapping of subsequent images captured at subsequent mobile apparatus locations.

First claim

The invention claimed is:

1. A method of sensor calibration relative to a common frame of reference having orthogonal depth, height and shift dimensions, by an imaging controller, the method comprising:
obtaining, via a camera mounted on a mobile automation apparatus, an image of a calibration target associated with a location of the mobile automation apparatus in the common frame of reference; the calibration target including:
a first surface at a first predefined depth bearing a first set of indicia at respective first heights and having respective first predefined shifts, each of the first indicia encoding the corresponding first height; and
a second surface at a second predefined depth bearing a second set of indicia at respective second heights and having respective second predefined shifts, each of the second indicia encoding the corresponding second height;

decoding, by the imaging controller, the first and second heights from the indicia of the first and second sets;
generating, by the imaging controller, a first transform between an image frame of reference and a first plane at the first predefined depth in the common frame of reference, and a second transform between the image frame of reference and a second plane at the second predefined depth in the common frame of reference;
applying, by the imaging controller, the first and second transforms to each of a plurality of calibration pixels selected from the image to generate, for each calibration pixel, a position pair including a first calibration position on the first plane and a second calibration position on the second plane; and
determining, by the imaging controller, a camera position in the common frame of reference from an intersection of calibration lines defined by the position pairs.

2. The method of claim 1 , further comprising: storing the camera position in the common frame of reference in association with the location of the mobile automation apparatus.

3. The method of claim 2 , further comprising: mapping subsequent images captured at subsequent mobile automation apparatus locations to the common frame of reference.

4. The method of claim 1 , wherein obtaining the image further comprises controlling a plurality of cameras mounted on the mobile automation apparatus to capture respective images of the calibration target; and
wherein the method further comprises repeating the decoding, generating, applying, determining and storing for each of the images.

5. The method of claim 4 , wherein controlling the plurality of cameras further comprises:
controlling a first subset of the cameras to simultaneously capture first respective images; and
subsequently controlling a second subset of the cameras to simultaneously capture second respective images.

6. The method of claim 1 , wherein decoding the indicia of the first and second sets further comprises:
identifying the indicia of the first and second sets in the image; and
classifying each of the indicia into one of the first and second sets.

7. The method of claim 1 , wherein generating the first transform is based on (i) image positions of the indicia of the first set in the image frame of reference, (ii) the first heights decoded from the indicia of the first set, and (iii) the first predefined shifts; and
wherein generating the second transform is based on (i) the image positions of the indicia of the second set, (ii) the second heights decoded from the indicia of the second set, and (iii) the second predefined widths.

8. The method of claim 1 , further comprising:
determining at least one of an angle of an optical axis of the camera, a focal length of the camera, and an angle of rotation of the imaging frame of reference relative to the first plane.

9. The method of claim 1 , further comprising:
obtaining, via a lidar sensor mounted on the mobile automation apparatus, a plurality of depth measurements corresponding to the calibration target simultaneously with the image such that a line of light emitted by the lidar sensor is depicted in the image.

10. The method of claim 9 , further comprising:
deriving by the imaging controller, based on the depiction of the line of light in the image, a definition of a light emission plane of the lidar sensor in the common frame of reference.

11. The method of claim 10 , further comprising:
selecting, by the imaging controller, (i) a first pixel corresponding to the intersection of the line of light with an edge of the second surface and (ii) a second pixel corresponding to an adjacent intersection of the line of light with an edge of the first surface;
applying, by the imaging controller, the first transform to the first pixel and the second transform to the second pixel to generate corresponding first and second positions of the first and second pixels in the common frame of reference;
determining a lidar sensor position in the common frame of reference based on a line extending between the first and second positions and a depth measurement corresponding to one of the first and second pixels.

12. The method of claim 10 , further comprising:
selecting, by the imaging controller, a pair of the depth measurements corresponding to opposing edges of the second surface;
determining edge positions in the common frame of reference corresponding to each of the depth measurements; and
determining a lidar sensor position in the common frame of reference based on the edge positions and the pair of depth measurements.

13. An imaging controller for calibrating a sensor relative to a common frame of reference having orthogonal depth, height and shift dimensions, the imaging controller comprising:
a data capture controller configured to obtain, via a camera mounted on a mobile automation apparatus, an image of a calibration target associated with a location of the mobile automation apparatus in the common frame of reference; the calibration target including:
a first surface at a first predefined depth bearing a first set of indicia at respective first heights and having respective first predefined shifts, each of the first indicia encoding the corresponding first height; and
a second surface at a second predefined depth bearing a second set of indicia at respective second heights and having respective second predefined shifts, each of the second indicia encoding the corresponding second height;

a decoder configured to decode the first and second heights from the indicia of the first and second sets;
a transform generator configured to generate a first transform between an image frame of reference and a first plane at the first predefined depth in the common frame of reference, and a second transform between the image frame of reference and a second plane at the second predefined depth in the common frame of reference; and
a calibrator configured to:
apply the first and second transforms to each of a plurality of calibration pixels selected from the image to generate, for each calibration pixel, a position pair including a first calibration position on the first plane and a second calibration position on the second plane; and
determine a camera position in the common frame of reference from an intersection of calibration lines defined by the position pairs.

14. The imaging controller of claim 13 , wherein the calibrator is further configured to: store the camera position in the common frame of reference in association with the location of the mobile automation apparatus.

15. The imaging controller of claim 14 , wherein the calibrator is further configured to: map subsequent images captured at subsequent mobile automation apparatus

Assignees

• Symbol Technologies Llc

Inventors

• Obotnine Alexandre

Classifications

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• H04N25/674

  based on the scene itself, e.g. defocusing · CPC title

• H04N1/00307

  with a mobile telephone apparatus (constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00) · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/30244

  Camera pose · CPC title