Calibration device

Patent metadata

Field	Value
Publication number	US-10638115-B2
Application number	US-201515513842-A
Country	US
Kind code	B2
Filing date	Oct 16, 2015
Priority date	Oct 24, 2014
Publication date	Apr 28, 2020
Grant date	Apr 28, 2020

Abstract

In a calibration device, entry work of coordinate information on a real space is not required. An imaging system is configured to include an image acquisition unit which acquires an image from an imaging device, an object extraction unit which extracts a plurality of objects from the image, a characteristic information addition unit which adds geometry information that indicates geometric relationships among the plurality of objects to each set of objects as characteristic information, a camera parameter estimation unit which obtains image coordinates of the objects in accordance with the characteristic information to estimate camera parameters based on the characteristic information and the image coordinates, and a camera parameter output unit which outputs the camera parameters.

First claim

The invention claimed is:

1. An imaging system, comprising:
a plurality of imaging devices;
a calibration device installed on a management server connected to a network, the calibration device including:
an image acquisition unit which acquires an image from the plurality of imaging devices;
an object extraction unit which extracts a plurality of objects from the image;
a characteristic information addition unit which adds geometry information that indicates geometric relationships among the plurality of objects to each of a set of objects as characteristic information;
a camera parameter estimation unit which obtains image coordinates of the plurality of objects in accordance with the characteristic information to estimate camera parameters of the plurality of imaging devices based on the characteristic information and the image coordinates, the camera parameter estimation unit including:
an object information acquisition unit which inputs the characteristic information output from the characteristic information addition unit and obtains the image coordinates;
an initial parameter acquisition unit which acquires an initial value of each of the estimated camera parameters; and
a parameter calculation unit which calculates a final camera parameter from the initial value of each of the estimated camera parameters based on the characteristic information added to the each of the plurality of objects from the image and the image coordinates;

a camera parameter output unit which outputs the estimated camera parameters estimated in the camera parameter estimation unit; and
a three-dimensional information acquisition unit which acquires three-dimensional information of the plurality of objects in the image and the estimated camera parameters obtained from the camera parameter output unit;

an interface which displays the three-dimensional information of the plurality of objects and a plurality of viewpoint-converted images comprising the three-dimensional information viewed from different viewpoints based on the estimated camera parameters, the interface including:
a parameter adjustment unit configured to receive a change in a value of one or more estimated camera parameters; and
a camera parameter accuracy confirmation unit configured to confirm an accuracy of the estimated camera parameters from the three-dimensional information of the plurality of objects acquired by the three-dimensional information acquisition unit and re-estimate the estimated camera parameters;

wherein a user can adjust the value of the estimated parameters when necessary by viewing the plurality of viewpoint-converted images displayed in the interface;
wherein the viewpoint-converted images are created from the three-dimensional information; and
wherein the user can re-input an adjusted camera parameter value again from the camera parameter accuracy confirmation unit into the camera parameter estimation unit to acquire a more accurate camera parameter.

2. The imaging system according to claim 1 , wherein, in a case where the characteristic information is vertical, the camera parameter output unit obtains, among the plurality of objects, for one configuring a horizontal plane on a real space, two arbitrary image coordinates of the object, and among the plurality of objects, for one vertical to a horizontal plane on a real space, two image coordinates on an axis in a height direction on a real space in an object area of the object.

3. The imaging system according to claim 1 , wherein, in a case where the characteristic information is level, the camera parameter output unit obtains, among the plurality of objects, for one vertical to a horizontal plane on a real space, two image coordinates on an axis in a height direction on a real space in an object area.

4. The imaging system according to claim 1 , wherein the interface is capable of displaying the image, selecting the plurality of objects, and adding characteristic information to the each arbitrary set of objects among the plurality of objects.

5. The imaging system according to claim 1 , wherein the object extraction unit carries out scene recognition of the image, detects a plurality of objects from the image based on results of the scene recognition, and adds the characteristic information.

6. An imaging system, comprising:
a plurality of imaging devices;
a calibration device installed on a management server connected to a network, the calibration device including:
an image acquisition unit which acquires an image from the plurality of imaging devices;
an object tracking unit which acquires characteristic information of a plurality of objects and image information of the image, the object tracking unit including:
an object detection unit which detects an object from the image;
an object recognition unit which determines whether a first object detected at a first time and a second object detected at a second time are identical;
an ID number addition unit which adds an ID number to the object from the image based upon a determination result by the object recognition unit; and
an object information storage unit which acquires, in a case where the first object detected at the first time and the second object detected at the second time are identical, information on height of the objects on images at the first and second times and adds geometry information that indicates a geometric relationship established between the first and second objects as characteristic information to each set of objects;

a camera parameter estimation unit which estimates camera parameters of the plurality of imaging devices based on the characteristic information and the image coordinates, the camera parameter estimation unit including:
an object information acquisition unit which inputs the characteristic information output from the characteristic information addition unit and obtains an image coordinate;
an initial parameter acquisition unit which acquires an initial value of each of the estimated camera parameters; and
a parameter calculation unit which calculates a final camera parameter from the initial value of each of the estimated camera parameters based on the characteristic information added to the each of the plurality of objects from the image and the image coordinates;

a camera parameter output unit which outputs the estimated camera parameters estimated in the camera parameter estimation unit; and
a three-dimensional information acquisition unit which acquires three-dimensional information of the plurality of objects in the image and the estimated camera parameters obtained from the camera parameter output unit:

an interface which displays the three-dimensional information of the plurality of objects and a plurality of viewpoint-converted images comprising the three-dimensional information viewed from different viewpoints based on the estimated camera parameters, the interface including:
a parameter adjustment unit configured to receive a change in a value of one or more estimated camera parameters; and
a camera parameter accuracy confirmation unit configured to confirm an accuracy of the estimated camera parameters from the three-dimensional information of the plurality of objects acquired by the three-dimensional information acquisition unit and re-estimate the estimated camera parameters;

wherein a user can adjust the value of the estimated parameters when necessary by viewing the plurality of viewpoint-converted images displayed in the interface;
wherein the viewpoint-converted images are created from the three-dimensional information; and
wherein the user can re-input an adjusted camera parameter value again from the camera parameter accuracy confirmation unit into the camera parameter estimation unit to acquire a more accurate camera paramete

Assignees

• Hitachi Ltd

Inventors

• Sasatani So
• Yumiba Ryou
• Itoh Masaya

Classifications

• G06T7/85

  Stereo camera calibration · CPC title

• H04N13/246 Primary

  Calibration of cameras · CPC title

• H04N13/302

  for viewing without the aid of special glasses, i.e. using autostereoscopic displays · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• H04N17/002

  for television cameras · CPC title