Method and device for determining external parameter of stereoscopic camera

Patent metadata

Field	Value
Publication number	US-11010924-B2
Application number	US-201716340901-A
Country	US
Kind code	B2
Filing date	Sep 30, 2017
Priority date	Oct 12, 2016
Publication date	May 18, 2021
Grant date	May 18, 2021

Abstract

Embodiments of application discloses a method and device for determining external parameters of a stereo camera. The method comprises: acquiring ground surface data of a location of the stereo camera; generating, according to three-dimensional coordinates of each pixel in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels; determining, according to a distance between the origin of the camera coordinate system and the ground surface, a setting height of the stereo camera; and generating, according to the first plane equation, a plane normal vector corresponding to the first plane equation; and determining, according to the plane normal vector, a pitch angle of the stereo camera and an inclination angle of the stereo camera. The external parameters of the stereo camera can be determined without using a reference point in a scene.

First claim

The invention claimed is:

1. A method for determining external parameters of a stereo camera, comprising:
acquiring ground surface data of a location where the stereo camera is positioned, wherein the ground surface data includes a two-dimensional image and a disparity image of the ground surface or includes a two-dimensional image and a depth image of the ground surface;
generating, according to three-dimensional coordinates of each of pixels in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels;
determining, according to a distance between an origin of the camera coordinate system and the ground surface, a setting height of the stereo camera;
generating, according to the first plane equation, a plane normal vector corresponding to the first plane equation; and
determining, according to the plane normal vector, the external parameters of the stereo camera without using a vertical edge perpendicular to the ground in a scene as a reference point, wherein the external parameters comprises a pitch angle of the stereo camera and an inclination angle of the stereo camera.

2. The method according to claim 1 , further comprising:
performing filtering processing on the ground surface data when the ground surface data does not meet a preset condition; wherein the filtering processing comprises median filtering in time domain and median filtering in spatial domain.

3. The method according to claim 1 , wherein, generating, according to three-dimensional coordinates of each of pixels in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels comprises:
selecting at least three pixels from pixels in the ground surface data, and using three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as a first sample set;
generating, according to the first sample set, a second plane equation that conforms to three-dimensional coordinates of pixels in the first sample set in the camera coordinate system;
determining, among the pixels in the ground surface data, all pixels that meet the second plane equation, and using the three-dimensional coordinates of the pixels that meet the second plane equation in the camera coordinate system as a second sample set; and
generating, according to the second sample set, the first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels in the ground surface data in the camera coordinate system.

4. The method according to claim 3 , wherein, generating, according to the second sample set, the first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels in the ground surface data in the camera coordinate system comprises:
performing fitting processing on the second sample set by using a least squares method to generate the first plane equation that conforms to the three-dimensional coordinates of the pixels in the ground surface data in the camera coordinate system.

5. The method according to claim 3 , wherein, selecting at least three pixels from pixels in the ground surface data comprises:
selecting at least three pixels from the pixels in the ground surface data by using a random sample consensus RANSAC algorithm.

6. The method according to claim 5 , wherein, selecting at least three pixels from pixels in the ground surface data, and using three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as a first sample set and generating, according to the first sample set, a second plane equation that conforms to three-dimensional coordinates of pixels in the first sample set in the camera coordinate system comprises:
selecting at least three pixels from the pixels in the ground surface data by using a RANSAC algorithm, and using the three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as the first sample set;
generating, according to the first sample set, a current plane equation that conforms to the three-dimensional coordinates of pixels in the first sample set in the camera coordinate system;
determining whether the current plane equation satisfies a preset condition;
using the current plane equation as the second plane equation when the current plane equation satisfies the preset condition;
increasing the number of iterations by 1 when the current plane equation does not satisfy the preset condition and returning to perform the steps of selecting at least three pixels from pixels in the ground surface data, and using three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as a first sample set; wherein an initial value of the number of iterations is 0.

7. The method according to claim 6 , wherein, the preset condition comprises:
the number of pixels, among the pixels in the ground surface data, that meet the current plane equation reaches a preset first threshold, or the number of iterations corresponding to the current plane equation reaches a preset second threshold.

8. The method according to claim 1 , wherein, determining, according to the plane normal vector, a pitch angle of the stereo camera and an inclination angle of the stereo camera comprises:
determining, in the camera coordinate system, a first angle by which a corresponding axis z c rotates about an axis x in a clockwise direction when the stereo camera rotates from a standard attitude to a current attitude, and using the first angle as the pitch angle of the stereo camera; and
determining, in the camera coordinate system, a second angle by which a corresponding axis x c rotates about an axis z in a clockwise direction when the stereo camera rotates from the standard attitude to the current attitude, and using the second angle as the inclination angle of the stereo camera;
wherein a three-dimensional coordinate system defined by the axis x c , the axis z c , and an axis y c is the camera coordinate system; a three-dimensional coordinate system defined by the axis x, the axis z, and an axis y is a world coordinate system; when directions of the axis x c and the axis z c coincide with directions of the axis x and the axis z, and when direction of the axis y c is opposite to direction of the axis y, the stereo camera is in the standard attitude.

9. A non-transitory storage medium, which is used for storing executable program codes, wherein the executable program codes, when being executed, are configured to implement the method for determining external parameters of a stereo camera according to claim 1 .

10. A device for determining external parameters of a stereo camera, comprising:
an acquiring unit, configured for acquiring ground surface data of a location where the stereo camera is positioned, wherein the ground surface data includes a two-dimensional image and a disparity image of the ground surface or includes a two-dimensional image and a depth image of the ground surface;
a first generating unit, configured for generating, according to three-dimensional coordinates of each of pixels in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels;
a first determining unit

Assignees

• Hangzhou Hikvision Digital Tec

Inventors

• Zhang Di
• Gong Hui
• Ren Ye

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• G06T2207/20032

  Median filtering · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T2200/04

  involving 3D image data · CPC title