Range sensor based calibration of roadside cameras in ADAS applications

Patent metadata

Field	Value
Publication number	US-12415527-B2
Application number	US-202318352666-A
Country	US
Kind code	B2
Filing date	Jul 14, 2023
Priority date	Jul 14, 2023
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

A system for range sensor-based calibration of roadside cameras includes one or more cameras and range sensors with overlapping fields of view of a roadway. Control logic stored within and executed by a controller includes control logic for capturing image and range sensor data, for filtering the data to focus on first and second regions of interest (ROIs); for filtering the data to focus on first and second movements of interest (MOIs); for filtering the data to focus on first and second objects of interest (OOIs); for filtering the data to focus on first and second positions of interest (POIs); and for defining that objects detected by the camera and sensor and that satisfy both the first and second ROI, MOI, OOI, and POI filters are matching objects. The control logic calibrates the camera by applying a correction factor based on the matching objects.

First claim

What is claimed is:

1. A system for range sensor-based calibration of roadside cameras comprising:
one or more roadside cameras, the roadside cameras fixedly mounted to infrastructure and having a first field of view of at least a portion of a roadway;
one or more range sensors, the range sensors fixedly mounted to infrastructure and having a second field of view of at least a portion of a roadway, wherein the first and second fields of view at least partially overlap;
one or more controllers, each of the one or more controllers having a processor, a memory, and one or more input/output (I/O) ports, the I/O ports in communication with the one or more range sensors and the one or more roadside cameras; the memory storing programmatic control logic; the processor executing the programmatic control logic; the programmatic control logic including:
a first control logic for causing each of the one or more roadside cameras to capture image data, and for causing each of the range sensors to capture range sensor data;
a second control logic that filters the image data to focus on a first region of interest (ROI), and that filters the range sensor data to focus on a second ROI;
a third control logic that filters the image data to focus on a first movement of interest (MOI), and that filters the range sensor data to focus on a second MOI;
a fourth control logic that filters the image data to focus on a first object of interest (OOI), and that filters the range sensor data to focus on a second OOI;
a fifth control logic that filters the image data to focus on a first position of interest (POI), and that filters the range sensor data to focus on a second POI;
a sixth control logic that selectively determines that objects detected by the roadside camera and by the range sensor satisfy both the first and second ROI, MOI, OOI, and POI filters and defines the objects as matching objects;
a seventh control logic that saves the matching objects in the memory for calibration; and
an eighth control logic that calibrates the roadside camera by applying a correction factor based on the matching objects, so that the roadside camera accurately and precisely reports locations and movements of the matching objects for use in ADAS applications.

2. The system of claim 1 , wherein the second control logic further comprises:
control logic that constrains the image data from the roadside camera to a sub-region or ROI within the first field of view in which the image data captured by the roadside camera is reliably accurate and precise;
control logic that constrains the range sensor data from the range sensor to a sub-region or ROI within the second field of view in which the range sensor data captured by the range sensor is reliably accurate and precise; and
wherein the ROIs within the first and second fields of view are substantially identical.

3. The system of claim 1 , wherein the third control logic further comprises:
control logic that constrains the image data to focus on the first MOI, wherein the first MOI includes predictable and consistent motions of objects along the roadway; and
control logic that constrains the range sensor data to focus on the second MOI, wherein the second MOI includes predictable and consistent motions of objects along the roadway.

4. The system of claim 3 , wherein the predictable and consistent motions of objects along the roadway further comprise:
motions in a predefined set of directions, linear or straight-line motions, and motions at predefined accelerations and velocities.

5. The system of claim 1 , wherein the fourth control logic further comprises:
control logic that constrains the image data to define first OOIs by selecting objects that satisfy the first ROI filter, the first MOI filter, and that are directly and unobstructedly viewable by the roadside camera; and
control logic that constrains the range sensor data to define second OOIs by selecting objects that satisfy the second ROI filter, the second MOI filter, and that are directly and unobstructedly viewable by the range sensor.

6. The system of claim 5 , wherein the fifth control logic further comprises:
control logic that constrains the image data by selecting only first OOIs that are locations within the first ROI having predefined fixed or mobile positions;
control logic that constrains the range sensor data by selecting only second OOIs that are at locations within the second ROI having predefined fixed or mobile positions; and
wherein the predefined fixed or mobile positions comprise: locations of lane lines, positions of roadway signage, trees, traffic signals, telecommunications poles, fire hydrants, and/or curbs, and positions of movable or moving OOIs.

7. The system of claim 1 , wherein the sixth control logic further comprises:
control logic that defines the objects as matching objects by comparing objects that satisfy both the first and second ROI, MOI, OOI, and POI filters and verifying one or more of the objects that satisfy both the first and second ROI, MOI, OOI, and POI filters is an identical object detected by both the roadside camera and the range sensor.

8. The system of claim 7 , wherein the eighth control logic further comprises:
control logic that applies the correction factor to image data captured by the roadside camera, wherein the correction factor is based on known physical and geometrical locations of the roadside camera relative to known physical and geometrical locations of the range sensor, and takes a point location in a real-world coordinate system and translates the point to a camera coordinate system according to a calibration matrix K[I 3 |0 3 ] is a calibration matrix based on a world-homography matrix “H”:

1.



H
=

KR
[

I
3

|

-

C
˜

]

 where R is a rotation and C takes from the world coordinate system to the center of projection of the camera coordinate system;

2.



[

u
′

v
′

w
′

Assignees

• Gm Global Tech Operations Llc

Inventors

• Ali Kamran

Classifications

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• B60W2050/0088

  Adaptive recalibration · CPC title

• B60W2050/0052

  Filtering, filters · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• B60W50/06

  Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot · CPC title