Method for harmonizing images acquired from non overlapping camera views

Patent metadata

Field	Value
Publication number	US-11832019-B2
Application number	US-202217821874-A
Country	US
Kind code	B2
Filing date	Aug 24, 2022
Priority date	Aug 24, 2021
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

An image processing method for harmonizing images acquired by a first camera and a second camera connected to a vehicle and arranged in such a way as their fields of view cover a same road space at different times as the vehicle travels along a travel direction is disclosed. The method includes: acquiring by a selected camera, a first image at a first time; selecting a first region of interest bounding a road portion from the first image; sampling the first region of interest; acquiring by the other camera, a second image in such a way that the road portion is included in a second region of interest; sampling the second region of interest; and determining one or more correction parameters for harmonizing images acquired by the first and second cameras, based on a comparison between the image content of the first and second regions of interest.

First claim

The invention claimed is:

1. An image processing method for harmonizing images acquired by a first camera and a second camera connected to a vehicle and arranged in such a way as their fields of view cover a same road space at different times as the vehicle travels along a travel direction, the method comprising:
determining the travel direction of the vehicle;
based on the travel direction, selecting one of the first and second cameras to acquire a first image;
acquiring, by the selected camera, the first image at a first time;
selecting at least one first region of interest of the first image, the first region of interest potentially bounding a road portion;
sampling the at least one first region of interest from the first image;
monitoring a distance travelled by the vehicle along the travel direction after the first time to determine a second time to acquire, by the other camera, a second image, in such a way that the potential road portion is included in a second region of interest of the second image;
acquiring, by the other camera, the second image at the second time;
sampling the second region of interest from the second image;
verifying whether both the first and second regions of interest include a road portion; and
responsive to a verification that both the first and second regions of interest include the road portion, determining one or more correction parameters for harmonizing images acquired by the first and second cameras, based on a comparison between the image content of the first and second regions of interest.

2. The method of claim 1 , wherein the first camera is a rear camera of the vehicle and the second camera is a rear camera of a trailer connected to the vehicle via connecting means.

3. The method of claim 2 , where in response to determining that the vehicle is travelling in a forward direction, selecting the first camera to acquire the first image at the first time, and wherein the or each at least one first region of interest of the first image corresponds to a region known to comprise a road space between the trailer and the vehicle.

4. The method of claim 3 , wherein the or each at least one first region of interest of the first image is beside a region bounding the connecting means.

5. The method of claim 4 , wherein selecting the at least one first region of interest comprises:
defining at least two first regions of interest in the first image, one at a first side of the region bounding the connecting means, and the other at a second side of the region bounding the connecting means, when the trailer is aligned to the vehicle along a longitudinal axis;
determining whether any of the two first regions of interest includes a portion of at least one of the trailer and the connecting means;
responsive to a determination that none of the two first regions of interest include a portion of at least one of the trailer and the connecting means, selecting both the two first regions of interest to be sampled from the first image; and
responsive to a determination that one of the two first regions of interest includes a portion of at least one of the trailer and the connecting means, selecting the other first region of interest to be sampled from the first image.

6. The method of claim 5 , wherein determining whether any of the first two regions of interest includes a portion of at least one of the trailer and the connecting means comprises:
detecting whether a portion of at least one of the trailer and the connecting means is included in any of the first two regions of interest.

7. The method of claim 5 , comprising:
measuring at least one of: a steering angle of the vehicle; and a hitch angle between the vehicle and the trailer; and
wherein determining whether any of the two first regions of interest includes a portion of at least one of the trailer and the connecting means is based on the at least one measured steering angle and hitch angle.

8. The method of claim 2 , wherein in response to determining that the vehicle is travelling in a reverse direction, selecting the second camera to acquire the first image at the first time.

9. The method of claim 8 , wherein the potential road portion corresponding to the at least one first region of interest of the first image becomes viewable within the field of view of the first camera as the vehicle travels in the reverse direction.

10. The method of claim 9 , wherein selecting the at least one first region of interest of the first image comprises:
defining at least two first regions of interest, one to include a corresponding first road portion that can become viewable within the field of view of the first camera at a first side of the region bounding the connecting means, and the other at a second side of the region bounding the connecting means, when the trailer is aligned to the vehicle along a longitudinal axis;
measuring at least one of: a steering angle of the vehicle; and a hitch angle between the vehicle and the trailer;
in response to determining that the at least one measured steering angle and hitch angle is below a threshold, selecting both the two first regions of interest to be sampled from the first image; and
in response to determining that the at least one measured steering angle and hitch angle exceeds the threshold, selecting one of the two regions of interest based on the at least one measured steering angle and hitch angle.

11. The method of claim 1 , wherein the first camera and the second camera are a front camera and a rear camera of the vehicle.

12. The method of claim 1 , comprising:
converting the image data within the first and second regions of interest into a YUV format;
wherein determining whether both the first and second regions of interest include the road portion comprises:
estimating at least one first luma component from at least a portion of the image data within the first region of interest, and at least one second luma component from at least a portion of the image data within the second region of interest;
comparing the difference between the estimated first and second luma components with a threshold; and
responsive to a determination that the difference is below the threshold, verifying that both the first and second regions of interest include the road portion.

13. The method of claim 12 , further comprising responsive to the verification that both the first and second regions of interest include the road portion:
determining one or more brightness correction parameters for harmonizing brightness of images acquired by the first and second cameras, based on the difference between the estimated first and second luma components;
estimating first chrominance components from at least a portion of the image data within the first region of interest and second chrominance components from at least a portion of the image data within the second region of interest; and
determining one or more chrominance correction parameters for harmonizing the colour of images acquired by the first and second cameras, based on a difference between the estimated first and second chrominance components.

14. The method of claim 1 , further comprising:
generating a combined view including merged images acquired by the first and second cameras, including applying the determined one or more harmonization correction parameters.

15. An automotive multi-camera vision system comprising:
first and second cameras connected to a vehicle and arranged in such a way as their fields of view cover a same road space at different times as the vehicle travels along a travel direction; and
one or more processing unit

Assignees

• Connaught Electronics Ltd

Inventors

• Griffin Mark
• Casey Aidan
• Turgay Emre
• Perkins Alex

Classifications

• H04N5/265 Primary

  Mixing · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• H04N5/268

  Signal distribution or switching · CPC title

• H04N9/646

  for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters · CPC title