Dymanic fusion method and device of images

Patent metadata

Field	Value
Publication number	US-9311818-B2
Application number	US-201414158224-A
Country	US
Kind code	B2
Filing date	Jan 17, 2014
Priority date	May 17, 2013
Publication date	Apr 12, 2016
Grant date	Apr 12, 2016

Abstract

A dynamic fusion method of images includes: receiving broadcast information from surrounding vehicles of a host vehicle; determining whether at least one of the surrounding vehicles travels in the same lane as the host vehicle to become a neighboring vehicle of the host vehicle according to the broadcast information; determining whether the neighboring vehicle is too close to the host vehicle and blocks the view of the host vehicle; and performing a transparentization or translucentization process on the neighboring vehicle in an image captured by the host vehicle when the neighboring vehicle blocks the view of the host vehicle.

First claim

What is claimed is:

1. A dynamic fusion method of images, comprising the steps of:
receiving broadcast information transmitted from surrounding vehicles of a host vehicle;
determining according to the broadcast information whether at least one of the surrounding vehicles travels longitudinally with the host vehicle in the same lane, and defining the at least one of the surrounding vehicles individually and correspondingly as at least one neighboring vehicle of the host vehicle;
determining whether a preceding vehicle from the at least one neighboring vehicle exists, wherein the preceding vehicle travels along just in front of the host vehicle and blocks a view of the host vehicle; and
performing a transparentization process on the preceding vehicle when the preceding vehicle is spaced from the host vehicle at a distance less than a predetermined minimum spacing.

2. The dynamic fusion method of claim 1 , wherein the transparentization process includes the steps of:
obtaining feature correspondences from the images captured by the preceding vehicle and the host vehicle so as to determine the two respective epipoles in the corresponding images and a scale factor defining a ratio relation of pixels near the two epipoles;
extracting an image near the epipole of the image of the preceding vehicle, and scaling down the extracted image according to the scale factor; and
blending the scaled image with pixels within a boundary near the epipole of the image of the host vehicle.

3. The dynamic fusion method of claim 2 , further including the step of:
while in obtaining the feature points from the respective images of the preceding vehicle and the host vehicle, locating a correspondence from the images, and calculating a corresponding relation between the feature correspondences with respect to the correspondence in the images of the two vehicles according to an equation of p′ t Fp=0, wherein the p is coordinate of the correspondence in the image of the preceding vehicle, the p′ is coordinate of the correspondence in the image of the host vehicle, and the F is a fundamental matrix.

4. The dynamic fusion method of claim 3 , wherein the equation is one of a scale-invariant feature transform (SIFT) algorithm and a speeded-up robust feature (SURF) algorithm.

5. The dynamic fusion method of claim 2 , further including the steps of:
estimating a fundamental matrix from the feature correspondences so as thereby to obtain the respective epipoles of the images of the preceding vehicle and the host vehicle and the ratio relation of the pixels near the two epipoles; and
extracting pixels around the epipole of the image of the preceding vehicle and pixels around the epipole inside the preceding vehicle body in the image of the host vehicle according to the ratio relation, and blending the extracted pixels in the image of the host vehicle with the ones in the image of the preceding vehicle, wherein a boundary defining the vehicle body is determined by a vehicle detection algorithm.

6. The dynamic fusion method of claim 2 , wherein the images of the preceding vehicle and the host vehicle are obtained by a plurality of image sensors.

7. The dynamic fusion method of claim 6 , further comprising:
obtaining a geometric relation between the images based on the feature correspondences of the images; and
blending the images of the preceding vehicle and the host vehicle according to the geometric relation.

8. The dynamic fusion method of claim 6 , wherein one of the image sensors is one of an infrared image capturing device, an optoelectronic coupling element and a metal-oxide complementary (MOS) photosensitive element.

9. The dynamic fusion method of claim 6 , wherein, when the image sensors are installed in the preceding vehicle and the host vehicle, the images are mutually transmitted via a wireless communication.

10. The dynamic fusion method of claim 9 , wherein the broadcast information or the images are transmitted via the wireless communication, and the wireless communication is one of Dedicated Short Range Communications/Wireless Access in a Vehicular Environment (DSRC/WAVE), a Wi-Fi, a WiMax and an LTE.

11. The dynamic fusion method of claim 5 , wherein the ratio relation is expressed by one of a linear formula and a non-linear formula.

12. The dynamic fusion method of claim 5 , wherein an origin of a coordinate system for elucidating the ratio relation is one of the epipole of the image of the host vehicle and an arbitrary point in the image of the host vehicle.

13. The dynamic fusion method of claim 1 , wherein the broadcast information includes image information and positioning information of the surrounding vehicles.

14. The dynamic fusion method of claim 1 , wherein a resulted level of the transparentization process is one of a transparent state, a semitransparent state, and a middle state between the transparent state and an opaque state.

15. A dynamic fusion device of images, comprising:
a capturing unit, for capturing surrounding images of a host vehicle;
a receiving unit, for receiving broadcast information transmitted from surrounding vehicles of the host vehicle;
a processing unit, coupled to the receiving unit and the capturing unit, for determining whether at least one of the surrounding vehicles travels longitudinally with the host vehicle in the same lane, for defining at least one of the surrounding vehicles individually and correspondingly as at least one neighboring vehicle of the host vehicle, for determining whether a preceding vehicle from the at least one neighboring vehicle exists, wherein the preceding vehicle travels along just in front of the host vehicle and blocks a view of the host vehicle, and for performing a transparentization process on the preceding vehicle when the preceding vehicle is spaced from the host vehicle at a distance less than a predetermined minimum spacing.

16. The dynamic fusion device of claim 15 , wherein the transparentization process performed by the processing unit includes the steps of:
obtaining feature correspondences from images captured by the preceding vehicle and the host vehicle so as to determine the two respective epipoles of the images and a scale factor defining a ratio relation of pixels near the two epipoles;
extracting an image near the epipole of the image of the preceding vehicle, and scaling down the extracted image according to the ratio factor; and
blending the scaled image with pixels within a boundary near the epipole of the image of the host vehicle.

17. The dynamic fusion device of claim 16 , wherein the processing unit further obtains the feature points from the respective images of the preceding vehicle and the host vehicle, matches the feature points in the images of the two vehicles to form feature correspondences according to a predetermined algorithm.

18. The dynamic fusion device of claim 16 , wherein the processing unit further includes
estimating a fundamental matrix based on the feature correspondences so as thereby to obtain the respective epipoles of the images of the preceding vehicle and the host vehicle and further extracting pixels defining the preceding vehicle body in the image of the host vehicle based on the ratio relation; wherein the extracted pixels defining the preceding vehicle body in the image of the host vehicle are blended with the corresponding pixels in the image of the preceding vehicle around the epipole thereof as a center, wherein a boundary for the blending is determined by a vehicle detection algorithm.

19. The dynamic fusion devic

Assignees

• Ind Tech Res Inst

Inventors

• Lin Che-Tsung
• Lin Yu-Chen

Classifications

• G06T2210/62

  Semi-transparency · CPC title

• G08G1/166 Primary

  for active traffic, e.g. moving vehicles, pedestrians, bikes · CPC title

• G06T19/00

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title

• G08G1/167

  Driving aids for lane monitoring, lane changing, e.g. blind spot detection · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title