Sensor information fusion device and method thereof

Patent metadata

Field	Value
Publication number	US-11630199-B2
Application number	US-202017085713-A
Country	US
Kind code	B2
Filing date	Oct 30, 2020
Priority date	Jul 16, 2020
Publication date	Apr 18, 2023
Grant date	Apr 18, 2023

Abstract

A sensor information fusion device and a method thereof are provided. The sensor information fusion device includes a processor that generates a first track box and a second track box based on an object detected by a plurality of sensors and determines whether the first track box and the second track box are overlapped with each other and a storage storing data obtained by the processor and an algorithm run by the processor. The processor generates a merge gate expanded from the first track box and determines the first track box and the second track box are overlapped with each other when the second track box is included in the merge gate.

First claim

What is claimed is:

1. A sensor information fusion device, comprising:
a processor configured to:
when an object is detected by overlapping a plurality of track boxes through a plurality of sensors,
generate a first track box in which the object is detected by a first sensor among the plurality of sensors, and generate a second track box in which the object is detected by a second sensor among the plurality of sensors; and
determine whether the first track box and the second track box are overlapped with each other; and

a storage configured to store data obtained by the processor and to store a set of instructions that when executed by the processor, implement an algorithm,
wherein the processor is further configured to execute the set of instructions stored in the storage and to:
generate a merge gate expanded from the first track box; and
determine that the first track box and the second track box are overlapped with each other when the second track box is included in the merge gate.

2. The sensor information fusion device of claim 1 , wherein the processor is further configured to:
calculate midpoints of the first track box and the second track box based on information received from the plurality of sensors.

3. The sensor information fusion device of claim 2 , wherein the processor is further configured to:
convert the midpoint of the first track box into an origin; and
calculate coordinates of vertices of the first track box based on length information and width information of the first track box.

4. The sensor information fusion device of claim 3 , wherein the processor is further configured to:
convert the midpoint of the second track box as relative coordinates for the origin of the first track box by applying the midpoint of the first track box and the midpoint of the second track box to a rotation transformation formula.

5. The sensor information fusion device of claim 3 , wherein the processor is further configured to:
calculate coordinates of vertices of the second track box using the midpoint, length information, width information, and a heading value of the second track box.

6. The sensor information fusion device of claim 1 , wherein the processor is further configured to:
determine a size of the merge gate based on length information and width information of the first track box.

7. The sensor information fusion device of claim 1 , wherein the processor is further configured to:
determine whether the first track box and the second track box overlap with each other based on a box-in point function; and
determine whether the first track box and the second track box overlap with each other by applying a box crossed function to targets which are not determined by the box-in point function.

8. The sensor information fusion device of claim 1 , wherein the processor is further configured to:
determine that the first track box and the second track box overlap with each other when at least one of coordinates of vertices of the second track box is located in the first track box.

9. The sensor information fusion device of claim 1 , wherein the processor is further configured to:
divide a region outside the first track box into a plurality of regions when all of coordinates of vertices of the second track box are not located in the first track box; and
determine whether the first track box and the second track box overlap with each other based on a location of the coordinates when the coordinates of the vertices of the second track box are located in the plurality of regions.

10. The sensor information fusion device of claim 9 , wherein the processor is further configured to:
sequentially divide and define a region above the first track box as a first region, a second region, and a third region;
define a region to left of the first track box as a fourth region;
define a region to right of the first track box as a fifth region; and
sequentially divide and define a region below the first track box as a sixth region, a seventh region, and an eighth region.

11. The sensor information fusion device of claim 10 , wherein the processor is further configured to:
determine that the first track box and the second track box overlap with each other when at least one of the coordinates of the vertices of the second track box is present in the second region and the seventh region or is present in the fourth region and the fifth region.

12. The sensor information fusion device of claim 10 , wherein the processor is further configured to:
form a first triangle by a line extended after a first vertex of the second track box is connected with a first vertex of the first track box, a perpendicular line drawn from the first vertex of the second track box to the first track box, and a portion of a first surface of the first track box; and
form a second triangle by a line extended after a second vertex of the second track box is connected with the first vertex of the first track box, a perpendicular line drawn from the second vertex of the second track box to the first track box, and a portion of a second surface of the first track box, when at least one of the coordinates of the vertices of the second track box is not present in the second region and the seventh region and is not present in the fourth region and the fifth region.

13. The sensor information fusion device of claim 12 , wherein the processor is further configured to:
calculate a tangent value of the first triangle and a tangent value of the second triangle; and
compare the tangent value of the first triangle with the tangent value of the second triangle.

14. The sensor information fusion device of claim 13 , wherein the processor is further configured to:
determine that the first track box and the second track overlap with each other when the tangent value of the first triangle is less than the tangent value of the second triangle.

15. The sensor information fusion device of claim 1 , wherein the processor is further configured to:
select a track box for fusion depending on reliability of the first track box and reliability of the second track box.

16. The sensor information fusion device of claim 15 , wherein the processor is further configured to:
perform fusion using a track box with higher reliability when the reliability of the first track box and the reliability of the second track box have a difference value equal to or greater than a predetermined reference value.

17. The sensor information fusion device of claim 15 , wherein the processor is further configured to:
select a track box having an old generation time when the reliability of the first track box and the reliability of the second track box have a difference value less than a predetermined reference value.

18. The sensor information fusion device of claim 15 , wherein the processor is further configured to:
calculate the reliability of the first track box and the reliability of the second track box using reliability of each of sensors; and generate the first track box and the second track box and a time when the first track box and the second track box are generated.

19. The sensor information fusion device of claim 15 , wherein the processor is further configured to:
delete a track box which is not selected for fusion.

20. A sensor information fusion method, comprising:
when an object is detected by overlapping a plurality of track boxes through a plurality of sensors,
generating a first track box in which the object is detected by a first se

Assignees

• Hyundai Motor Co Ltd
• Kia Motors Corp

Inventors

• Kim Seong Hwan

Classifications

• B60W2554/00

  Input parameters relating to objects · CPC title

• G01S17/931

  of land vehicles · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G01S13/867 Primary

  Combination of radar systems with cameras · CPC title

• B60W2420/00

  Indexing codes relating to the type of sensors based on the principle of their operation · CPC title