Enhanced camera object detection for automated vehicles

Patent metadata

Field	Value
Publication number	US-10366310-B2
Application number	US-201715680854-A
Country	US
Kind code	B2
Filing date	Aug 18, 2017
Priority date	Sep 12, 2016
Publication date	Jul 30, 2019
Grant date	Jul 30, 2019

Abstract

An illustrative example object detection system includes a camera having a field of view. The camera provides an output comprising information regarding potential objects within the field of view. A processor is configured to select a portion of the camera output based on information from at least one other type of detector that indicates a potential object in the selected portion. The processor determines an Objectness of the selected portion based on information in the camera output regarding the selected portion.

First claim

We claim:

1. An object detection system, comprising:
a camera having a field of view, the camera providing an output comprising information regarding potential objects within the field of view; and
a processor that is configured to select a portion of the camera output based on information from at least one other type of detector that indicates a potential object in the selected portion, the processor being configured to ignore other portions of the camera output that, based on the information from the at least one other type of detector, do not include a potential object, the processor determining an Objectness of the selected portion based on information in the camera output regarding the selected portion.

2. The object detection system of claim 1 , wherein the processor is configured to
locate a plurality of segments in the selected portion;
divide at least one of the segments into patches; and
determine the Objectness of each of the patches, respectively.

3. The object detection system of claim 2 , wherein
the processor is configured to determine a total Objectness of an entire at least one of the segments; and
the processor is configured to determine the Objectness of the at least one of the segments based on the Objectness of each of the patches and the total Objectness.

4. The object detection system of claim 1 , wherein
the processor is configured to divide up the selected portion into segments; and
the processor is configured to arrange the segments based on an Objectness of the respective segments.

5. The object detection system of claim 4 , wherein the segments include at least one segment having a first geometry and at least one other segment having a second, different geometry.

6. The object detection system of claim 5 , wherein at least the first geometry corresponds to a distribution of data points from the at least one other type of detector within the at least one segment.

7. The object detection system of claim 1 , comprising the at least one other type of detector and wherein the at least one other type of detector comprises one of a radar detector or a LIDAR detector.

8. The object detection system of claim 1 , wherein
the processor is configured to recognize a clustered set of data points from the at least one other type of detector; and
the processor is configured to select at least one clustered set of data points as the selected portion.

9. The object detection system of claim 1 , wherein
the at least one other type of detector provides a LIDAR output having an intensity; and
the processor determines the Objectness from at least one of the camera output and the intensity of the LIDAR output.

10. The object detection system of claim 1 , wherein
the camera output comprises a plurality of images;
the plurality of images are in a time-based sequence;
the processor is configured to use the plurality of images to determine motion cues corresponding to movement of a potential object in the selected portion; and
the processor is configured to use the motion cues when determining the Objectness of the selected portion.

11. The object detection system of claim 1 , wherein
the processor is configured to determine a respective Objectness of a plurality of segments of the selected portion;
the processor is configured to rank the Objectness of each of the segments; and
the processor is configured to select a highest ranked Objectness to identify a location of a potential object.

12. The object detection system of claim 1 , wherein the processor is configured to provide an object location estimation within an identified area of the selected portion of the camera output.

13. A method of detecting at least one potential object, the method comprising:
selecting a portion of a camera output based on information from at least one other type of detector that indicates a potential object in the selected portion;
ignoring other portions of the camera output that, based on the information from the at least one other type of detector, do not include a potential object; and
determining an Objectness of the selected portion based on information in the camera output regarding the selected portion.

14. The method of claim 13 , comprising
dividing up the selected portion into segments; and
determining the Objectness of each of the segments, respectively.

15. The method of claim 14 , comprising
dividing at least one of the segments into a plurality of patches;
determining a total Objectness of the entire at least one of the segments;
determining an Objectness of each of the patches; and
determining the Objectness of the at least one of the segments based on the Objectness of each of the patches and the total Objectness.

16. The method of claim 13 , wherein
dividing up the selected portion into segments comprises configuring respective geometries of the segments based on information from the at least one other type of detector;
the segments include at least one segment having a first geometry and at least one other segment having a second, different geometry; and
at least the first geometry corresponds to a distribution of data points from the at least one other type of detector within the at least one segment.

17. The method of claim 13 , wherein selecting the portion of the camera output comprises recognizing a clustered set of data points from the at least one other type of detector.

18. The method of claim 13 , wherein the camera output comprises a plurality of images in a time-based sequence and the method comprises
using the plurality of images to determine movement cues of a potential object in the selected portion; and
using the movement cues to determine the Objectness of the selected portion.

19. The method of claim 13 , comprising
determining a respective Objectness of a plurality of segments of the selected portion;
ranking the Objectness of each of the segments; and
selecting a highest ranked Objectness to identify a location of a potential object.

20. The method of claim 13 , comprising providing an object location estimation within an identified area of the selected portion of the camera output.

21. The method of claim 13 , wherein
the at least one other type of detector provides a LIDAR output having an intensity; and
determining the Objectness comprises using at least one of the camera output and the intensity of the LIDAR output.

Assignees

• Aptiv Tech Ltd

Inventors

• Zermas Dimitris
• Izzat Izzat H
• Mangalgiri Anuradha

Classifications

• G01S13/867 Primary

  Combination of radar systems with cameras · CPC title

• G06F18/2323

  based on graph theory, e.g. minimum spanning trees [MST] or graph cuts · CPC title

• G06T7/248

  involving reference images or patches · CPC title

• G06T2207/30261

  Obstacle · CPC title

• G01S17/86

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title