Object tracking using sensor fusion within a probabilistic framework

Patent metadata

Field	Value
Publication number	US-2018126984-A1
Application number	US-201615346210-A
Country	US
Kind code	A1
Filing date	Nov 8, 2016
Priority date	Nov 8, 2016
Publication date	May 10, 2018
Grant date	—

Abstract

A controller receives outputs form a plurality of sensors such as a camera, LIDAR sensor, RADAR sensor, and ultrasound sensor. Sensor outputs corresponding to an object are assigned to a tracklet. Subsequent outputs by any of the sensors corresponding to that object are also assigned to the tracklet. A trajectory of the object is calculated from the sensor outputs assigned to the tracklet, such as by means of Kalman filtering. For each sensor output assigned to the tracklet, a probability is updated, such as using a Bayesian probability update. When the probability meets a threshold condition, the object is determined to be present and an alert is generated or autonomous obstacle avoidance is performed with respect to an expected location of the object.

First claim

1 . A method comprising performing, by a vehicle controller:
receiving a plurality of sensor outputs each indicating presence of an object, the plurality of sensors having a plurality of different sensing modalities; updating a probability according to each sensor output of the plurality of sensor outputs; and determining that the probability is sufficiently high to perform collision avoidance with respect to the object.

2 . The method of claim 1 , wherein the plurality of different sensing modalities include at least two of a two-dimensional camera image, radio distancing and ranging (RADAR), light distancing and ranging (LIDAR), and ultrasound.

3 . The method of claim 1 , further comprising, for each output of the plurality of sensor outputs:
identifying features in the each output; determining that one of the features corresponds to the object; and assigning the feature to a tracklet representing the object.

4 . The method of claim 3 , further comprising determining a trajectory for the tracklet.

5 . The method of claim 4 , further comprising determining the trajectory for the tracklet by performing Kalman filtering with respect to a plurality of features assigned to the tracklet.

6 . The method of claim 3 , further comprising updating the probability according to each sensor output of the plurality of sensor outputs by updating a probability associated with the tracklet for each feature assigned to the tracklet.

7 . The method of claim 6 , wherein updating the probability comprises performing a Bayesian probability update for each feature assigned to the tracklet.

8 . The method of claim 3 , further comprising:
creating the tracklet in response to detecting a minimum number of contiguous sensor outputs of the plurality of sensor outputs indicating presence of the object.

9 . The method of claim 3 , further comprising:
creating the tracklet in response to both of (a) detecting a minimum number of contiguous sensor outputs of the plurality of sensor outputs indicating presence of the object and (b) the minimum number of contiguous sensor outputs indicating movement of the object that is consistent with expected object behavior.

10 . The method of claim 1 , further comprising:
actuating at least one of a steering actuator, an accelerator actuator, and a brake actuator effective to avoid the object in response to determining that the probability is sufficiently high to perform collision avoidance with respect to the object.

11 . A system comprising:
a plurality of sensors having a plurality of sensing modalities; a vehicle controller operably coupled to the plurality of sensors, the vehicle controller programmed to—
receive a plurality of sensor outputs from the plurality of sensors;
identify a portion of the plurality of sensor outputs indicate presence of an object;
update a probability according to each sensor output of the portion of the plurality of sensor outputs; and
if the probability meets a threshold condition, determine that the object is present.

12 . The system of claim 11 , wherein the plurality of sensors include at least two of a two-dimensional camera, a radio distancing and ranging (RADAR) sensor, a light distancing and ranging (LIDAR) sensor, and an ultrasonic sensor.

13 . The system of claim 11 , wherein the vehicle controller is further programmed to:
identify a plurality of features in each output of the plurality of sensor outputs; and if a feature of the plurality of features indicates presence of the object, assign the feature to a tracklet representing the object.

14 . The system of claim 13 , wherein the vehicle controller is further programmed to determine a trajectory for the tracklet according to features assigned to the tracklet.

15 . The system of claim 14 , wherein the vehicle controller is further programmed to determine the trajectory for the tracklet by performing Kalman filtering with respect to features assigned to the tracklet.

16 . The system of claim 13 , wherein the vehicle controller is further programmed to update the probability for each feature assigned to the tracklet.

17 . The system of claim 16 , wherein the vehicle controller is further programmed to update the probability by performing a Bayesian probability update for each feature assigned to the tracklet.

18 . The system of claim 13 , wherein the vehicle controller is further programmed to create the tracklet in response to detecting a minimum number of contiguous sensor outputs of the plurality of sensor outputs indicating presence of the object.

19 . The system of claim 13 , wherein the vehicle controller is further programmed to create the tracklet in response to both of (a) detecting a minimum number of contiguous sensor outputs of the plurality of sensor outputs indicating presence of the object and (b) the minimum number of contiguous sensor outputs indicating movement of the object that is consistent with expected object behavior.

20 . The system of claim 11 , wherein the vehicle controller is further programmed to, if the probability meets the threshold condition, actuate at least one of a steering actuator, an accelerator actuator, and a brake actuator effective to avoid the object.

Assignees

• Ford Global Tech Llc

Inventors

• Liu Dongran
• Kadetotad Sneha
• Gerardo Castro Marcos Paul
• Jain Jinesh J

Classifications

• G06F18/253

  of extracted features · CPC title

• B60W10/20

  including control of steering systems · CPC title

• B60W10/18

  including control of braking systems · CPC title

• B60W2420/54

  Audio sensitive means, e.g. ultrasound · CPC title

• B60W30/0956 Primary

  the prediction being responsive to traffic or environmental parameters · CPC title