Prediction of target object's behavior based on world and image frames

Patent metadata

Field	Value
Publication number	US-11845439-B2
Application number	US-202117449409-A
Country	US
Kind code	B2
Filing date	Sep 29, 2021
Priority date	Sep 29, 2021
Publication date	Dec 19, 2023
Grant date	Dec 19, 2023

Abstract

A method includes obtaining sensor data associated with a target object at an ego vehicle, where the sensor data includes image data within an image frame. The method also includes identifying a first longitudinal distance of a first point of the target object from the ego vehicle within a world frame. The method further includes identifying a first lateral distance of the first point of the target object from the ego vehicle within the world frame using a first normalization ratio that is based on the image data. The method also includes predicting a future path of the target object based on the first longitudinal distance and the first lateral distance. In addition, the method includes controlling at least one operation of the ego vehicle based on the predicted future path of the target object.

First claim

What is claimed is:

1. A method comprising:
obtaining sensor data associated with a target object at an ego vehicle, the sensor data comprising image data within an image frame;
identifying a first longitudinal distance and a first lateral distance of a first point of the target object from the ego vehicle as measured within a world frame;
identifying a first modified lateral distance of the first point of the target object from the ego vehicle as measured within the world frame using a first normalization ratio and the first lateral distance, the first normalization ratio determined using (i) a distance as measured in pixels in the image frame of the first point of the target object from a lane-marking line and (ii) a width as measured in pixels in the image frame of a traffic lane associated with the target object or the ego vehicle, the traffic lane partially defined by the lane-marking line;
predicting a future path of the target object based on the first longitudinal distance and the first modified lateral distance; and
controlling at least one operation of the ego vehicle based on the predicted future path of the target object.

2. The method of claim 1 , further comprising:
identifying a second longitudinal distance and a second lateral distance of a second point of the target object from the ego vehicle as measured within the world frame; and
identifying a second modified lateral distance of the second point of the target object from the ego vehicle as measured within the world frame using a second normalization ratio that is based on the image data and the second lateral distance;
wherein the future path of the target object is predicted based on the first and second longitudinal distances and the first and second modified lateral distances.

3. The method of claim 2 , wherein the second normalization ratio is determined using (i) a distance as measured in pixels of the second point of the target object from the lane-marking line and (ii) the width as measured in pixels of the traffic lane associated with the target object or the ego vehicle.

4. The method of claim 2 , wherein:
the sensor data further comprises a radio detection and ranging (RADAR) measurement identifying a relative distance and a relative angle of the first point of the target object with respect to the ego vehicle; and
identifying the first longitudinal distance and the first lateral distance comprises decomposing the relative distance into the first longitudinal distance and a third the first lateral distance, the first longitudinal distance associated with a smaller error than the first lateral distance.

5. The method of claim 4 , wherein:
the first normalization ratio comprises a first unit-less normalization ratio applied to a width of the traffic lane as measured within the world frame in order to identify the first modified lateral distance; and
the second normalization ratio comprises a second unit-less normalization ratio applied to the width of the traffic lane as measured within the world frame in order to identify the second modified lateral distance.

6. The method of claim 2 , wherein:
the first point of the target object represents a rear corner of a target object; and
the second point of the target object represents a front corner of the target object.

7. The method of claim 1 , wherein controlling the at least one operation of the ego vehicle comprises at least one of:
adjusting at least one of: a steering of the ego vehicle, a speed of the ego vehicle, and a braking of the ego vehicle; and
activating an audible, visible, or haptic warning.

8. The method of claim 1 , wherein the first modified lateral distance of the first point of the target object from the ego vehicle is determined using:
the first normalization ratio;
a width of the traffic lane as measured within the world frame; and
a lateral offset of the target object from a longitudinal dimension of the world frame at the first longitudinal distance as measured in the world frame.

9. The method of claim 1 , wherein the first modified lateral distance is determined using a formula of:
y _( k )= L _( k )/ L t_ ( k ) ·L t ( k )+Δ y ( x )

wherein:
y_(k) represents the first modified lateral distance;
L_(k) represents the distance as measured in pixels of the first point of the target object from the lane-marking line;
L t_ (k) represents the width as measured in pixels of the traffic lane associated with the target object or the ego vehicle;
L t (k) represents a width of the traffic lane as measured within the world frame; and
Δ y (x) represents the first lateral offset of the target object from a longitudinal dimension of the world frame at the first longitudinal distance x_ as measured in the world frame.

10. The method of claim 1 , wherein identifying the first modified lateral distance comprises multiplying the first normalization ratio by one of:
a width of a first traffic lane associated with the target object as measured within the world frame when the width of the first traffic lane is accurately measured; or
a width of a second traffic lane associated with the ego vehicle as measured within the world frame when the width of the first traffic lane cannot be accurately measured and assuming that the first and second traffic lanes are relatively equal in width.

11. An apparatus associated with an ego vehicle, the apparatus comprising:
at least one processing device configured to:
obtain sensor data associated with a target object, the sensor data comprising image data within an image frame;
identify a first longitudinal distance and a first lateral distance of a first point of the target object from the ego vehicle as measured within a world frame;
identify a first modified lateral distance of the first point of the target object from the ego vehicle as measured within the world frame using a first normalization ratio and the first lateral distance, the first normalization ratio determined using (i) a distance as measured in pixels in the image frame of the first point of the target object from a lane-marking line and (ii) a width as measured in pixels in the image frame of a traffic lane associated with the target object or the ego vehicle, the traffic lane partially defined by the lane-marking line;
predict a future path of the target object based on the first longitudinal distance and the first modified lateral distance; and
control at least one operation of the ego vehicle based on the predicted future path of the target object.

12. The apparatus of claim 11 , wherein:
the at least one processing device is further configured to:
identify a second longitudinal distance and a second lateral distance of a second point of the target object from the ego vehicle as measured within the world frame; and
identify a second modified lateral distance of the second point of the target object from the ego vehicle as measured within the world frame using a second normalization ratio that is based on the image data and the second lateral distance; and

the future path of the target object is based on the first and second longitudinal distances and the first and second modified lateral distances.

13. The apparatus of claim 12 , wherein the second normalization ratio is determined using (i) a distance as measured in pixels of the second point of the target object from the lane-marking line and (ii) the width as measured in pixels of the traffic lane associated with the target object or the ego vehicle.

14. The apparatus of claim 12 , wherein:
the sensor data further comprises a radio detection and ranging (RADAR) measurement ide

Assignees

• Canoo Tech Inc

Inventors

• Kim Kilsoo
• Choi Jongmoo
• Sattiraju Mayukh
• Uppalapati Dheemanth
• Darji Dhruvil
• Agarwal Siddharth
• Bhumireddy Shanmukha M
• Singh Aviral K

Classifications

• B60W40/02 Primary

  related to ambient conditions · CPC title

• B60W10/18

  including control of braking systems · CPC title

• B60W10/20

  including control of steering systems · CPC title

• B60W30/143

  Speed control (B60W30/16 takes precedence) · CPC title

• B60W50/0097

  Predicting future conditions · CPC title