Clustering in automotive imaging

Patent metadata

Field	Value
Publication number	US-2023139751-A1
Application number	US-202117907390-A
Country	US
Kind code	A1
Filing date	May 28, 2021
Priority date	May 31, 2020
Publication date	May 4, 2023
Grant date	—

Abstract

In an imaging system comprising an imaging sensor generating successive point clouds from detected objects, tracking points of interest, or targets, across multiple point clouds/frames can be performed to enable robust object detection by clustering the targets based on one or more tracking parameters. An imaging sensor may comprise a radar sensor or lidar sensor, and tracking the one or more parameters of a target may be performed by a state model of the target.

First claim

What is claimed is:

1 . A method of clustering targets detected by an imaging sensor in support of object detection, the method implemented on a device and comprising:
determining a target in a point cloud, wherein the point cloud is generated by the imaging sensor and corresponds to a sensor image; tracking one or more parameters of the target across a plurality of point clouds, wherein the one or more parameters are indicative of one or more characteristics of the target; and including the target in a cluster of targets based on the tracking, wherein the cluster indicates a detected object.

2 . The method of claim 1 , wherein the imaging sensor comprises a radar sensor or a lidar sensor.

3 . The method of claim 1 , further comprising providing an output indicative of the detected object.

4 . The method of claim 1 , wherein the one or more parameters comprise:
a Cartesian parameter comprising a Cartesian position, a velocity, or an acceleration of the target, or a combination thereof; a polarization parameter comprising a radial position or a radial speed of the target, or a combination thereof; an energy measurement of the target; or a time duration during which the target is detected; or a combination thereof.

5 . The method of claim 1 , wherein tracking one or more parameters of the target across the plurality of point clouds comprises tracking a change in a measured value of the target in multiple successive sensor images.

6 . The method of claim 1 , wherein tracking one or more parameters of the target is based, at least in part, on measurements of a parameter of the target, wherein the measurements correspond to a predefined number of successive sensor images.

7 . The method of claim 1 , wherein tracking one or more parameters of the target comprises updating a state model of the target, wherein the state model comprises:
an alpha-beta-gamma filter, a Kalman filter, an unscented Kalman filter, an extended Kalman filter, a converted measurements Kalman filter, an adaptive Kalman filter, a likelihood of a tracker hypothesis, or a particle filter tracker, or a combination thereof.

8 . The method of claim 7 , wherein including the target in a cluster of targets based on tracking comprise including the target in a cluster of targets based on the one or more characteristics of the target provided by the state model, the one or more characteristics comprising:
a position of the target, a radial speed of the target, a gain of the target, a position change per sensor image, a rate of the position change, a radial speed change per sensor image, a rate of the radial speed change, a gain change per sensor image, a rate of the gain change, or a time duration during which the target is detected, or a combination thereof.

9 . The method of claim 8 , wherein the state model comprises a Kalman filter, and wherein the method further comprises providing input to the state model, the input comprising:
a position innovation, a radial speed innovation, a gain innovation, a position covariance matrix, a radial speed covariance matrix, or a gain covariance matrix, or a combination thereof.

10 . The method of claim 8 , wherein the state model comprises an unscented Kalman filter, and wherein the method further comprises providing input to the state model, the input comprising:
a position innovation, a radial speed innovation, a gain innovation, a position covariance matrix, a radial speed covariance matrix, a gain covariance matrix, or a set of points that represents posterior mean and covariance of the position, or a combination thereof.

11 . The method of claim 8 , wherein the state model comprises an unscented particle filter tracker, and wherein the method further comprises providing input to the state model, the input comprising:
a position innovation, a radial speed innovation, a gain innovation, a position covariance matrix, a radial speed covariance matrix, a gain covariance matrix, or an uncertainty distribution using weighted particles of the one or more parameters, or a combination thereof.

12 . The method of claim 1 , wherein tracking one or more parameters of the target across the plurality of point clouds is performed at a first frame rate, and wherein the including the target in the cluster of targets is performed at a second frame rate.

13 . The method of claim 1 , wherein tracking the one or more parameters of the target comprises determining a characteristic, the characteristic comprising:
a position of the target, a speed of the target, a gain of the target, or a time period during which the target is detected, or a combination thereof.

14 . The method of claim 13 , wherein tracking the one or more parameters of the target further comprises determining a change in the characteristic.

15 . The method of claim 14 , wherein tracking one or more parameters of the target comprises updating a state model by storing values, the values comprising a value per imaging frame of the characteristic, and a value per imaging frame of the change in characteristic.

16 . The method of claim 15 , wherein:
the cluster of targets comprises a one or more additional targets; and including the target in the cluster of targets comprises clustering the target with the one or more additional targets based on a similarity between the stored values and corresponding values of the one or more additional targets.

17 . The method of claim 1 , wherein determining the target in the point cloud comprises determining to track a data point in the point cloud based on a parameter of the data point in the point cloud, a parameter of the data point across multiple point clouds, or both, wherein the data point corresponds to the target.

18 . The method of claim 1 , wherein the imaging sensor further performs one or more super resolution techniques on sensor data to generate the point cloud.

19 . A device for clustering targets detected by an imaging sensor in support of object detection, the device comprising:
a memory; and one or more processors communicatively coupled with the memory, wherein the one or more processors are configured to:
determine a target in a point cloud, wherein the point cloud is generated by the imaging sensor and corresponds to a sensor image;
track one or more parameters of the target across a plurality of point clouds, wherein the one or more parameters are indicative of one or more characteristics of the target; and
include the target in a cluster of targets based on the tracking, wherein the cluster indicates a detected object.

20 . The device of claim 19 , wherein the imaging sensor comprises a radar sensor or a lidar sensor.

21 . The device of claim 19 , wherein the device further comprises the imaging sensor.

22 . The device of claim 19 , wherein the device is further configured to provide an output indicative of the detected object.

23 . The device of claim 19 , wherein the one or more processors, to track the one or more parameters of the target across the plurality of point clouds, are configured to track a change in a measured value of the target in multiple successive sensor images.

24 . The device of claim 19 , wherein the one or more processors are configured to track the one or more parameters of the target based, at least in part, on measurements of a parameter of the target, whe

Assignees

• Qualcomm Inc

Inventors

• Sanderovich Amichai
• Hof Eran
• Radovsky Olga

Classifications

• G06T7/277 Primary

  involving stochastic approaches, e.g. using Kalman filters · CPC title

• G01S2013/9319

  Controlling the accelerator · CPC title

• G01S17/86

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G01S13/89 Primary

  for mapping or imaging · CPC title