Extracting features from queued radar frames

Patent metadata

Field	Value
Publication number	US-2025321321-A1
Application number	US-202418634244-A
Country	US
Kind code	A1
Filing date	Apr 12, 2024
Priority date	Apr 12, 2024
Publication date	Oct 16, 2025
Grant date	—

Abstract

A computerized technique is disclosed of identifying object features in an environment of a vehicle. The technique includes receiving, by an encoder, data representing a plurality of frames, the frames providing point-in-time versions of a segmented pointed cloud derived from output of one or more radar sensors of the vehicle and including points that represent radar detections corresponding to an object in the environment at respective instants in time. The technique further includes arranging the plurality of frames in a time-ordered queue and processing the frames in the queue, including (i) selecting, from among the points, a plurality of sample points that spans multiple frames of the queue, (ii) forming a plurality of groups of points based on respective sample points of the plurality of sample points, and (iii) extracting features of the object based on the plurality of sample points and the plurality of groups.

First claim

What is claimed is:

1 . A device comprising control circuitry that includes a set of processors coupled to memory, the control circuitry constructed and arranged to perform a method of identifying object features in an environment of a vehicle, the method including:
receiving, by an encoder that runs on the control circuitry, data representing a plurality of frames, the frames of the plurality of frames providing point-in-time versions of a segmented pointed cloud derived from output of one or more radar sensors of the vehicle and including points that represent radar detections corresponding to an object in the environment at respective instants in time; arranging the plurality of frames in a time-ordered queue; and processing the frames in the queue, including (i) selecting, from among the points, a plurality of sample points that spans multiple frames of the queue, (ii) forming a plurality of groups of points based on respective sample points of the plurality of sample points, and (iii) extracting features of the object based on the plurality of sample points and the plurality of groups.

2 . The device of claim 1 , wherein the method further includes providing the extracted features of the object to a classification head constructed and arranged to classify the object as one of a plurality of object types, the object types including one or more of (i) pedestrians, (ii) bicyclists, or (iii) motorcyclists.

3 . The device of claim 1 , wherein selecting the plurality of sample points includes performing a farthest point sampling (FPS), the FPS including searching for a next sample point of the plurality of sample points based on distances of other points of the plurality of frames from a current sample point of the plurality of sample points, wherein the distances are based on both spatial offsets and temporal offsets.

4 . The device of claim 3 , wherein the method further includes determining the distances of the other points from the current sample point of the plurality of sample points based on the spatial offsets and the temporal offsets, wherein determining the distances includes weighting contributions of the spatial offsets and temporal offsets using at least one tunable parameter.

5 . The device of claim 3 , wherein performing the FPS includes limiting a temporal search range within which the next sample point is selected, such that points from at least one frame in the queue are excluded as candidates for the next sample point.

6 . The device of claim 5 , wherein the method further includes detecting a speed of motion of the object relative to the vehicle and increasing the temporal search range responsive to the speed falling below a threshold speed.

7 . The device of claim 5 , wherein limiting the temporal search range further includes:
(i) searching in a first direction only until a sample point is selected from a first end frame of the queue; (ii) then searching only in the first end frame for a single sample point; (iii) then searching in a second direction opposite the first direction until a sample point is selected from a second end frame of the queue opposite the first end frame; and (iv) then searching only in the second end frame for a single sample point.

8 . The device of claim 3 , wherein selecting the plurality of sample points includes selecting fewer than all of the points in the frames of the queue.

9 . The device of claim 1 , wherein forming the plurality of groups includes providing a respective group for each of the plurality of sample points, and wherein at least one group includes points from multiple frames.

10 . The device of claim 9 , further comprising limiting frames from which points may be selected for a group to fewer than all frames in the queue.

11 . The device of claim 10 , wherein forming the plurality of groups further includes:
limiting candidate points within a current frame that may be selected for inclusion in a particular group to points within a first spatial radius of a current sample point; and limiting candidate points within a time-adjacent frame that may be selected for inclusion in the particular group to points within a second spatial radius of the current sample point, wherein the first spatial radius is larger than the second spatial radius.

12 . The device of claim 1 , wherein selecting the plurality of sample points is performed by a sampling component, wherein forming the plurality of groups is performed by a grouping component, and wherein the method further includes:
storing the points and associated attributes in an array in computer memory, the array having different indices for respective points, and identifying, by the sampling component, the plurality of sample points to the grouping component by providing array indices of the plurality of sample points but not by providing the associated attributes.

13 . The device of claim 1 , wherein extracting features of the object includes:
constructing a tensor having a first dimension for different sample points of the plurality of sample points, a second dimension for points per group of the plurality of groups, and a third dimension for attributes of points within the groups; and providing the tensor as input to a neural network trained to identify object features from sample points, groups, and attributes.

14 . The device of claim 1 , wherein the method further includes adding frames to the queue until a total number of points in the frames of the queue meets a minimum limit.

15 . The device of claim 1 , wherein the method further includes removing at least one oldest frame from the queue responsive to a total number of points in the frames of the queue exceeding a maximum limit.

16 . The device of claim 1 , wherein the encoder is a hierarchical encoder that includes multiple encoder stages, the stages including:
a first encoder stage constructed and arranged to extract features of the object on a first spatial scale; and a second encoder stage cascaded with the first encoder stage, the second encoder stage constructed and arranged to extract features of the object on a second spatial scale different from the first spatial scale and to receive features of the object extracted by the first encoder stage as inputs.

17 . A computer-implemented method of identifying object features in an environment of a vehicle, comprising:
receiving data representing a plurality of frames derived from output of one or more radar sensors of the vehicle, the frames of the plurality of frames providing point-in-time versions of a segmented pointed cloud derived from output of one or more radar sensors of the vehicle and including points that represent radar detections corresponding to an object in the environment at respective instants in time; arranging the plurality of frames in a time-ordered queue; and processing the frames in the queue, including (i) selecting, from among the points, a plurality of sample points that spans multiple frames of the queue, (ii) forming a plurality of groups of points based on respective sample points of the plurality of sample points, and (iii) extracting features of the object based on the plurality of sample points and the plurality of groups.

18 . The method of claim 17 , wherein processing the frames in the queue includes:
extracting, by a first encoder stage, features of the object on a first spatial scale; providing the features of the object extracted by the first encoder stage as inputs to a second encoder stage cascaded with the first encoder stage; and operating the

Assignees

• Nxp Bv

Inventors

• Chen Lihui
• Ravindran Satish
• Hassan Syed Mujtaba
• Wu Ryan Haoyun

Classifications

• G01S13/931 Primary

  of land vehicles · CPC title

• G01S7/41

  using analysis of echo signal for target characterisation; Target signature; Target cross-section · CPC title

• G01S7/295

  Means for transforming co-ordinates or for evaluating data, e.g. using computers · CPC title

• G01S7/2923 Primary

  based on data belonging to a number of consecutive radar periods · CPC title

• G01S13/42

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S13/46) · CPC title