Systems, methods, and media for semantic segmentation of a point cloud frame

Patent metadata

Field	Value
Publication number	US-12394227-B2
Application number	US-202217944936-A
Country	US
Kind code	B2
Filing date	Sep 14, 2022
Priority date	Sep 14, 2021
Publication date	Aug 19, 2025
Grant date	Aug 19, 2025

Abstract

A computer-implemented method, a computing system, and a non-transitory machine-readable medium for semantic segmentation of a point cloud frame are provided. Point cloud frames including a target point cloud frame are received. For each sequence of a sliding set of sequences of point cloud frames, the sequence including the target point cloud frame, each point cloud frame in the sequence of point cloud frames is semantically segmented to apply semantic labels to points. A most prevalent semantic label is determined for each point in the target point cloud frame across the sliding set of sequences of point cloud frames.

First claim

The invention claimed is:

1. A computer-implemented method for semantic segmentation of a point cloud frame, comprising:
receiving point cloud frames including a target point cloud frame;
for each sequence of a sliding set of sequences of point cloud frames, the sequence including the target point cloud frame, semantically segmenting each point cloud frame in the sequence of point cloud frames to apply semantic labels to points; and
determining a most prevalent semantic label for each point in the target point cloud frame across the sliding set of sequences of point cloud frames.

2. The computer-implemented method of claim 1 , further comprising:
identifying a rare semantic label for at least one of the points in the target point cloud frame across the sliding set of sequences of point cloud frames; and
using the rare semantic label in place of the most prevalent semantic label for the at least one point in the target point cloud frame.

3. The computer-implemented method of claim 1 , further comprising:
performing a single-scan semantic segmentation of the target point cloud frame; and
using ensemble learning to combine the single-scan semantic segmentation with the most prevalent labels for each point in the target point cloud frame across the sliding set of sequences of point cloud frames.

4. The computer-implemented method of claim 2 , further comprising:
performing a single-scan semantic segmentation of the target point cloud frame; and
using ensemble learning to combine the single-scan semantic segmentation with the rare semantic labels and the most prevalent labels for each point in the target point cloud frame across the sliding set of sequences of point cloud frames.

5. The computer-implemented method of claim 1 , wherein the semantically segmenting includes:
voxelizing each of the point cloud frames in the sequence of point cloud frames;
generating a 4D tensor from the voxelized point cloud frames; and
processing the 4D tensor to identify features in the point cloud frames.

6. The computer-implemented method of claim 5 , wherein the processing includes:
for each point in the point cloud frames, finding k nearest neighbors; and
determining a semantic label for the point at least partially based on the features of the k nearest neighbors.

7. The computer-implemented method of claim 6 , wherein the determining includes:
calculating a weight for each of the k nearest neighbors at least partially based on a Euclidian distance between the point and the nearest neighbor.

8. The computer-implemented method of claim 7 , wherein, during the calculating of the Euclidian distance, a unit of distance is used to compensate for a unit of time between point cloud frames.

9. The computer-implemented method of claim 8 , wherein, during the determining, the weights are used to determine a weighted sum of the features of the nearest neighbors.

10. The computer-implemented method of claim 9 , wherein the weighted sum of the features of the nearest neighbors is used in combination with the features of the point to semantically segment the point.

11. A computing system for semantic segmentation of a point cloud frame, the computing system comprising:
a processor;
a memory storing machine-executable instructions that, when executed by the processor, cause the processor to:
receive point cloud frames including a target point cloud frame;
for each sequence of a sliding set of sequences of point cloud frames, the sequence including the target point cloud frame, semantically segment each point cloud frame in the sequence of point cloud frames to apply semantic labels to points; and
determine a most prevalent semantic label for each point in the target point cloud frame across the sliding set of sequences of point cloud frames.

12. The computing system of claim 11 , wherein the machine-executable instructions, when executed by the processor, cause the processor to:
identify a rare semantic label for at least one of the points in the target point cloud frame across the sliding set of sequences of point cloud frames; and
use the rare semantic label in place of the most prevalent semantic label for the at least one point in the target point cloud frame.

13. The computing system of claim 11 , wherein the machine-executable instructions, when executed by the processor, cause the processor to:
perform a single-scan semantic segmentation of the target point cloud frame; and
use ensemble learning to combine the single-scan semantic segmentation with the most prevalent labels for each point in the target point cloud frame across the sliding set of sequences of point cloud frames.

14. The computing system of claim 12 , wherein the machine-executable instructions, when executed by the processor, cause the processor to:
perform a single-scan semantic segmentation of the target point cloud frame; and
use ensemble learning to combine the single-scan semantic segmentation with the rare semantic labels and the most prevalent labels for each point in the target point cloud frame across the sliding set of sequences of point cloud frames.

15. The computing system of claim 11 , wherein the machine-executable instructions, when executed by the processor, cause the processor to, during semantic segmentation:
voxelize each of the point cloud frames in the sequence of point cloud frames;
generate a 4D tensor from the voxelized point cloud frames; and
process the 4D tensor to identify features in the point cloud frames.

16. The computing system of claim 15 , wherein the machine-executable instructions, when executed by the processor, cause the processor to, during the processing:
for each point in the point cloud frames, find k nearest neighbors; and
determine a semantic label for the point at least partially based on the features of the k nearest neighbors.

17. The computing system of claim 16 , wherein the machine-executable instructions, when executed by the processor, cause the processor to, during the determining:
calculate a weight for each of the k nearest neighbors at least partially based on a Euclidian distance between the point and the nearest neighbor.

18. The computing system of claim 17 , wherein the machine-executable instructions, when executed by the processor, cause the processor to, during the calculation of the Euclidian distance, use a unit of distance to compensate for a unit of time between point cloud frames.

19. The computing system of claim 18 , wherein the machine-executable instructions, when executed by the processor, cause the processor to, during the determining, use the weights to determine a weighted sum of the features of the nearest neighbors.

20. The computing system of claim 19 , wherein the weighted sum of the features of the nearest neighbors is used in combination with the features of the point to semantically segment the point.

Assignees

• Li Xinhai
• Ren Yuan
• Cheng Ran
• Liu Bingbing
• Huawei Tech Co Ltd

Inventors

• Li Xinhai
• Ren Yuan
• Cheng Ran
• Liu Bingbing

Classifications

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/11

  Region-based segmentation · CPC title