Three-dimensional point cloud label learning estimation device, three-dimensional point cloud label learning estimation method, and 3D point cloud label learning estimation program

Patent metadata

Field	Value
Publication number	US-12067763-B2
Application number	US-201917612373-A
Country	US
Kind code	B2
Filing date	May 23, 2019
Priority date	May 23, 2019
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

A three-dimensional point cloud label learning and estimation device includes: a clustering unit that clusters a three-dimensional point cloud into clusters; a learning unit that makes a neural network learn to estimate a label corresponding to an object to which points contained in each of the clusters belong; and an estimation unit that estimates a label for the cluster using the neural network learned at the learning unit. In the three-dimensional point cloud label learning and estimation device, the neural network uses a total sum of sigmoid function values (sum of sigmoid) when performing feature extraction on the cluster.

First claim

The invention claimed is:

1. A three-dimensional point cloud label learning and estimation device comprising:
a cluster generator configured to cluster a three-dimensional point cloud into clusters;
a learner configured to cause a neural network to learn to estimate a label corresponding to an object to which points contained in each of the clusters belong; and
an estimator configured to estimate a label for the cluster using the neural network learned at the learning unit,
wherein the neural network uses a total sum of sigmoid function values when performing feature extraction on the cluster, wherein

during learning, the cluster generator outputs a labeled clustering result by performing clustering on a three-dimensional point cloud with application of learning point cloud labels and clustering parameters, the learning point cloud labels being labels previously assigned to respective points in the three-dimensional point cloud, and during estimation, the cluster generator performs clustering on a target three-dimensional point cloud with application of the clustering parameters and outputs an unlabeled clustering result,
the learner uses the labeled clustering result and deep neural network hyper-parameters to learn label estimation parameters for estimating labels to be assigned to respective clusters that result from the clustering at the cluster generator, and outputs learned deep neural network parameters, and
the estimator estimates a label for each cluster in the unlabeled clustering result by using the unlabeled clustering result, the deep neural network hyper-parameters, and the learned deep neural network parameters output by the learner.

2. The three-dimensional point cloud label learning and estimation device according to claim 1 , wherein
the cluster generator outputs three-dimensional attribute information for the points contained in the cluster and attribute information for a scalar of the cluster, and
the neural network is configured to:
use the three-dimensional attribute information for the points contained in the cluster and the attribute information for the scalar of the cluster as input information, and
subject the three-dimensional attribute information for the points contained in the cluster to geometric transformation.

3. The three-dimensional point cloud label learning and estimation device according to claim 2 , wherein the three-dimensional attribute information is a normal direction and a direction of extrusion of each of the points contained in the cluster.

4. The three-dimensional point cloud label learning and estimation device according to claim 2 , wherein
during learning, the cluster generator outputs a labeled clustering result by performing clustering on a three-dimensional point cloud with application of learning point cloud labels and clustering parameters, the learning point cloud labels being labels previously assigned to respective points in the three-dimensional point cloud, and during estimation, the cluster generator performs clustering on a target three-dimensional point cloud with application of the clustering parameters and outputs an unlabeled clustering result,
the learner uses the labeled clustering result and deep neural network hyper-parameters to learn label estimation parameters for estimating labels to be assigned to respective clusters that result from the clustering at the cluster generator, and outputs learned deep neural network parameters, and
the estimator estimates a label for each cluster in the unlabeled clustering result by using the unlabeled clustering result, the deep neural network hyper-parameters, and the learned deep neural network parameters output by the learner.

5. The three-dimensional point cloud label learning and estimation device according to claim 3 , wherein
during learning, the cluster generator outputs a labeled clustering result by performing clustering on a three-dimensional point cloud with application of learning point cloud labels and clustering parameters, the learning point cloud labels being labels previously assigned to respective points in the three-dimensional point cloud, and during estimation, the cluster generator performs clustering on a target three-dimensional point cloud with application of the clustering parameters and outputs an unlabeled clustering result,
the learner uses the labeled clustering result and deep neural network hyper-parameters to learn label estimation parameters for estimating labels to be assigned to respective clusters that result from the clustering at the cluster generator, and outputs learned deep neural network parameters, and
the estimator estimates a label for each cluster in the unlabeled clustering result by using the unlabeled clustering result, the deep neural network hyper-parameters, and the learned deep neural network parameters output by the learner.

6. A three-dimensional point cloud label learning and estimation device comprising:
a cluster generator configured to cluster a three-dimensional point cloud into clusters;
a learner configured to cause a neural network to learn to estimate a label corresponding to an object to which points contained in each of the clusters belong; and
an estimator configured to estimate a label for the cluster using the neural network learned at the learner,
wherein the cluster generator outputs three-dimensional attribute information for the points contained in the cluster and attribute information for a scalar of the cluster, and
the neural network is configured to:
take as input the three-dimensional attribute information for the points contained in the cluster and the attribute information for a scalar of the cluster output by the cluster generator, and

subject the three-dimensional attribute information for the points contained in the cluster to geometric transformation, wherein
during learning, the cluster generator outputs a labeled clustering result by performing clustering on a three-dimensional point cloud with application of learning point cloud labels and clustering parameters, the learning point cloud labels being labels previously assigned to respective points in the three-dimensional point cloud, and during estimation, the cluster generator performs clustering on a target three-dimensional point cloud with application of the clustering parameters and outputs an unlabeled clustering result,
the learner uses the labeled clustering result and deep neural network hyper-parameters to learn label estimation parameters for estimating labels to be assigned to respective clusters that result from the clustering at the cluster generator, and outputs learned deep neural network parameters, and
the estimator estimates a label for each cluster in the unlabeled clustering result by using the unlabeled clustering result, the deep neural network hyper-parameters, and the learned deep neural network parameters output by the learner.

7. The three-dimensional point cloud label learning and estimation device according to claim 6 , wherein
the cluster generator outputs three-dimensional attribute information for the points contained in the cluster and attribute information for a scalar of the cluster, and
the neural network is configured to:
use the three-dimensional attribute information for the points contained in the cluster and the attribute information for the scalar of the cluster as input information, and
subject the three-dimensional attribute information for the points contained in the cluster to geometric transformation.

8. The three-dimensional point cloud label learning and estimation device according to claim 7 , wherein
the three-dimensional attribute information is a normal direction and a direction of extrusion of each of the points containe

Assignees

• Nippon Telegraph & Telephone

Inventors

• Yao Yasuhiro
• Niigaki Hitoshi
• Kurata Kana
• Murasaki Kazuhiko
• Ando Shingo
• Sagata Atsushi

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06V10/762

  using clustering, e.g. of similar faces in social networks · CPC title