3D image labeling method based on labeling information of 2D image and 3D image labeling device

Patent metadata

Field	Value
Publication number	US-11100702-B2
Application number	US-201916728358-A
Country	US
Kind code	B2
Filing date	Dec 27, 2019
Priority date	Dec 27, 2019
Publication date	Aug 24, 2021
Grant date	Aug 24, 2021

Abstract

A 3D image labeling device and a 3D image labeling method are provided. The 3D image labeling device includes a point cloud cluster module, a projection module, an integration module and a point cloud recovery module. The point cloud cluster module clusters multiple points of a 3D unlabeled image as one or more first point clusters according to a 3D unlabeled image and a cluster algorithm to generate a 3D clustered image. The projection module generates a first 2D image with first objects according to the 3D clustered image, wherein each first point cluster corresponds to one of the first objects. The integration module labels the first objects of the first 2D image according to one or more object frames of a 2D labeled image to generate a second 2D image. The point cloud recovery module generates a 3D labeled image according to the second 2D image.

First claim

What is claimed is:

1. A 3D image labeling device, comprising:
a processor; and
a memory, configured to store a point cloud cluster module, a projection module, an integration module and a point cloud recovery module;
wherein the point cloud cluster module, when executed by the processor, causes the processor to cluster a plurality of points of a 3D unlabeled image as one or more first point clusters according to the 3D unlabeled image and a cluster algorithm to generate a 3D clustered image;
wherein the projection module, when executed by the processor, causes the processor configured to generate a first 2D image with one or more first objects according to the 3D clustered image, wherein each of the one or more first point clusters corresponds to one of the one or more first objects;
wherein the integration module, when executed by the processor, causes the processor to label the one or more first objects of the first 2D image according to one or more object frames of one or more 2D labeled images to generate a second 2D image where capture time associated with the first 2D image is same as capture time associated with the one or more 2D labeled images, and the labeling of the one or more first objects of the first 2D image according to one or more object frames of one or more 2D labeled images to generate the second 2D image includes:
for each of the one or more first objects of the first 2D image, determine whether a respective object of the one or more first objects falls within a range of space corresponding to one of the one or more object frames of the one or more 2D labeled images, wherein when the determination is affirmative, the respective object is labeled according to a category of the corresponding object frame; otherwise, the respective object is not labeled, and the second 2D image is generated;

wherein the point cloud recovery module, when executed by the processor, causes the processor to generate a 3D labeled image according to the second 2D image.

2. The 3D image labeling device according to claim 1 , wherein the cluster algorithm is based on a density of the plurality of points of the 3D unlabeled image.

3. The 3D image labeling device according to claim 1 , wherein for each of a plurality of objects of the second 2D image with a respective object category, the point cloud recovery module, when executed by the processor, further causes the processor to label all points of the 3D clustered image corresponding to the one or more first point clusters of a respective object of the one or more first objects according to the respective object category of each of the plurality of objects of the second 2D image.

4. The 3D image labeling device according to claim 1 , wherein each of the one or more first point clusters corresponds to a piece of an actual object.

5. A 3D image labeling method, comprising the steps of:
executing, by a processor, a point cloud cluster module to cluster a plurality of points of a 3D unlabeled image as one or more first point clusters according to the 3D unlabeled image and a cluster algorithm to generate a 3D clustered image;
executing, by a processor, a projection module to generate a first 2D image with one or more first objects according to the 3D clustered image, wherein each of the one or more first point clusters corresponds to one of the one or more first objects;
executing, by a processor, an integration module to label the one or more first objects of the first 2D image according to one or more object frames of one or more 2D labeled images to generate a second 2D image where capture time associated with the first 2D image is same as capture time associated with the one or more 2D labeled images, wherein the labeling of the one or more first objects of the first 2D image according to one or more object frames of one or more 2D labeled images to generate the second 2D image includes:
for each of the one or more first objects of the first 2D image, determine whether a respective object of the one or more first objects falls within a range of space corresponding to one of the one or more object frames of the one or more 2D labeled images, wherein when the determination is affirmative, the respective object is labeled according to a category of the corresponding object frame; otherwise, the respective object is not labeled, and the second 2D image is generated; and
executing, by a processor, a point cloud recovery module to generate a 3D labeled image according to the second 2D image.

6. The 3D image labeling method according to claim 5 , wherein the cluster algorithm is based on a density of the plurality of points of the 3D unlabeled image.

7. The 3D image labeling method according to claim 5 , wherein the method comprising further steps of:
executing, by a processor, the point cloud recovery module to, for each of a plurality of objects of the second 2D image with a respective object category, label all points of the 3D clustered image corresponding to the one or more first point clusters of a respective object of the one or more first objects according to the respective object category of each of the plurality of objects of the second 2D image.

8. The 3D image labeling method according to claim 5 , wherein each of the one or more first point clusters corresponds to a piece of an actual object.

Assignees

• Ind Tech Res Inst

Inventors

• Yang Chung-Hsien
• Hsu Bo-Chun
• Cheng Chih-Hao
• Huang Po-Sheng
• Chan Chih-Hsiang

Classifications

• G06V20/647 Primary

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V10/762

  using clustering, e.g. of similar faces in social networks · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title