Interactive object selection

Patent metadata

Field	Value
Publication number	US-11893313-B2
Application number	US-202017124452-A
Country	US
Kind code	B2
Filing date	Dec 16, 2020
Priority date	Dec 16, 2019
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

A computer-implemented method of machine-learning including obtaining a dataset of 3D point clouds. Each 3D point cloud includes at least one object. Each 3D point cloud is equipped with a specification of one or more graphical user-interactions each representing a respective selection operation of a same object in the 3D point cloud. The method further includes teaching, based on the dataset, a neural network configured for segmenting an input 3D point cloud including an object. The segmenting is based on the input 3D point cloud and on a specification of one or more input graphical user-interactions each representing a respective selection operation of the object in the 3D point cloud.

First claim

The invention claimed is:

1. A computer-implemented method of machine-learning, the method comprising:
obtaining a dataset of 3D point clouds, each 3D point cloud including at least one object, each 3D point cloud being equipped with a specification of one or more graphical user-interactions each representing a respective selection operation of a same object in the 3D point cloud; and
teaching, based on the dataset, a neural network configured for segmenting an input 3D point cloud including an object, the segmenting being based on the input 3D point cloud and on a specification of one or more input graphical user-interactions each representing a respective selection operation of the object in the 3D point cloud,
wherein the obtaining of the dataset of 3D point clouds further includes:
obtaining the 3D point clouds and, for each 3D point cloud, information about location of said same object in the 3D point cloud, and
for each 3D point cloud:
determining the specification by simulating each graphical user-interaction of the one or more graphical user-interactions, and
equipping the 3D point cloud with the specification of the one or more graphical user-interactions.

2. The method of claim 1 , wherein each graphical user-interaction of the one or more graphical user-interactions corresponds to one or more seed locations each defined over the 3D point cloud, and the simulating of the graphical user-interaction further includes determining the one or more seed locations.

3. The method of claim 2 , wherein the one or more graphical user-interactions include a first graphical user-interaction for selecting said same object and corresponding to one or more first seed locations each defined over said same object.

4. The method of claim 3 , wherein the determining of the one or more seed locations includes spreading the one or more first seed locations over said same object.

5. The method of claim 3 , wherein for at least one 3D point cloud, the one of more graphical user-interactions further include a second graphical user-interaction for discarding a region outside of said same object and corresponding to one or more second seed locations each defined outside said same object.

6. The method of claim 1 , wherein the equipping of the 3D point cloud includes, based on the simulating of each graphical user-interaction, adding to each point of the 3D point cloud a coordinate quantifying an intensity of the selection operation at the point.

7. The method of claim 6 , wherein each graphical user interaction of the one or more graphical user-interactions is for selecting said same object or for discarding a region outside of said same object, and for each point of the 3D point cloud the adding of the coordinate includes:
setting the coordinate to an initial value,
for each graphical user-interaction for selecting said same object, increasing the coordinate according to a closeness between the graphical user-interaction and the point, and
for each graphical user-interaction for discarding a region outside of said same object, decreasing the coordinate according to a closeness between the graphical user-interaction and the point.

8. A computer-implemented method of applying a neural network teachable by obtaining a dataset of 3D point clouds, each 3D point cloud including at least one object, each 3D point cloud being equipped with a specification of one or more graphical user-interactions each representing a respective selection operation of a same object in the 3D point cloud, and teaching, based on the dataset, a neural network configured for segmenting an input 3D point cloud including an object, the segmenting being based on the input 3D point cloud and on a specification of one or more input graphical user-interactions each representing a respective selection operation of the object in the 3D point cloud, the method comprising:
obtaining a 3D point cloud including an object; and
one or more iterations of:
performing a selection operation of the object by performing one or more graphical user-interactions, and
by applying the neural network, segmenting the 3D point cloud based on the 3D point cloud and on a specification of the one or more graphical user-interactions,
wherein the obtaining of the dataset of 3D point clouds further includes:
obtaining the 3D point clouds and, for each 3D point cloud, information about location of said same object in the 3D point cloud, and
for each 3D point cloud:
determining the specification by simulating each graphical user-interaction of the one or more graphical user-interactions, and
equipping the 3D point cloud with the specification of the one or more graphical user-interactions.

9. The method of claim 8 , further comprising, after the performing of the selection operation and before the applying of the neural network:
determining the specification by determining, for each graphical user-interaction of the one or more graphical user-interactions, positions of one or more seed locations defined by the graphical user-interaction; and
equipping the 3D point cloud with the specification, the equipping including adding to each point of the 3D point cloud a coordinate quantifying an intensity of the selection operation at the point.

10. The method of claim 8 , wherein the one or more graphical user-interactions include one or more of:
performing one or more clicks over the object,
performing a stroke over the object,
defining a bounding box over an object and/or around the object,
performing one or more clicks outside the object, and/or
performing a stroke outside the object.

11. A device comprising:
a processor; and
a non-transitory data storage medium having recorded thereon a neural network and a computer program,
wherein the non-transitory data storage medium includes instructions that when executed by the processor causes the processor to be configured to:
obtain a 3D point cloud including an object, and
in one or more iterations:
perform a selection operation of the object by performing one or more graphical user-interactions, and
by applying the neural network, segment the 3D point cloud based on the 3D point cloud and on a specification of the one or more graphical user-interactions, and/or

wherein the non-transitory data storage medium includes instructions that when executed by the processor causes the processor to be configured to:
teach by machine-learning the neural network by the processor being further configured to:
obtain a dataset of 3D point clouds, each 3D point cloud including at least one object, each 3D point cloud being equipped with a specification of one or more graphical user-interactions each representing a respective selection operation of a same object in the 3D point cloud, and
teach, based on the dataset, a neural network configured for segmenting an input 3D point cloud including an object, the segmenting being based on the input 3D point cloud and on a specification of one or more input graphical user-interactions each representing a respective selection operation of the object in the 3D point cloud,

wherein in the machine-learning, the processor is further configured to obtain the dataset of 3D point clouds by being configured to
obtain the 3D point clouds and, for each 3D point cloud, information about location of said same object in the 3D point cloud; and
for each 3D point cloud:
determine the specification by simulating each graphical user-interaction of the one or more graphical user-interactions, and
equip the 3D point cloud with the specification of the one or more graphical user-interactions.

12. The device of claim 11 , wherein in the machine-learning

Assignees

• Dassault Systemes

Inventors

• Rejeb Sfar Asma
• Durand Tom
• Boulkenafed Malika

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F30/10 Primary

  Geometric CAD · CPC title

• G06N3/08

  Learning methods · CPC title

• G06T7/11

  Region-based segmentation · CPC title