Application of convolutional neural networks to object meshes

Patent metadata

Field	Value
Publication number	US-10643382-B2
Application number	US-201715478534-A
Country	US
Kind code	B2
Filing date	Apr 4, 2017
Priority date	Apr 4, 2017
Publication date	May 5, 2020
Grant date	May 5, 2020

Abstract

Convolutional Neural Networks are applied to object meshes to allow three-dimensional objects to be analyzed. In one example, a method includes performing convolutions on a mesh, wherein the mesh represents a three-dimensional object of an image, the mesh having a plurality of vertices and a plurality of edges between the vertices, performing pooling on the convolutions of an edge of a mesh, and applying fully connected and loss layers to the pooled convolutions to provide metadata about the three-dimensional object.

First claim

What is claimed is:

1. A method comprising:
determining a response value for each of a plurality of edges of a mesh representative of a three-dimensional (3D) object, wherein the mesh comprises a plurality of vertices and the plurality of edges between the vertices, and wherein a first response value for a first edge of the plurality of edges comprises one of a first angle between second and third edges of the mesh that meet at a first vertex of the first edge, a second angle between the second edge and a fourth edge extending from a second vertex of the first edge to a third vertex of the second edge, or a relative height of the third vertex orthogonally from the first edge;
performing one or more convolutions on the mesh including applying, for each of the plurality of edges, a convolutional kernel to a plurality of first order response values corresponding to each edge, wherein a first plurality of the first order response values for the first edge comprises the first response value followed by a second plurality of response values each corresponding to an edge extending from the first vertex or the second vertex;
performing one or more edge collapse operations after corresponding ones of the convolutions on the mesh; and
applying fully connected and loss layers to provide metadata corresponding to the 3D object.

2. The method of claim 1 , wherein the first plurality of the first order response values is in an order of the first response value, followed by a second response value corresponding to the third edge, followed by a third response value corresponding to a fifth edge of the plurality of edges extending from the second vertex to a fourth vertex of the third edge, followed by a fourth response value corresponding to the fourth edge, followed by a fifth response value corresponding to the second edge in response to a first mean response of the first vertex being greater than a second mean response of the second vertex.

3. The method of claim 1 , wherein performing the convolutions comprises applying, for each edge of the plurality of edges, a second convolutional kernel to a second order plurality of response values corresponding to each edge, wherein a first plurality of the second order plurality of response values for the first edge comprises the first response value followed by the second plurality of response values and followed by a third plurality of response values for each second edge extending from vertices corresponding to each edge extending from the first vertex or the second vertex.

4. The method of claim 3 , wherein performing the convolutions comprises applying, for each edge of the plurality of edges, a third convolutional kernel only to each response value corresponding to each edge.

5. The method of claim 1 , wherein performing the one or more edge collapse operations comprises:
collapsing the first edge to the first vertex to form a new edge and to eliminate the fourth edge; and
determining a new response value for the new edge based at least in part on a first order response value corresponding to the fourth edge.

6. The method of claim 5 , wherein the collapsing further eliminates a fifth edge of the plurality of edges extending from the second vertex to a fourth vertex of the third edge and wherein determining the new response value is based on a maximum or a median of the first order response value corresponding to the fourth edge and a first order response value corresponding to the fourth edge.

7. The method of claim 5 , wherein applying the convolutional kernel comprises multiplying each of a plurality of weights of the convolutional kernel with a corresponding one of the first response value or one of the second plurality of response values.

8. The method of claim 1 , further comprising:
applying one or more transformations to the mesh before performing said one or more convolutions on the mesh, wherein the one or more transformations comprise one or more of translating the mesh, a scaling of the mesh, or a re-meshing of the mesh.

9. The method of claim 1 , further comprising applying the metadata to a computer vision system for 3D object recognition.

10. A non-transitory machine-readable medium having instructions thereon that when operated on by the machine cause the machine to perform operations comprising:
determining a response value for each of a plurality of edges of a mesh representative of a three-dimensional (3D) object, wherein the mesh comprises a plurality of vertices and the plurality of edges between the vertices, and wherein a first response value for a first edge of the plurality of edges comprises one of a first angle between second and third edges of the mesh that meet at a first vertex of the first edge, a second angle between the second edge and a fourth edge extending from a second vertex of the first edge to a third vertex of the second edge, or a relative height of the third vertex orthogonally from the first edge;
performing one or more convolutions on the mesh including applying, for each of the plurality of edges, a convolutional kernel to a plurality of first order response values corresponding to each edge, wherein a first plurality of the first order response values for the first edge comprises the first response value followed by a second plurality of response values each corresponding to an edge extending from the first vertex or the second vertex;
performing one or more edge collapse operations after corresponding ones of the convolutions on the mesh; and
applying fully connected and loss layers to provide metadata corresponding to the 3D object.

11. The non-transitory machine-readable medium of claim 10 , wherein the first plurality of the first order response values is in an order of the first response value, followed by a second response value corresponding to the third edge, followed by a third response value corresponding to a fifth edge of the plurality of edges extending from the second vertex to a fourth vertex of the third edge, followed by a fourth response value corresponding to the fourth edge, followed by a fifth response value corresponding to the second edge in response to a first mean response of the first vertex being greater than a second mean response of the second vertex.

12. The non-transitory machine-readable medium of claim 10 , wherein performing the convolutions comprises applying, for each edge of the plurality of edges, a second convolutional kernel to a second order plurality of response values corresponding to each edge, wherein a first plurality of the second order plurality of response values for the first edge comprises the first response value followed by the second plurality of response values and followed by a third plurality of response values for each second edge extending from vertices corresponding to each edge extending from the first vertex or the second vertex.

13. The non-transitory machine-readable medium of claim 12 , wherein performing the convolutions comprises applying, for each edge of the plurality of edges, a third convolutional kernel only to each response value corresponding to each edge.

14. A computing system comprising:
a memory to store a mesh representative of a three-dimensional (3D) object, wherein the mesh comprises a plurality of vertices and the plurality of edges between the vertices;
a processor coupled to the memory, the processor to:
determine a response value for each of the plurality of edges, wherein a first response value for a first edge of the plurality of edges comprises one of a first angle between second and third edges of the mesh that meet at a first vertex of the first edge, a second angle bet

Assignees

• Intel Corp

Inventors

• Fleishman Shahar
• Kliger Mark

Classifications

• G06T17/205

  Re-meshing · CPC title

• G06T17/20 Primary

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06V10/82 Primary

  using neural networks · CPC title