Material classification using multiview capture

Patent metadata

Field	Value
Publication number	US-9613300-B2
Application number	US-201514742644-A
Country	US
Kind code	B2
Filing date	Jun 17, 2015
Priority date	Jun 17, 2015
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

Multiple pixels are selected from one or more images of an object fabricated from an unknown material captured from one or more viewing directions and one or more trained classification engines are applied to the selected pixels so as to obtain initial estimates of the material at each selected pixel. The one or more trained classification engines are each trained at a training direction and are each characterized by performance characteristics that vary based at least in part on directional deviation from that training direction. A posterior probability for the collection of selected multiple pixels is calculated, wherein the posterior probability is calculated based at least in part on the performance characteristics and directional deviation of a surface normal of the object at a selected pixel from the one or more training directions. The material of the object is classified based on the calculated posterior probability.

First claim

What is claimed is:

1. A method for material classification of an object fabricated from an unknown material, comprising:
selecting multiple pixels from a first image of the object captured at a first viewing direction;
applying a first trained classification engine to the selected pixels so as to obtain an initial estimate of material at each selected pixel, wherein the first trained classification engine is trained at a first training direction, and wherein the first trained classification engine is characterized by performance characteristics that vary based at least in part on directional deviation from the first training direction;
calculating a posterior probability for the collection of selected multiple pixels, wherein the posterior probability is calculated based at least in part on the performance characteristics and a directional deviation of a surface normal of the object at a selected pixel from the first training direction; and
classifying material of the object based on the calculated posterior probability.

2. The method according to claim 1 , wherein the material of the object is classified by maximizing the calculated posterior probability.

3. The method according to claim 1 , wherein the selected pixels are selected from the first image of the object at spatially random positions.

4. The method according to claim 1 , wherein the selected pixels are selected from the first image of the object such that the pixels are spatially independent.

5. The method according to claim 1 , wherein a predetermined number of selected pixels are selected, the predetermined number being chosen so that an a priori estimate of the posterior probability is relatively high.

6. The method according to claim 1 , wherein the first image of the object is captured under the same illumination conditions as when the first classification engine was trained and the first viewing direction is aligned with the first training direction.

7. The method according to claim 1 , wherein calculation of the posterior probability comprises:
determining a surface normal for each selected pixel;
estimating a likelihood probability of the initial estimate of material at each selected pixel based on the directional deviation of the determined surface normal from the first training direction;
deducing a joint likelihood probability for the collection of selected pixels from the estimated likelihood probabilities at the selected pixels, and
applying Bayes' rule to the deduced joint likelihood probability, yielding the posterior probability.

8. The method according to claim 1 , wherein the first trained classification engine is optimized for a surface normal that is aligned with the first training direction.

9. The method according to claim 1 , wherein the first trained classification engine is optimized for a surface normal that is aligned with a fixed direction different from the first training direction.

10. The method according to claim 1 , further comprising capturing multiple images of the object illuminated from respectively different ones of multiple light sources; and
estimating the directional deviation of the surface normal of the object at each of the selected pixels from the first training direction using photometric stereo methodology based on the multiple captured images.

11. The method according to claim 1 , further comprising assigning a confidence level to the classified material.

12. The method according to claim 11 , further comprising the capture of additional images from different viewing directions responsive to a determination that the assigned confidence level is below a predetermined acceptance value.

13. The method according to claim 1 , further comprising:
selecting multiple pixels from a second image of the object captured at a second viewing direction different from the first viewing direction; and
applying a second trained classification engine to the selected pixels of the second image so as to obtain initial estimates of material at each selected pixel of the second image;
wherein the posterior probability is calculated for the collection of both of the selected pixels of the first image and the selected pixels of the second image.

14. The method according to claim 13 , wherein the second trained classification engine is trained at a second training direction, and wherein the second trained classification engine is characterized by performance characteristics that vary based at least in part on directional deviation from the second training direction; and
wherein the posterior probability is calculated using a combination of initial estimates of material from the first trained classification engine and initial estimates of material from the second trained classification engine.

15. The method according to claim 1 , wherein the first trained classification engine is trained on one or more captured images of one or more objects fabricated from a known material oriented in alignment with the first training direction.

16. The method according to claim 1 , wherein the first trained classification engine classifies material by a material property based on the bidirectional reflectance distribution function (BRDF).

17. The method according to claim 1 , wherein the performance characteristics are defined using a confusion matrix.

18. The method according to claim 1 , wherein the performance characteristics are defined for a range of angles relative to the first training direction.

19. The method according to claim 1 , wherein the performance characteristics are predetermined based on applying the first trained classification engine to pixels of multiple images of one or more objects fabricated from a known material oriented at respectively multiple angles relative to the first training direction.

20. An apparatus for material classification of an object fabricated from an unknown material, comprising:
a selection unit configured to select multiple pixels from a first image of the object captured at a first viewing direction;
an application unit configured to apply a first trained classification engine to the selected pixels so as to obtain an initial estimate of material at each selected pixel, wherein the first trained classification engine is trained at a first training direction, and wherein the first trained classification engine is characterized by performance characteristics that vary based at least in part on directional deviation from the first training direction;
a calculation unit configured to calculate a posterior probability for the collection of selected multiple pixels, wherein the posterior probability is calculated based at least in part on the performance characteristics and a directional deviation of a surface normal of the object at a selected pixel from the first training direction; and
a classification unit configured to classify material of the object based on the calculated posterior probability, wherein
the selection unit, the application unit, the calculation unit and the classification unit, are all implemented by at least one processor which executes computer-executable program steps stored on at least one non-transitory computer-readable storage medium.

21. A non-transitory computer-readable storage medium storing a program for causing a computer to implement a method for material classification of an object fabricated from an unknown material, the method comprising the steps of
selecting multiple pixels from a first image of the object captured at a first viewi

Assignees

• Canon Kk

Inventors

• Tin Siu-Kei
• Skaff Sandra

Classifications

• G06V10/811

  the classifiers operating on different input data, e.g. multi-modal recognition · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F18/24155

  Bayesian classification · CPC title