Methods and apparatus for surface classification

Patent metadata

Field	Value
Publication number	US-9482622-B2
Application number	US-201514677898-A
Country	US
Kind code	B2
Filing date	Apr 2, 2015
Priority date	May 16, 2014
Publication date	Nov 1, 2016
Grant date	Nov 1, 2016

Abstract

In illustrative implementations of this invention, light sources illuminate a surface with multi-spectral, multi-directional illumination that varies in direction, wavelength, coherence and collimation. One or more cameras capture images of the surface while the surface is illuminated under different lighting conditions. One or more computers take, as input, data indicative of or derived from the images, and determine a classification of the surface. Based on the computed classification, the computers output signals to control an I/O device, such that content displayed by the I/O device depends, at least in part, on the computed classification. In illustrative implementations, this invention accurately classifies a wide range of surfaces, including transparent surfaces, specular surfaces, and surfaces with few features.

First claim

What is claimed is:

1. A method comprising, in combination:
(a) a first set of one or more light sources illuminating an external surface of an object with coherent light during a first time period;
(b) a second set of one or more light sources illuminating the external surface with incoherent light during a second time period, such that the first time period and the second time period do not overlap;
(c) one or more cameras capturing images of the external surface
(i) during the first time period while the external surface is illuminated with the coherent light, and
(ii) during the second time period while the external surface is illuminated with the incoherent light; and

(d) one or more computers
(i) taking, as input, data indicative of or derived from the images,
(ii) determining a classification of the external surface, and
(iii) based on the classification, outputting signals to control an input/output (I/O) device, such that what content is displayed by the I/O depends, a least in part, on the classification;

wherein
(1) features are embedded in a transparent layer, which layer is illuminated by the coherent light during the first time period and by the incoherent light during the second time period,
(2) the features are invisible to an unaided human eye and have material properties that are different than the transparent layer, and
(3) the content that is displayed by the I/O device depends on data encoded by the features.

2. The method of claim 1 , wherein direction and spectrum of illumination produced by the first set of light sources vary over time.

3. The method of claim 1 , wherein:
(a) the one or more computers include a first computer and a second computer;
(b) at least the one or more cameras, the first and second sets of light sources, and the first computer are housed in a housing;
(c) the second computer is outside of the housing;
(d) the first computer takes, as an input, data from the one or more cameras, extracts parameter values from the data, and outputs the parameter values for wireless transmission to the second computer; and
(e) the second computer takes the parameter values as input, and computes a classification of the surface in real time.

4. The method of claim 1 , wherein:
(a) the one or more cameras include a first camera and a second camera;
(b) the first and second cameras capture images of the surface; and
(c) the first camera has a spectral sensitivity that is different than the spectral sensitivity of the second camera.

5. The method of claim 3 , wherein the parameters values, which are extracted by the first computer and are wirelessly transmitted to the second computer, include average pixel intensity of a frame.

6. The method of claim 1 , wherein the surface that is classified is specular) The method of claim 3 , wherein the parameter values, which are extracted from the data and are wirelessly transmitted to the second computer, include number of valid feature points in a frame.

7. An apparatus comprising, in combination:
(a) a first set of one or more coherent light sources for illuminating an external surface of an object with coherent light during a first time period;
(b) a second set of one or more incoherent light sources for illuminating the external surface with incoherent light during a second time period, such that the first time period and the second time period do not overlap;
(c) one or more cameras for capturing images of the external surface
(i) during the first time period while the external surface is illuminated with the coherent light, and
(ii) during the second time period while the external surface is illuminated with the incoherent light; and

(d) one or more computers programmed
(i) to take, as input, data indicative of or derived from the images, and
(ii) to determine a classification of the external surface, and based on the classification, to output signals to control an input/output (I/O) device, such that what content is displayed by the I/O device depends, at least in part, on the classification;

wherein
(1) features are embedded in a transparent layer, which layer is illuminated by the coherent light during the first time period and by the incoherent light during the second time period,
(2) the features are invisible to an unaided human eye and have material properties that are different than the transparent layer, and
(3) the content that is displayed by the I/O device depends on data encoded by the features.

8. The apparatus of claim 7 , wherein the one or more computers are programmed to cause direction and spectrum of illumination produced by the first set of light sources to vary over time.

9. The apparatus of claim 7 , wherein:
(a) the one or more computers include a first computer and a second computer;
(b) at least the one or more cameras, the first and second sets of light sources, and the first computer are housed in a housing;
(c) the second computer is outside of the housing;
(d) the first computer is programmed to extract parameter values from the images, and to output the parameter values for wireless transmission to the second computer; and
(e) the second computer is programmed to take the parameter values as input, and to compute a classification of the surface in real time.

10. The apparatus of claim 9 , wherein the parameter values, which are extracted from the images by the first computer and are wirelessly transmitted to the second computer, include average pixel intensity of a frame.

11. The apparatus of claim 9 , wherein the parameter values, which are extracted from the images by the first computer and are wirelessly transmitted to the second computer, include number of valid feature points in a frame.

12. The apparatus of claim 9 , wherein, for a given frame, the parameter values, which are extracted from the images by the first computer and are wirelessly transmitted to the second computer:
(a) are represented by four bytes of wirelessly transmitted data; and
(b) include (i) average pixel intensity of the given frame, (ii) darkest pixel of the given frame, (iii) brightest pixel of the given frame, and (iv) number of valid feature points in the given frame.

Assignees

• Sato Munehiko
• Raskar Ramesh
• Shi Boxin
• Olwal Alex
• Massachusetts Inst Technology

Inventors

• Sato Munehiko
• Raskar Ramesh
• Shi Boxin
• Olwal Alex

Classifications

• G01N2021/8444

  Fibrous material · CPC title

• G01N2021/3155

  Measuring in two spectral ranges, e.g. UV and visible · CPC title

• G01N2201/062

  LED's · CPC title

• G01N2201/12

  Circuits of general importance; Signal processing · CPC title

• G01N21/251 Primary

  Colorimeters; Construction thereof · CPC title