Specular reflection removal in time-of-flight camera apparatus

Patent metadata

Field	Value
Publication number	US-10234561-B2
Application number	US-201615150291-A
Country	US
Kind code	B2
Filing date	May 9, 2016
Priority date	May 9, 2016
Publication date	Mar 19, 2019
Grant date	Mar 19, 2019

Abstract

A method for facilitating removal of specular reflection noise from light data can include illuminating, using an illumination unit, a target with a light source. The illumination unit is configured to project light with a spatial light pattern onto the target. The method can also include acquiring, with a sensor unit, light data that is reflected from the target. The light data may comprise a directly reflected spatial light pattern and a specular reflected spatial light pattern. The directly reflected spatial light pattern and the specular reflected spatial light pattern comprise at least one spatial distinction that distinguishes the directly reflected spatial light pattern from the specular reflected spatial light pattern. The method can further comprise processing the light data to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern based upon the at least one spatial distinction.

First claim

We claim:

1. A time-of-flight camera apparatus configured to facilitate removal of specular reflection noise from light data, the apparatus comprising:
an illumination unit for illuminating a target with a light source, wherein:
the illumination unit is configured to project light with at least one spatial light pattern onto the target, wherein the at least one spatial light pattern comprises a pattern that is oriented according to a known direction;

a sensor unit for acquiring light data that is reflected from the target, wherein:
the light data comprises a directly reflected spatial light pattern oriented in a first direction that corresponds to the known direction, and a specular reflected spatial light pattern comprises light data oriented in a second direction that is different from the first direction, and
the difference between the first direction and the second direction comprises at least one spatial distinction that distinguishes the directly reflected spatial light pattern from the specular reflected spatial light pattern; and

a processing unit for processing the light data, wherein:
the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern based upon the at least one spatial distinction.

2. The time-of-flight camera apparatus as recited in claim 1 , wherein the spatial light pattern corresponds with the directly reflected spatial light pattern.

3. The time-of-flight camera apparatus as recited in claim 1 , wherein the directly reflected spatial light pattern comprises parallel lines extending in the first direction.

4. The time-of-flight camera apparatus as recited in claim 3 , wherein:
the specular reflected spatial light pattern comprises parallel lines extending in the second direction; and
the at least one spatial distinction comprises the second direction being different than the first direction.

5. The time-of-flight camera apparatus as recited in claim 4 , wherein the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern by applying at least one directional filter.

6. The time-of-flight camera apparatus as recited in claim 5 , wherein the at least one directional filter includes at least two directional filters, the at least two directional filters including a high-pass filter and a low pass filter.

7. The time-of-flight camera apparatus as recited in claim 1 , wherein the at least one spatial light pattern is a pattern of dots.

8. The time-of-flight camera apparatus as recited in claim 7 , wherein a dot detection or intensity thresholding algorithm is applied to the data.

9. The time-of-flight camera apparatus as recited in claim 7 , wherein a comparison to a known spatial relationship between dots in the directly reflected spatial light is used to isolate directly reflected spatial light from specular reflected spatial light.

10. The time-of-flight camera apparatus as recited in claim 1 , wherein the at least one spatial distinction used to isolate directly reflected spatial light from specular reflected spatial light is a relative time-of-flight.

11. The time-of-flight camera apparatus as recited in claim 1 , wherein the time-of-flight camera utilizes a processing algorithm to distinguish between the at least one spatial light pattern and a set of affine transformations of the at least one spatial light pattern.

12. The time-of-flight camera apparatus as recited in claim 1 , wherein the time-of-flight camera apparatus comprises a Lidar.

13. A computer system for facilitating removal of specular reflection noise from light data, comprising:
one or more processors; and
one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to perform at least the following:
illuminate, using an illumination unit, a target with a light source, wherein the illumination unit is configured to project light with a spatial light pattern onto the target, wherein the spatial light pattern comprises a pattern that is oriented according to a known direction;
acquire, with a sensor unit, light data that is reflected from the target, wherein the light data comprises:
time-of-flight data for light that illuminated the target,
a directly reflected spatial light pattern that comprises light oriented in a first direction that corresponds to the known direction, and a specular reflected spatial light pattern that comprises light oriented in a second direction that is different from the first direction, and
the difference between the first direction and the second direction comprises at least one spatial distinction that distinguishes the directly reflected spatial light pattern from the specular reflected spatial light pattern;

generate specular-reflection-compensated light data by processing, with the one or more processors, the light data, wherein the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern based upon the at least one spatial distinction; and
determine, with the one or more processors and based upon the specular-reflection-compensated light data, one or more distances that extend between the light source and one or more portions of the target.

14. The computer system as recited in claim 13 , wherein the spatial light pattern corresponds with the directly reflected spatial light pattern.

15. The computer system as recited in claim 13 , wherein the directly reflected spatial light pattern comprises parallel lines extending in the first direction.

16. The computer system as recited in claim 15 , wherein:
the specular reflected spatial light pattern comprises parallel lines extending in the second direction; and
the at least one spatial distinction comprises the second direction being different than the first direction.

17. The computer system as recited in claim 16 , wherein the processing unit is configured to distinguish the directly reflected spatial light pattern from the specular reflected spatial light pattern by applying at least one directional filter.

18. The computer system as recited in claim 13 , wherein a processing algorithm is used to distinguish between the spatial light pattern and a set of affine transformations of the spatial light pattern.

19. The computer system as recited in claim 13 , wherein the at least one spatial distinction used to isolate directly reflected spatial light from specular reflected light is a relative time-of-flight.

20. A method, implemented at a computer system that includes one or more processors, for facilitating removal of specular reflection noise from light data, the method comprising:
illuminating, using an illumination unit, a target with a light source, wherein the illumination unit is configured to project light with a spatial light pattern onto the target, wherein the spatial light pattern comprises a pattern that is oriented according to a known direction;
acquiring, with a sensor unit, light data that is reflected from the target, wherein:
the light data comprises a directly reflected spatial light pattern oriented in a first direction that corresponds to the known direction, and a specular reflected spatial light pattern oriented in a second direction that is different from the first direction, and
the difference between the first direction and the second direction comprises at l

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Godbaz John Peter
• Bamji Cyrus S
• Schmidt Mirko

Classifications

• H04N25/60

  Noise processing, e.g. detecting, correcting, reducing or removing noise · CPC title

• G01S7/4814 Primary

  of transmitters alone · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• H04N13/122

  Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues (H04N13/128 takes precedence) · CPC title

• G01S17/08

  for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48) · CPC title