Method and apparatus for de-noising data from a distance sensing camera

Patent metadata

Field	Value
Publication number	US-10003757-B2
Application number	US-201214655618-A
Country	US
Kind code	B2
Filing date	Dec 28, 2012
Priority date	Dec 28, 2012
Publication date	Jun 19, 2018
Grant date	Jun 19, 2018

Abstract

It is inter alia disclosed to determine a phase difference between a light signal transmitted by a time of flight camera system and a reflected light signal received by at least one pixel sensor of an array of pixel sensors in an image sensor of the time of flight camera system, wherein the reflected light signal received by the at least one pixel sensor is reflected from an object illuminated by the transmitted light signal ( 301 ); determine an amplitude of the reflected light signal received by the at least one pixel sensor ( 301 ); combine the amplitude and phase difference for the at least one pixel sensor into a combined signal parameter for the at least one pixel sensor ( 307 ); and de-noise the combined signal parameter for the at least one pixel sensor by filtering with a filter the combined parameter for the at least one pixel sensor ( 309 ).

First claim

The invention claimed is:

1. A method comprising:
determining a phase difference between a light signal transmitted by a time of flight camera system and a reflected light signal received by at least one pixel sensor of an array of pixel sensors in an image sensor of the time of flight camera system, wherein the reflected light signal received by the at least one pixel sensor is reflected from an object illuminated by the transmitted light signal;
determining an amplitude of the reflected light signal received by the at least one pixel sensor;
de-noising at least one of the phase difference or the amplitude for the at least one pixel sensor by filtering the phase difference or the amplitude for the at least one pixel sensor;
following de-noising of at least one of the phase difference or the amplitude, combining the amplitude and phase difference for the at least one pixel sensor into a combined signal parameter for the at least one pixel sensor; and
de-noising the combined signal parameter for the at least one pixel sensor by filtering the combined parameter for the at least one pixel sensor to generate a de-noised combined signal parameter from which a distance map image that is less effected by noise is determinable by the time of flight camera system.

2. The method as claimed in claim 1 , wherein the filtering further comprises:
filtering with a non-local spatial transform filter.

3. The method as claimed in claim 2 , wherein the non-local spatial transform filter is a non-local means filter.

4. The method as claimed in claim 1 , further comprising calculating a distance range to the object from the de-noised combined signal parameter for the at least one pixel sensor by:
determining the de-noised phase difference for the at least one pixel sensor from the de-noised combined signal parameter for the at least one pixel sensor; and
calculating the distance range to the object for the at least one pixel sensor using the de-noised phase difference for the at least one pixel sensor.

5. The method as claimed in claim 1 , wherein the combined signal parameter is a complex signal parameter formed from combining the amplitude and phase difference for the at least one pixel sensor.

6. The method as claimed in claim 1 , wherein the image sensor of the time of flight camera system is based at least in part on a photonic mixer device.

7. An apparatus comprising at least one processor and at least one memory including computer code for one or more programs, the at least one memory and the computer code configured with the at least one processor to cause the apparatus at least to:
determine a phase difference between a light signal transmitted by a time of flight camera system and a reflected light signal received by at least one pixel sensor of an array of pixel sensors in an image sensor of the time of flight camera system, wherein the reflected light signal received by the at least one pixel sensor is reflected from an object illuminated by the transmitted light signal;
determine an amplitude of the reflected light signal received by the at least one pixel sensor;
de-noise at least one of the phase difference or the amplitude for the at least one pixel sensor by filtering the phase difference or the amplitude for the at least one pixel sensor;
following de-noising of at least one of the phase difference or the amplitude, combine the amplitude and phase difference for the at least one pixel sensor into a combined signal parameter for the at least one pixel sensor; and
de-noise the combined signal parameter for the at least one pixel sensor by filtering the combined parameter for the at least one pixel sensor to generate a de-noised combined signal parameter from which a distance map image that is less effected by noise is determinable by the time of flight camera system.

8. The apparatus as claimed in claim 7 , wherein filtering comprises filtering with a non-local spatial transform filter.

9. The apparatus as claimed in claim 8 , wherein the non-local spatial transform filter is a non-local means filter.

10. The apparatus as claimed in claim 7 , wherein the at least one memory and the computer code configured with the at least one processor is further configured to calculate a distance range to the object from the de-noised combined signal parameter for the at least one pixel sensor by being configured to:
determine the de-noised phase difference for the at least one pixel sensor from the de-noised combined signal parameter for the at least one pixel sensor, and
calculate the distance range to the object for the at least one pixel sensor using the de-noised phase difference for the at least one pixel sensor.

11. The apparatus as claimed in claim 7 , wherein the combined signal parameter is a complex signal parameter formed from combining the amplitude and phase difference for the at least one pixel sensor.

12. The apparatus as claimed in claim 7 , wherein the image sensor of the time of flight camera system is based at least in part on a photonic mixer device.

13. A computer program product comprising at least one computer-readable storage medium, the computer-readable storage medium comprising a set of instructions, which, when executed by one or more processors, causes an apparatus to:
determine a phase difference between a light signal transmitted by a time of flight camera system and a reflected light signal received by at least one pixel sensor of an array of pixel sensors in an image sensor of the time of flight camera system, wherein the reflected light signal received by the at least one pixel sensor is reflected from an object illuminated by the transmitted light signal;
determine an amplitude of the reflected light signal received by the at least one pixel sensor;
de-noise at least one of the phase difference or the amplitude for the at least one pixel sensor by filtering the phase difference or the amplitude for the at least one pixel sensor;
following de-noising of at least one of the phase difference or the amplitude, combine the amplitude and phase difference for the at least one pixel sensor into a combined signal parameter for the at least one pixel sensor; and
de-noise the combined signal parameter for the at least one pixel sensor by filtering the combined parameter for the at least one pixel sensor to generate a de-noised combined signal parameter from which a distance map image that is less effected by noise is determinable by the time of flight camera system.

14. The computer program product as claimed in claim 13 , wherein the set of instructions when executed by the processor causes the apparatus to filter, further causes the apparatus to:
filter with a non-local spatial transform filter.

15. The computer program product as claimed in claim 14 , wherein the non-local spatial transform filter is a non-local means filter.

16. The computer program product as claimed in claim 13 , wherein the set of instructions when executed by the processor further causes the apparatus to calculate a distance range to the object from the de-noised combined signal parameter for the at least one pixel sensor by being caused to:
determine the de-noised phase difference for the at least one pixel sensor from the de-noised combined signal parameter for the at least one pixel sensor; and
calculate the distance range to the object for the at least one pixel sensor using the de-noised phase difference for the at least one pixel sensor.

17. The computer program product as claimed in claim 13 , wherein the combined signal parameter is a complex signal parameter formed

Assignees

• Nokia Technologies Oy

Inventors

• Georgiev Mihail
• Gotchev Atanas

Classifications

• G06T5/20 Primary

  using local operators · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title

• G01S17/36 Primary

  with phase comparison between the received signal and the contemporaneously transmitted signal · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title