Methods and apparatus for virtual sensor array

Patent metadata

Field	Value
Publication number	US-9897699-B2
Application number	US-201514795113-A
Country	US
Kind code	B2
Filing date	Jul 9, 2015
Priority date	Jul 9, 2014
Publication date	Feb 20, 2018
Grant date	Feb 20, 2018

Abstract

A time-of-flight camera images an object around a corner or through a diffuser. In the case of imaging around a corner, light from a hidden target object reflects off a diffuse surface and travels to the camera. Points on the diffuse surface function as a virtual sensors. In the case of imaging through a diffuser, light from the target object is transmitted through a diffusive media and travels to the camera. Points on a surface of the diffuse media that is visible to the camera function as virtual sensors. In both cases, a computer represents phase and intensity measurements taken by the camera as a system of linear equations and solves a linear inverse problem to (i) recover an image of the target object; or (ii) to compute a 3D position for each point in a set of points on an exterior surface of the target object.

First claim

What is claimed is:

1. An apparatus comprising, in combination:
(a) an active light source;
(b) a time-of-flight camera which is configured to take measurements of phase and amplitude of incident light at each pixel, respectively, in a set of pixels of the camera, which incident light comprises amplitude-modulated continuous-wave light that (i) is emitted by the active light source and reflects from an object, and then (ii) travels to a diffusely reflective surface, and then (iii) reflects from the diffusely reflective surface and travels to the time-of-flight camera; and
(c) a computer that is programmed to perform a calculation that computes, for each respective point in a set of points on a surface of the object, a 3D position of the respective point, wherein the calculation involves solving a system of linear equations that represents the measurements.

2. The apparatus of claim 1 , wherein the calculation includes solving {right arrow over (y)}=D{right arrow over (x)} s.t. ∥{right arrow over (x)}∥ 0 =K, where: (a) each entry of vector {right arrow over (y)} represents measured amplitude and phase at a position on the diffusely reflective surface; (b) each entry of vector {right arrow over (x)} is a weight of confidence that a target on the object is at a voxel in a discretized voxel grid consisting of N voxel grid points; (c) there are K targets in the voxel grid; (d) D is a matrix defined as D=[S(u 1 ,w 1 ) S(u 2 ,w 1 ) . . . S(u R ,w Q )], (e) S(u,w) is an operator that takes as input a potential location of a target point on the object and that outputs measurements expected at positions on the diffusively reflective surface; (f) u and w are spatial coordinates of locations on the diffusely reflective surface; and (g) there are R discrete locations on a u-axis and Q discrete locations on a w-axis, such that N=RQ.

3. The apparatus of claim 1 , wherein the calculation includes solving {right arrow over ({circumflex over (x)})}=D H {right arrow over (y)}, where: (a) each entry of vector {right arrow over (y)} represents measured amplitude and phase at a position on the diffusely reflective surface; (b) each entry of vector {right arrow over ({circumflex over (x)})} is a weight of confidence that a target on the object is at a voxel in a discretized voxel grid consisting of N voxel grid points; (c) there are K targets in the voxel grid; (d) D is a matrix defined as D=[S(u 1 ,w 1 ) S(u 2 ,w 1 ) . . . S(u R , w Q )], (e) S(u,w) is an operator that takes as input a potential location of a target point on the object and that outputs measurements expected at positions on the diffusively reflective surface; (f) u and w are spatial coordinates of locations on the diffusely reflective surface; (g) there are R discrete locations on a u-axis and Q discrete locations on a w-axis, such that N=RQ; and (h) matrix D H is the conjugate transpose of matrix D.

4. The apparatus of claim 1 , wherein the calculation includes executing a pseudoinverse algorithm.

5. The apparatus of claim 1 , wherein the calculation includes executing a basis pursuit denoising algorithm.

6. The apparatus of claim 1 , wherein the computer is programmed to estimate a set of points on the diffusely reflective surface that are in the camera's field-of-view and that are each, respectively, closest, out of all points on the diffusely reflective surface, to a corresponding point on the surface of the object.

7. The apparatus of claim 1 , wherein the calculation includes computing an image of the object in real time.

8. The apparatus of claim 1 , wherein the diffusely reflective surface is not a Lambertian reflector.

9. The apparatus of claim 1 , wherein:
(a) an occluder prevents light from traveling in a straight line from the object to the time-of-flight camera; and
(b) the light source is on the same side of the occluder as the object.

10. The apparatus of claim 1 , wherein:
(a) an occluder prevents light from traveling in a straight line from the object to the time-of-flight camera; and
(b) the light source is on the same side of the occluder as the time-of-flight camera.

11. The apparatus of claim 1 , wherein the system of linear equations is represented by a matrix.

12. An apparatus comprising, in combination:
(a) an active light source;
(b) a time-of-flight camera which is configured to take measurements of phase and amplitude of incident light at each pixel, respectively, in a set of pixels of the camera, which incident light comprises amplitude-modulated continuous-wave light that (i) is emitted by the active light source and reflects from an object, and then (ii) travels through a diffuse media, and (iii) then travels to the time-of-flight camera; and
(c) a computer for performing a calculation that computes, for each respective point in a set of points on a surface of the object, a 3D position of the respective point, wherein the calculation involves solving a system of linear equations that represents the measurements.

13. The apparatus of claim 12 , wherein the diffuse media comprises a solid diffuser.

14. The apparatus of claim 12 , wherein the diffuse media comprises an aerosol, colloid, suspension or emulsion.

15. A method comprising, in combination:
(a) a light source emitting amplitude-modulated continuous-wave light which light, after leaving the light source (i) is at least partially scattered during transmission through a diffuse media or during diffuse reflection, and (ii) then travels to a time-of-flight camera;
(b) the time-of-flight camera taking measurements at each pixel, respectively, in a set of pixels of the camera, of phase and amplitude of the light; and
(c) a computer performing a calculation that computes, for each respective point in a set of points on a surface of an object, a 3D position of the respective point, wherein the calculation involves solving a system of linear equations that represents the measurements.

16. The method of claim 15 , wherein the light reflects from a diffusely reflective surface during the diffuse reflection.

17. The method of claim 15 , wherein the light is transmitted through a diffuse media that comprises a solid diffuser, aerosol, colloid, suspension or emulsion.

18. The method of claim 16 , wherein the calculation includes solving {right arrow over (y)}=D{right arrow over (x)} s.t. ∥{right arrow over (x)}∥ 0 =K, where: (a) each entry of vector {right arrow over (y)} represents measured amplitude and phase at a position on the diffusely reflective surface; (b) each entry of vector {right arrow over (x)} is a weight of confidence that a target on the object is at a voxel in a discretized voxel grid consisting of N voxel grid points; (c) there are K targets in the voxel grid; (d) D is a matrix defined as D=[S(u 1 ,w 1 ) S(u 2 ,w 1 ) . . . S(u R ,w Q )], (e) S(u,w) is an operator that takes as input a potential location of a target point on the object and that outputs measurements expected at positions on the diffusively reflective surface; (f) u and w are spatial coordinates of locations on the diffusely reflective surface; and (g) there are R discrete locations on a u-axis and Q discrete locations on a w-axis, such that N=RQ.

19. The method of claim 16 , wherein the computer is programmed to estimate a set of points on the diffusely reflective surface that are in the camera's field-of-view and that are each, respectively, closest, out of all points on the diffusely reflective surface, to a corresponding point on the surface of the object.

20. The method of claim 16 , wherein the calculation inc

Assignees

• Kadambi Achuta
• Zhao Hang
• Shi Boxin
• Bhandari Ayush
• Raskar Ramesh
• Massachusetts Inst Technology

Inventors

• Kadambi Achuta
• Zhao Hang
• Shi Boxin
• Bhandari Ayush
• Raskar Ramesh

Classifications

• G01S17/003 Primary

  Bistatic lidar systems; Multistatic lidar systems · CPC title

• G01S17/89 Primary

  for mapping or imaging · CPC title

• G01S17/46

  Indirect determination of position data · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title