Time-of-flight depth mapping with flexible scan pattern

Patent metadata

Field	Value
Publication number	US-9525863-B2
Application number	US-201514698912-A
Country	US
Kind code	B2
Filing date	Apr 29, 2015
Priority date	Apr 29, 2015
Publication date	Dec 20, 2016
Grant date	Dec 20, 2016

Abstract

Imaging apparatus includes an image sensor, which acquires an image of a scene, and a scanner, which includes an optical transmitter, which emits a sequence of optical pulses toward the scene, and an optical receiver, which receives the optical pulses reflected from the scene and generates an output indicative of respective times of flight of the pulses. Scanning optics are configured to scan the optical pulses over the scene in a scan pattern that covers and is contained within a non-rectangular area within the scene. A processor identifies an object in the image of the scene, defines the non-rectangular area so as to contain the identified object, and processes the output of the optical receiver so as to extract a three-dimensional (3D) map of the object.

First claim

The invention claimed is:

1. Imaging apparatus, comprising:
an image sensor, which is configured to acquire an image of a scene;
a scanner, comprising:
an optical transmitter, which is configured to emit a sequence of optical pulses toward the scene;
an optical receiver, which is configured to receive the optical pulses reflected from the scene and to generate an output indicative of respective times of flight of the pulses; and
scanning optics, which are configured to scan the optical pulses over the scene in a scan pattern that covers and is contained within a non-rectangular area within the scene,
wherein the scanning optics comprise first and second scanning mirrors, which are configured to rotate in accordance with the scan pattern, wherein the first scanning mirror directs the sequence of optical pulses toward the scene along a transmit path, while the second scanning mirror directs the pulses reflected from the scene along a return path, separated from the transmit path, toward the optical receiver; and

a processor, which is configured to identify an object in the image of the scene, to define the non-rectangular area so as to contain the identified object, and to process the output of the optical receiver as the scanning optics scan the optical pulses over the non-rectangular area so as to extract a three-dimensional (3D) map of the object.

2. The apparatus according to claim 1 , wherein the processor is configured to process the image so as to delineate an outer boundary of the identified object, and to define the non-rectangular area so as to coincide with the outer boundary.

3. The apparatus according to claim 1 , and comprising a user interface, configured to receive an input designating the object, wherein the processor is configured to identify the object responsively to the input.

4. The apparatus according to claim 3 , wherein the user interface comprises a touchscreen, and wherein the processor is configured to display the acquired image on the touchscreen.

5. The apparatus according to claim 1 , wherein the first and second scanning mirrors are contained, together with the optical transmitter and the optical receiver, in a single module.

6. The apparatus according to claim 1 , wherein the first scanning mirror and the optical transmitter are contained in a first module, while the second scanning mirror and the optical receiver are contained in a second module, separate from the first module.

7. The apparatus according to claim 1 , wherein the scanner comprises a first turning mirror, which reflects the optical pulses from the optical transmitter to the first scanning mirror, and a second turning mirror, which reflects the pulses reflected from the scene from the second scanning mirror to the optical receiver.

8. The apparatus according to claim 1 , wherein the scan pattern comprises a raster pattern comprising multiple scan lines of different, respective lengths.

9. The apparatus according to claim 1 , wherein the scan pattern comprises a non-raster pattern.

10. A method for imaging, comprising:
acquiring an image of a scene;
identifying an object in the image of the scene;
defining a non-rectangular area in the scene that contains the identified object;
scanning a sequence of optical pulses over the scene in a scan pattern that covers and is contained within the non-rectangular area,
wherein scanning the sequence of optical pulses comprises driving a first scanning mirror to rotate in accordance with the scan pattern so as to direct the sequence of optical pulses toward the scene along a transmit path;
receiving the optical pulses reflected from the scene and generating an output indicative of respective times of flight of the pulses,
wherein receiving the optical pulses comprises driving a second scanning mirror to rotate in accordance with the scan pattern so as to direct the pulses reflected from the scene along a return path, separated from the transmit path, toward an optical receiver; and
processing the output so as to extract a three-dimensional (3D) map of the object.

11. The method according to claim 10 , wherein defining the non-rectangular area comprises processing the image so as to delineate an outer boundary of the identified object, and defining the non-rectangular area so as to coincide with the outer boundary.

12. The method according to claim 10 , wherein identifying the object comprises receiving a user input designating the object.

13. The method according to claim 12 , wherein receiving the user input comprises display the acquired image on a touchscreen, and sensing a gesture designating the object on the touchscreen.

14. The method according to claim 10 , wherein the first and second scanning mirrors are contained, together with an optical transmitter and an optical receiver, in a single module.

15. The method according to claim 10 , wherein the first scanning mirror and an optical transmitter are contained in a first module, while the second scanning mirror and an optical receiver are contained in a second module, separate from the first module.

16. The method according to claim 10 , wherein scanning the sequence of the optical pulses comprises applying a first turning mirror to reflect the optical pulses from an optical transmitter to the first scanning mirror, and wherein receiving the optical pulses comprises applying a second turning mirror to reflect the pulses reflected from the scene from the second scanning mirror to the optical receiver.

17. The method according to claim 10 , wherein the scan pattern comprises a raster pattern comprising multiple scan lines of different, respective lengths.

18. The method according to claim 10 , wherein the scan pattern comprises a non-raster pattern.

Assignees

• Apple Inc

Inventors

• Nawasra Jawad
• Smith Scott T

Classifications

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/254 Primary

  in combination with electromagnetic radiation sources for illuminating objects · CPC title

• G01S17/42 Primary

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title

• H04N13/25

  using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor · CPC title