Devices and methods for providing foveated scanning laser image projection with depth mapping

Patent metadata

Field	Value
Publication number	US-10200683-B2
Application number	US-201615386576-A
Country	US
Kind code	B2
Filing date	Dec 21, 2016
Priority date	Dec 21, 2016
Publication date	Feb 5, 2019
Grant date	Feb 5, 2019

Abstract

Devices and methods are described herein for providing foveated image projection. In general, at least one source of laser light is used to generate a laser beam, and scanning mirror(s) that reflect the laser beam into a pattern of scan lines. The source of light is controlled to selectively generate projected image pixels during a first portion of the pattern of scan lines, and to selectively generate depth mapping pulses during a second portion of the pattern of scan lines. The projected image pixels generate a projected image, while the depth mapping pulses are reflected from the surface, received, and used to generate a 3-dimensional point clouds that describe the measured surface depth at each point. Thus, during each scan of the pattern both a projected image and a surface depth map can be generated, with the surface depth map used to modify some portion of the projected pixels.

First claim

What is claimed is:

1. A scanning laser projector, comprising:
at least one source of laser light, the at least one source of laser light configured to generate a laser beam;
at least one scanning mirror configured to reflect the laser beam;
a drive circuit configured to provide an excitation signal to excite motion of the at least one scanning mirror to reflect the laser beam in a pattern of scan lines;
a pixel drive generator, the pixel drive generator configured to control the at least one source of laser light to selectively generate projected image pixels with the laser beam and selectively generate depth mapping pulses with the laser beam, and wherein the pixel drive generator is configured to generate projected image pixels during a first portion of the pattern of scan lines and generate depth mapping pulses during a second portion of the pattern of scan lines; and
a depth mapping device configured to receive reflections of the depth mapping pulses and generate a 3-dimensional point cloud based at least in part on timing of the received reflections of the depth mapping pulses, and wherein the pixel drive generator is configured to modulate intensity of selected projected image pixels in response to the 3-dimensional point clouds.

2. The scanning laser projector of claim 1 , wherein the first portion of the pattern of scan lines comprises scan lines during an active display period and wherein the second portion of the pattern of scan lines comprises scan lines during a vertical retrace period.

3. The scanning laser projector of claim 1 , wherein the pixel drive generator is configured to modulate intensity of selected projected image pixels by reducing the intensity of the selected projected image pixels.

4. The scanning laser projector of claim 1 , wherein the depth mapping device is configured to generate a first 3-dimensional point cloud and then generate a second 3-dimensional point cloud, and wherein the depth mapping device is configured to compare the second 3-dimensional point cloud to the first 3-dimensional point cloud.

5. The scanning laser projector of claim 4 , wherein the depth mapping device is configured to compare the second 3-dimensional point cloud to the first 3-dimensional point cloud to determine an object location in a field of the projected image pixels.

6. The scanning laser projector of claim 5 , wherein the pixel drive generator is configured to modulate intensity of selected projected image pixels in response to the 3-dimensional point clouds by modulating pixels outside the object location in a first way and modulating pixels inside the object location in a second way different from the first way.

7. The scanning laser projector of claim 5 , wherein the pixel drive generator is configured to modulate intensity of selected projected image pixels in response to the 3-dimensional point clouds by not projecting pixels on the object location while continuing to project pixels outside the object location.

8. The scanning laser projector of claim 5 , wherein the pixel drive generator is configured to modulate intensity of selected projected image pixels in response to the 3-dimensional point clouds by not projecting pixels outside the object location while continuing to project pixels on the object.

9. The scanning laser projector of claim 1 , wherein the first portion of the pattern of scan lines comprises scan lines during an active display period and wherein the second portion of the pattern of scan lines comprises scan lines during a vertical retrace period, and wherein the pixel drive generator is configured to modulate intensity of the selected projected image pixels in an active display immediately following the vertical retrace period during which changes in the 3-dimensional point clouds are detected.

10. The scanning laser projector of claim 1 , wherein the depth mapping device is configured to generate 3-dimensional point clouds based at least in part on timing of the received reflections of the depth mapping pulses by determining a relatively coarse mapping based on received reflections during a vertical retrace period and determining a relatively detailed mapping based on received reflections during an active display period.

11. The scanning laser projector of claim 1 , wherein the depth mapping device is configured to generate 3-dimensional point clouds based at least in part on timing of the received reflections of the depth mapping pulses by determining a region of interest based on received reflections during a vertical retrace period and determining a detailed mapping of the region of interest during an active display period.

12. The scanning laser projector of claim 1 , wherein the first portion of the pattern of scan lines comprises scan lines moving in a first direction and wherein the second portion of the pattern of scan lines comprises scan lines moving in a second direction, where the first direction and the second direction are opposite directions.

13. The scanning laser projector of claim 1 , wherein the first portion of the pattern of scan lines and the second portion of the pattern of scan lines are horizontal scan lines interlaced together.

14. The scanning laser projector of claim 1 , wherein the first portion of the pattern of scan lines comprises scan lines during an active display period and wherein the second portion of the pattern of scan lines comprises scan lines during a vertical retrace period.

15. The scanning laser projector of claim 1 , wherein the first portion of the pattern of scan lines and the second portion of the pattern of scan lines comprises generating projected image pixels and depth mapping pulses in each scan line.

16. The scanning laser projector of claim 1 , wherein the mapping device generates the 3-dimensional point cloud by calculating a time of flight for a return of the depth mapping pulses in the laser beam.

17. The scanning laser projector of claim 1 , wherein the motion of the at least one scanning mirror has a horizontal scan rate, a horizontal scan amplitude, a vertical scan rate, and a vertical scan amplitude, and wherein the drive circuit is further configured to dynamically adjust at least one of the horizontal scan rate, the horizontal scan amplitude, the vertical scan rate, and the vertical scan amplitude to modify characteristics of the 3-dimensional point cloud.

18. The scanning laser projector of claim 17 , wherein the drive circuit is configured to dynamically decrease the vertical scan rate of the at least one scanning mirror to provide increased horizontal resolution in the 3-dimensional point cloud.

19. The scanning laser projector of claim 1 , wherein the at least one source of laser light comprises an infrared laser, and wherein a pixel drive generator is configured to generate the depth mapping pulses with the infrared laser.

20. The scanning laser projector of claim 19 , wherein the at least one source of laser light additionally comprises a red laser, a blue laser, and a green laser and wherein a pixel drive generator is configured to generate projected image pixels using the red laser, the blue laser, and the green laser.

21. A scanning laser projector, comprising:
at least one source of laser light, the at least one source of laser light configured to generate a laser beam;
at least one scanning mirror configured to reflect the laser beam;
a drive circuit configured to provide an excitation signal to excite motion of the at least one scanning mirror to reflect the laser beam in a pattern of scan

Assignees

• Microvision Inc

Inventors

• Viswanathan P Selvan
• Honkanen Jari
• Wade Douglas R
• Xue Bin

Classifications

• H04N13/324

  Colour aspects · CPC title

• H04N9/3161

  using laser light sources (using laser beams scanning the display screen H04N9/3129) · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title

• G01B11/25

  by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50}) · CPC title

• H04N13/388 Primary

  Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume · CPC title