Depth mapping with a head mounted display using stereo cameras and structured light

Patent metadata

Field	Value
Publication number	US-9870068-B2
Application number	US-201615077540-A
Country	US
Kind code	B2
Filing date	Mar 22, 2016
Priority date	Sep 19, 2010
Publication date	Jan 16, 2018
Grant date	Jan 16, 2018

Abstract

A tracking system generates a structured light pattern in a local area. The system includes an array of lasers that generate light. The array of lasers includes a plurality of lasers and an optical element. The plurality of lasers are grouped into at least two subsets of lasers, and each of the at least two subsets of lasers is independently switchable. The optical element includes a plurality of cells that are each aligned with a respective subset of the array of lasers. Each cell receives light from a corresponding laser of the array of lasers, and each cell individually applies a modulation to the received light passing through the cell to form a corresponding portion of the structured light pattern that is projected onto a local area.

First claim

What is claimed is:

1. A system for generating a structured light pattern, comprising:
an array of lasers that generate light, the array of lasers including a plurality of lasers that are grouped into at least two subsets of lasers, and wherein each of the at least two subsets of lasers is independently switchable;
an optical element that comprises a plurality of cells that are each aligned with a respective subset of the array of lasers such that each cell receives light from a corresponding laser of the array of lasers, and each cell individually and dynamically applies a modulation to the received light passing through the cell to form a corresponding portion of the structured light pattern that is projected onto a local area, wherein the structured light pattern includes at least a first portion and a second portion that has a different modulation than that of the first portion, the first portion generated in part by a first cell of the plurality of cells, and the second portion is generated in part by a second cell of the plurality of cells, and the first portion of the structured light pattern is projected onto the region of interest; and
a controller configured to analyze one or more images of the local area to determine a region of interest within the local area.

2. The system of claim 1 , wherein the modulation is one member of a group comprising: a diffractive modulation, a refractive modulation, and a combination of a diffractive modulation and a refractive modulation.

3. The system of claim 1 , wherein a subset of the array of lasers, of the at least two subsets, and its respective cells are constructed from a single molded element.

4. The system of claim 1 , wherein a width of a cell, of the plurality of cells, is 1 mm or less.

5. The system of claim 1 , wherein one or more of the plurality of cells are controllable to dynamically provide changes to the structured light pattern based on receiving and analyzing at least one captured frame, the frame comprising a plurality of pixels in a two-dimensional layout.

6. The system of claim 5 , wherein the dynamic control is configurable to apply the structured light pattern with increased resolution to a first part portion of the structured light pattern and to apply the structured light pattern with reduced resolution to a second portion of the local area.

7. The system of claim 5 , wherein changes to the structured light pattern comprise a change in at least one cell of a plurality of cells associated with an optical element.

8. The system of claim 5 , wherein changes to the structured light pattern comprise a change in at least one cell of a plurality of cells associated with an optical element.

9. The system of claim 8 , wherein changes to the structured light pattern comprise a change in at least one cell of a plurality of cells.

10. The system of claim 1 , wherein the plurality of cells is further controllable with respect to a position and a shape of the generated structured light pattern.

11. A system for generating a structured light pattern comprising:
an array of lasers that generate light, the array of lasers including a plurality of lasers that are grouped into at least two subsets of lasers, and wherein each of the at least two subsets of lasers is independently switchable;
an optical element that comprises a plurality of cells that are each aligned with a respective subset of the array of lasers such that each cell receives light from a corresponding laser of the array of lasers, and each cell individually and dynamically applies a modulation to the received light passing through the cell to form a corresponding portion of the structured light pattern that is projected onto a local area, wherein the structured light pattern includes at least a first portion and a second portion that has a different modulation than that of the first portion, the first portion generated in part by a first cell of the plurality of cells, and the second portion is generated in part by a second cell of the plurality of cells, and the first portion of the structured light pattern is projected onto the region of interest;
an imager configured to capture one or more images of the local area, and the one or more images include at least a portion of the structured light pattern; and
a controller configured to analyze the one or more images of the local area to determine a region of interest within the local area and determine depth information for one or more objects in the three-dimensional space using the one or more images.

12. The system of claim 11 , wherein the modulation is one member of a group comprising: a diffractive modulation, a refractive modulation, and a combination of a diffractive modulation and a refractive modulation.

13. The system of claim 11 , wherein a subset of the array of lasers, of the at least two subsets, and its respective cells are constructed from a single molded element.

14. The system of claim 11 , wherein a width of a cell, of the plurality of cells, is 1 mm or less.

15. The system of claim 11 , wherein one or more of the plurality of cells are controllable to dynamically provide changes to the structured light pattern based on receiving and analyzing at least one captured frame, said frame comprising a plurality of pixels in a two-dimensional layout.

16. The system of claim 15 , wherein the dynamic control is configurable to apply the structured light pattern with increased resolution to a first part portion of the structured light pattern and to apply the structured light pattern with reduced resolution to a second portion of the local area.

17. The system of claim 11 , wherein the plurality of cells are further controllable with respect to a position and a shape of the generated structured light pattern.

18. A system for generating a structured light pattern comprising:
an array of lasers that generate light, the array of lasers including a plurality of lasers that are grouped into at least two subsets of lasers, and wherein each of the at least two subsets of lasers is independently switchable;
an optical element that comprises a plurality of cells that are each aligned with a respective subset of the array of lasers such that each cell receives light from a corresponding laser of the array of lasers, and each cell individually and dynamically applies a modulation to the received light passing through the cell to form a corresponding portion of the structured light pattern that is projected onto a local area, wherein the structured light pattern includes at least a first portion and a second portion that has a different modulation than that of the first portion, the first portion generated in part by a first cell of the plurality of cells, and the second portion is generated in part by a second cell of the plurality of cells, and the first portion of the structured light pattern is projected onto the region of interest;
an imager configured to capture one or more images of the local area, and the one or more images include at least a portion of the structured light pattern; and
a controller configured to analyze the one or more images of the local area to determine a region of interest within the local area and change the modulation of one or more lasers based on the one or more captured frames.

Assignees

• Facebook Inc

Inventors

• Romano Nitay
• Grossinger Nadav
• Alpern Yair
• Alon Emil
• Raz Guy

Classifications

• H01S5/0085

  for modulating the output, i.e. the laser beam is modulated outside the laser cavity · CPC title

• H01S5/423

  having a vertical cavity · CPC title

• H04N13/0253

  Electricity · mapped topic

• H04N13/044

  Electricity · mapped topic

• G06F3/0317 Primary

  in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface · CPC title