Proximity sensing using structured light

Patent metadata

Field	Value
Publication number	US-12361576-B2
Application number	US-202217885573-A
Country	US
Kind code	B2
Filing date	Aug 11, 2022
Priority date	Sep 6, 2021
Publication date	Jul 15, 2025
Grant date	Jul 15, 2025

Abstract

Sensing apparatus includes a projector, which projects toward a target a pattern of spots extending over a predefined angular range about a projection axis. An imaging assembly includes an image sensor and objective optics, which form on the image sensor an image of the target along an imaging axis, which is offset transversely relative to the projection axis. A processor is configured to process signals output by the image sensor so as generate, while the target is within a first span of distances from the apparatus, a depth map of the target responsively to shifts of the spots in the image, and to estimate, while the target is within a second span of distances, nearer to the apparatus than the first span, a distance to the target from the apparatus by detecting in the image a part of the pattern located at an edge of the angular range.

First claim

The invention claimed is:

1. Sensing apparatus, comprising:
a projector, which is configured to project toward a target a pattern of spots extending over a predefined angular range about a projection axis;
an imaging assembly, comprising an image sensor and objective optics, which are configured to form on the image sensor an image of the target along an imaging axis, which is offset transversely relative to the projection axis; and
a processor, which is configured to process signals output by the image sensor so as generate, while the target is within a first span of distances from the apparatus, a depth map of the target responsively to shifts of the spots in the image, and to estimate, while the target is within a second span of distances, nearer to the apparatus than the first span, a distance to the target from the apparatus by detecting in the image a part of the pattern located at an edge of the angular range,
wherein the processor is configured to find, responsively to the detected part of the pattern, a location of the edge of the angular range in the image, and to estimate the distance to the target within the second span responsively to the location of the edge by a process of triangulation based on the location of the edge in the image and an offset between the projection axis and the imaging axis.

2. The apparatus according to claim 1 , wherein the processor is further configured to detect, when the pattern is absent from the image, that the target is located within a third span of distances, nearer to the apparatus than the second span.

3. The apparatus according to claim 2 , wherein within the second span of the distances, the image contains bars formed by the spots projected onto the target, and wherein the processor is configured to detect whether the target is in the second span or the third span of the distances responsively to a presence or absence of the bars in the image.

4. The apparatus according to claim 1 , wherein the projector comprises a diffractive optical element (DOE), and wherein the spots correspond to respective diffraction orders of the DOE.

5. The apparatus according to claim 1 , wherein the processor is configured to find the location of the edge by identifying one or more local extrema in an intensity profile of the image.

6. Sensing apparatus, comprising:
a projector, which is configured to project toward a target a pattern of spots extending over a predefined angular range about a projection axis;
an imaging assembly, comprising an image sensor and objective optics, which are configured to form on the image sensor an image of the target along an imaging axis, which is offset transversely relative to the projection axis; and
a processor, which is configured to process signals output by the image sensor so as generate, while the target is within a first span of distances from the apparatus, a depth map of the target responsively to shifts of the spots in the image, and to estimate, while the target is within a second span of distances, nearer to the apparatus than the first span, a distance to the target from the apparatus by detecting in the image a part of the pattern located at an edge of the angular range,
wherein the processor is configured to store reference data indicating respective expected locations in the image of the spots at the edge of the angular range for each of a plurality of values of the distance within the second span, and to estimate the distance to the target within the second span by matching actual locations of the spots in the image of the pattern to the expected locations of the spots in the stored reference data.

7. The apparatus according to claim 6 , wherein the processor is configured to find respective displacements between the actual locations and the expected locations of each of the spots for two or more of the values of the distance of the reference data, and to select a value of the distance to the target responsively to the respective displacements.

8. The apparatus according to claim 7 , wherein the processor is configured to compute respective scores for the two or more of the values of the distance based on the respective displacements, and to choose the value of the distance, responsively to the respective scores, at which the expected locations of the spots optimally match the actual locations.

9. The apparatus according to claim 6 , wherein the imaging axis is offset relative to the projection axis along a baseline direction, and wherein the spots in the projected pattern are arranged along rows that are angled obliquely relative to the baseline direction.

10. A method for sensing, comprising:
projecting from a projector toward a target a pattern of spots extending over a predefined angular range about a projection axis;
capturing an image of the target along an imaging axis, which is offset transversely relative to the projection axis; and
processing the image so as generate, while the target is within a first span of distances of the projector, a depth map of the target responsively to shifts of the spots in the image, and to estimate, while the target is within a second span of distances, nearer to the projector than the first span, a distance to the target by detecting in the image a part of the pattern located at an edge of the angular range,
wherein processing the image comprises finding, responsively to the detected part of the pattern, a location of the edge of the angular range in the image, and estimating the distance to the target within the second span by applying a process of triangulation based on the location of the edge in the image and an offset between the projection axis and the imaging axis.

11. The method according to claim 10 , wherein processing the image comprises detecting, when the pattern is absent from the image, that the target is located within a third span of distances, nearer to the projector than the second span.

12. The method according to claim 11 , wherein within the second span of the distances, the image contains bars formed by the spots projected onto the target, and wherein processing the image comprises detecting whether the target is in the second span or the third span of the distances responsively to a presence or absence of the bars in the image.

13. The method according to claim 10 , wherein the projector comprises a diffractive optical element (DOE), and wherein the spots correspond to respective diffraction orders of the DOE.

14. The method according to claim 10 , wherein processing the image comprises storing reference data indicating respective expected locations in the image of the spots at the edge of the angular range for each of a plurality of values of the distance within the second span, and estimating the distance to the target within the second span by matching actual locations of the spots in the image of the pattern to the expected locations of the spots in the stored reference data.

15. The method according to claim 14 , wherein estimating the distance comprises finding respective displacements between the actual locations and the expected locations of each of the spots for two or more of the values of the distance of the reference data, and selecting a value of the distance to the target responsively to the respective displacements.

16. The method according to claim 14 , wherein the imaging axis is offset relative to the projection axis along a baseline direction, and wherein the spots in the projected pattern are arranged along rows that are angled obliquely relative to the baseline direction.

Assignees

• Apple Inc

Inventors

• Hauser Jonathan
• Avraham Assaf
• Leshem Dror
• Della Pergola Refael
• Keren-Zur Shay

Classifications

• G06T7/74

  involving reference images or patches · CPC title

• G06T7/521 Primary

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title