Microlensing for real-time sensing of stray light

Patent metadata

Field	Value
Publication number	US-11678059-B2
Application number	US-202217659922-A
Country	US
Kind code	B2
Filing date	Apr 20, 2022
Priority date	Dec 26, 2019
Publication date	Jun 13, 2023
Grant date	Jun 13, 2023

Abstract

Example embodiments relate to microlensing for real-time sensing of stray light. An example device includes an image sensor that includes a plurality of light-sensitive pixels. The device also includes a first lens positioned over a first subset of light-sensitive pixels selected from the plurality of light-sensitive pixels. Further, the device includes a controller. The controller is configured to determine a first angle of incidence of a first light signal detected by the first subset of light-sensitive pixels. The controller is also configured to, based on the first determined angle of incidence, determine an amount of stray light incident on the image sensor.

First claim

What is claimed is:

1. A device comprising:
an image sensor comprising a plurality of light-sensitive pixels;
a first lens positioned over a first subset of light-sensitive pixels selected from the plurality of light-sensitive pixels; and
a controller configured to:
determine a first angle of incidence of a first light signal detected by the first subset of light-sensitive pixels;
based on the first determined angle of incidence, determine an amount of stray light incident on the image sensor; and
alter payload images captured using the image sensor based on the determined amount of stray light.

2. The device of claim 1 , wherein determining the amount of stray light incident on the image sensor comprises comparing the first determined angle of incidence to a threshold angle of incidence.

3. The device of claim 2 , wherein the threshold angle of incidence is 10°, and wherein the first determined angle of incidence and the threshold angle of incidence are measured relative to a vector that extends perpendicularly from a surface of the image sensor.

4. The device of claim 1 , further comprising a second lens positioned over a second subset of light-sensitive pixels selected from the plurality of light-sensitive pixels, wherein the controller is further configured to:
determine a second angle of incidence of a second light signal detected by the second subset of light-sensitive pixels; and
based on the second determined angle of incidence, determine the amount of stray light incident on the image sensor.

5. The device of claim 4 , wherein the controller is further configured to generate a stray-light map across the image sensor based on the first determined angle of incidence, the second determined angle of incidence, a location of the first subset of light-sensitive pixels on the image sensor, and a location of the second subset of light-sensitive pixels on the image sensor.

6. The device of claim 5 , wherein determining the stray-light map comprises generating a gamma-encoded histogram or a log-encoded histogram and interpolating values for light-sensitive pixel locations that are not within the first subset of light-sensitive pixels or the second subset of light-sensitive pixels.

7. The device of claim 4 , further comprising a third lens positioned over a third subset of light-sensitive pixels selected from the plurality of light-sensitive pixels, wherein the controller is also configured to:
determine a third angle of incidence of a third light signal detected by the third subset of light-sensitive pixels; and
based on the third determined angle of incidence, determine the amount of stray light incident on the image sensor.

8. The device of claim 7 , wherein the first subset of light-sensitive pixels corresponds to a first color channel of the image sensor, wherein the second subset of light-sensitive pixels corresponds to a second color channel of the image sensor, and wherein the third subset of light-sensitive pixels corresponds to a third color channel of the image sensor.

9. The device of claim 8 , wherein the controller is further configured to:
generate a first stray-light map across the image sensor corresponding to the first color channel;
generate a second stray-light map across the image sensor corresponding to the second color channel; and
generate a third stray-light map across the image sensor corresponding to the third color channel.

10. The device of claim 1 , wherein the first subset of light-sensitive pixels is oriented along a periphery of the image sensor.

11. The device of claim 10 , wherein the first subset of light-sensitive pixels is located in a corner of the image sensor.

12. The device of claim 1 , wherein the first angle of incidence is determined by comparing relative intensities detected by each light-sensitive pixel in the first subset of light-sensitive pixels.

13. The device of claim 1 , wherein determining the amount of stray light incident on the image sensor comprises comparing the first determined angle of incidence to a threshold angle of incidence, and wherein the threshold angle of incidence is based on:
a geometry of the image sensor;
materials of the image sensor;
a geometry of the first lens;
materials of the first lens;
environmental factors; or
a user setting.

14. The device of claim 1 , wherein the determined amount of stray light is stored in a memory as a look-up table.

15. The device of claim 1 , wherein the first subset of light-sensitive pixels comprises a 2×2 array of light-sensitive pixels, a 3×3 array of light-sensitive pixels, or a 4×4 array of light-sensitive pixels.

16. The device of claim 1 , wherein the controller is further configured to determine a true-black optical level based on the determined amount of stray light.

17. The device of claim 1 , wherein the image sensor is associated with a camera, and wherein the controller is further configured to adjust a pose of the camera, adjust a lens of the camera, or apply one or more filters to the camera based on the determined amount of stray light.

18. The device of claim 1 , wherein the device is used for object detection and avoidance within an autonomous vehicle.

19. A method comprising:
receiving, at a lens positioned over a first subset of light-sensitive pixels selected from a plurality of light-sensitive pixels that are part of an image sensor, a first light signal;
directing, using the lens, the first light signal toward the first subset of light-sensitive pixels;
detecting, by one or more light-sensitive pixels of the first subset, the first light signal;
determining a first angle of incidence of the detected first light signal;
determining, based on the first determined angle of incidence, an amount of stray light incident on the image sensor; and
altering payload images captured using the image sensor based on the determined amount of stray light.

20. A device comprising:
an image sensor comprising a plurality of light-sensitive pixels;
a plurality of subsets of light-sensitive pixels selected from the plurality of light-sensitive pixels positioned along an entire periphery of the image sensor;
a plurality of lenses, wherein each lens is positioned over a corresponding subset of light-sensitive pixels; and
a controller configured to:
determine an angle of incidence of light detected by each subset of light-sensitive pixels; and
based on the determined angles of incidence, determine a stray-light map across the image sensor; and
alter payload images captured using the image sensor based on the determined stray-light map.

Assignees

• Waymo Llc

Inventors

• Frevert Benjamin
• Ion Lucian
• Dharia Nirav

Classifications

• H04N23/71 Primary

  Circuitry for evaluating the brightness variation · CPC title

• H04N23/76

  by influencing the image signals · CPC title

• H04N23/811 Primary

  by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor · CPC title

• H04N25/671

  for non-uniformity detection or correction · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title