Identifying defects in optical detector systems based on extent of stray light

Patent metadata

Field	Value
Publication number	US-11172192-B2
Application number	US-201916377527-A
Country	US
Kind code	B2
Filing date	Apr 8, 2019
Priority date	Dec 27, 2018
Publication date	Nov 9, 2021
Grant date	Nov 9, 2021

Abstract

Example embodiments relate to identifying defects in optical detector systems based on extent of stray light. An example embodiment includes a method. The method includes capturing, using an optical detector system, an image of a scene that includes a bright object. The method also includes determining a location of the bright object within the image. Further, the method includes determining, based on the location of the bright object within the image, an extent of stray light from the bright object that is represented in the image. In addition, the method includes determining, by comparing the extent of stray light from the bright object that is represented in the image to a predetermined threshold extent of stray light, whether one or more defects are present within the optical detector system. The predetermined threshold extent of stray light corresponds to an expected extent of stray light.

First claim

What is claimed is:

1. A method comprising:
capturing, using a camera, a first image of a scene that includes a bright object;
determining a location of the bright object within the first image;
determining, based on the location of the bright object within the first image, a first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image;
determining, by comparing the first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image to a predetermined threshold extent of stray light, whether one or more optical imperfections are present on an imaging optic of the camera,
wherein the predetermined threshold extent of stray light corresponds to an expected extent of stray light;
upon determining that one or more optical imperfections are present on the imaging optic of the camera, performing a corrective action;
after performing the corrective action, capturing, using the camera, a second image of the scene;
determining a second extent of diffuse veiling glare or lens flare represented in the second image; and
comparing the second extent of diffuse veiling glare to the first extent of diffuse veiling glare to determine whether the one or more optical imperfections are still present on the imaging optic of the camera.

2. The method of claim 1 , wherein the location of the bright object within the first image is determined based on a geographical location of the camera.

3. The method of claim 1 , wherein the one or more optical imperfections present on the imaging optic of the camera comprise condensation present on the imaging optic of the camera, and wherein the corrective action comprises wiping the imaging optic of the camera using a windshield wiper.

4. The method of claim 1 , wherein the corrective action comprises performing image analysis to alter images captured by the camera to mitigate errors caused by the one or more optical imperfections.

5. The method of claim 1 , wherein the one or more optical imperfections comprise an impurity in the imaging optic of the camera, a crack in the imaging optic of the camera, an air bubble in the imaging optic of the camera, a scratch on the imaging optic of the camera, dirt on the imaging optic of the camera, dust on the imaging optic of the camera, a bug splatter on the imaging optic of the camera, a degradation of a lens coating of the imaging optic of the camera, condensation on the imaging optic of the camera, debris on the imaging optic of the camera, imperfect transparency of the imaging optic of the camera, or warping of the imaging optic of the camera.

6. The method of claim 1 , wherein the first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image is also determined based on a predetermined angular size of the sun when observed from Earth.

7. The method of claim 1 , wherein taking corrective action comprises cleaning, repairing, or recalibrating.

8. The method of claim 1 , wherein determining whether one or more optical imperfections are present on the imaging optic of the camera comprises determining whether the imaging optic of the camera is unclean or occluded by debris.

9. The method of claim 8 , wherein determining whether the imaging optic within the camera is unclean or occluded by debris comprises determining whether:
the camera is facing backwards on a vehicle traveling forwards;
the camera is facing forwards on a vehicle traveling forwards;
the camera is facing backwards on a vehicle traveling backwards; or
the camera is facing forwards on a vehicle traveling backwards.

10. The method of claim 1 , wherein determining the first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image comprises:
identifying each pixel within the first image that has an intensity value above a predetermined threshold intensity value; and
generating a derivative image by converting all those pixels having an intensity value greater than the predetermined threshold intensity value to white and all those pixels having an intensity value less than or equal to the predetermined threshold intensity value to black.

11. The method of claim 10 , wherein determining the first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image further comprises determining a maximum radius of contiguous white pixels within the derivative image relative to a center of the location of the bright object.

12. The method of claim 10 , wherein determining the first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image further comprises identifying a secondary region of stray light that is separated by one or more black pixels from a primary region of stray light within the derivative image.

13. The method of claim 10 , wherein the predetermined threshold intensity value is based on a rejection ratio relative to an inherent luminance level of the bright object.

14. The method of claim 1 ,
wherein determining the first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image comprises identifying one or more streaks or stripes of stray light within the first image, and
wherein determining whether one or more optical imperfections are present on the imaging optic of the camera comprises identifying, based on the one or more streaks or stripes of stray light within the first image, that streaks of water or debris are present on the imaging optic of the camera as a result of motion of a windshield wiper across the imaging optic.

15. The method of claim 1 ,
wherein determining the first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image comprises identifying an existence of diffuse veiling glare within the first image, and
wherein determining whether one or more optical imperfections are present on the imaging optic of the camera comprises identifying, based on the diffuse veiling glare within the first image, that dust or water is present on the imaging optic of the camera.

16. The method of claim 1 ,
wherein determining the location of the bright object within the first image comprises capturing a baseline image of the scene at a reduced intensity,
wherein the baseline image is captured at a substantially similar perspective relative to the scene as the first image, and
wherein the baseline image is captured using a secondary camera that has a different dynamic range than the camera or using the camera with a modified dynamic range.

17. A non-transitory, computer-readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to execute a method comprising:
receiving a first image of a scene that includes a bright object, wherein the first image was captured using a camera;
determining a location of the bright object within the first image;
determining, based on the location of the bright object within the first image, a first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image;
determining, by comparing the first extent of diffuse veiling glare or lens flare from the bright object that is represented in the first image to a predetermined threshold extent of stray light, whether one or more optical imperfections are present on an imaging optic of the camera,
wherein the predetermined threshold extent of stray

Assignees

• Waymo Llc

Inventors

• Lu Chen David
• Rinehart Matthew
• Grabe Volker

Classifications

• H04N23/811

  by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor · CPC title

• H04N25/61

  the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" · CPC title

• H04N17/002 Primary

  for television cameras · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• G01M11/0228 Primary

  by measuring refractive power · CPC title