Dynamic matrix filter for vehicle image sensor

Patent metadata

Field	Value
Publication number	US-11256013-B2
Application number	US-201916460421-A
Country	US
Kind code	B2
Filing date	Jul 2, 2019
Priority date	Mar 27, 2019
Publication date	Feb 22, 2022
Grant date	Feb 22, 2022

Abstract

Aspects of the present disclosure include systems, methods, and devices use a controllable matrix filter to selectively obscure regions of an image sensor's field of view. The controllable matrix filter is a physical component that may be placed in front of an image sensor and, in certain situations, one or more regions of the otherwise transparent matrix filter may be selectively configured to have an increased optical density such that the one or more regions become opaque thereby blocking out certain regions of the image sensor's field of view. In this way, the controllable matrix filter may be used to mask out certain regions in an image sensor's field of view that may present processing difficulties for downstream systems that utilize information from the image sensor.

First claim

What is claimed is:

1. A method comprising:
detecting, by one or more processors of a filter controller, a region of interest in a field of view of an image sensor of an autonomous vehicle (AV) system;
selecting, by the one or more processors of the filter controller, a target region in the field of view of the image sensor based on the region of interest in the field of view;
determining a first optical density for a first sub-region within the target region;
determining a second optical density for a second sub-region within the target region;
adjusting, by a dynamic matrix filter coupled to the image sensor, a first portion of the dynamic matrix filter corresponding to the first sub-region to the first optical density and a second portion of the dynamic matrix filter corresponding to the second sub-region to the second optical density; and
after adjusting the dynamic matrix filter, generating image data with the image sensor, image data.

2. The method of claim 1 , wherein the detecting of the region of interest comprises:
accessing map data comprising a map of an area that is a current location of the AV system;
accessing vehicle pose data comprising a position and an orientation of the AV system; and
determining the region of interest in the field of view of the image sensor based on the map data and vehicle pose data.

3. The method of claim 2 , wherein the selecting of the target region in the field of view of the image sensor comprises:
selecting a remaining region in the field of view of the image sensor as the target region, the remaining region excluding the region of interest.

4. The method of claim 1 , wherein:
the image data is second image data;
the detecting of the region of interest in the field of view of the image sensor comprises analyzing first image data to identify a saturated region, the first image data being generated prior to the adjusting of the dynamic matrix filter; and
the selecting of the target region comprises mapping the saturated region in the first image data to the target region in the field of view of the image sensor.

5. The method of claim 4 ,
the method further comprising:
determining that a saturation of the saturated region in the second image data satisfies a saturation condition; and
adjusting, by the dynamic matrix filter, at least the first portion of the dynamic matrix filter to a modified optical density different than the first optical density.

6. The method of claim 1 , further comprising:
generating, by the filter controller, a control signal that causes the dynamic matrix filter to apply the first optical density to the first portion of the dynamic matrix filter; and
transmitting, by the filter controller, the control signal to the dynamic matrix filter.

7. The method of claim 1 , wherein the adjusting comprises changing an optical density of the first portion of the dynamic matrix filter from a default optical density to the first optical density.

8. The method of claim 1 , wherein the adjusting comprises increasing an optical density of the first portion of the dynamic matrix filter.

9. The method of claim 7 , further comprising:
detecting, by the one or more processors of the filter controller, a termination event; and
based on detecting the termination event, causing the dynamic filter to set the first portion of the dynamic matrix filter to the default optical density.

10. A system comprising:
an image sensor to generate first image data comprising a depiction of a real-world scene that is visible within a field of view of the image sensor;
a dynamic filter positioned in a field of view of the image sensor, the dynamic filter comprising an array of filter elements having an adjustable optical density; and
a filter controller comprising one or more processors, the filter controller to perform operations comprising:
detecting a region of interest in the field of view of the image sensor;
selecting a target region in the field of view of the image sensor based on the region of interest;
determining a first optical density for a first sub-region within the target region;
determining a second optical density for a second sub-region within the target region;
causing an adjustment to the dynamic filter, the adjustment resulting in a first portion of the array of filter elements corresponding to the first sub-region having the first optical density and a second portion of the array of filter elements corresponding to the second sub-region having the second optical density different than the first optical density; and
and
the image sensor to generate second image data after the adjustment to the dynamic filter.

11. The system of claim 10 , further comprising:
mapping the target region in the field of view of the image sensor to at least the first portion of the array of filter elements.

12. The system of claim 10 , wherein the detecting of the region of interest in the field of view of the image sensor comprises:
accessing map data comprising a map of an area that includes the real-world scene and information describing one or more features of the area;
accessing vehicle pose data comprising a position and an orientation of an autonomous vehicle system; and
determining the region of interest based on the map data and vehicle pose data.

13. The system of claim 11 , wherein the selecting of the target region in the field of view of the image sensor comprises:
selecting a remaining region in the field of view of the image sensor that excludes the region of interest.

14. The system of claim 13 , wherein:
the detecting of the region of interest in the field of view of the image sensor comprises analyzing the first image data to identify a saturated region; and
the selecting of the target region comprises mapping the saturated region in the first image data to the target region in the field of view of the image sensor.

15. A dynamic filter comprising:
an array of filter elements positioned in a field of view of an image sensor, each filter element in the array of filter elements having an adjustable optical density, the image sensor to generate image data depicting a real-world scene; and
a filter controller comprising one or more processors, the filter controller being programmed to perform operations comprising:
detecting a region of interest in the field of view of the image sensor;
selecting a target region in the field of view of the image sensor based on the region of interest;
determining a first optical density for a first sub-region within the target region;
determining a second optical density for a second sub-region within the target region; and
causing an adjustment to the dynamic filter, the adjustment resulting in a first portion of the array of filter elements corresponding to the first sub-region having the first optical density and a second portion of the array of filter elements corresponding to the second sub-region having the second optical density different than the first optical density.

16. The dynamic filter of claim 15 , wherein:
the image sensor comprises a focal plane array; and
the optical density of the first portion of the array of filter elements is increased based in part on a mapping of the first portion of the array of filter elements in the focal plane array that correspond to the target region of the field of view of the image sensor.

17. The dynamic filter of claim 16 , wherein the image sensor further comprises a lens system, wherein the array of filter elements is positioned between the lens system and the focal plane arr

Assignees

• Uatc Llc

Inventors

• Gruver Daniel

Classifications

• G02B26/023 Primary

  comprising movable attenuating elements, e.g. neutral density filters · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G02B5/204 Primary

  in which spectral selection is performed by means of a conductive grid or array, e.g. frequency selective surfaces (for use with wavelengths longer than the infrared light H01Q15/0006) · CPC title