Monitoring of on-board vehicle image capturing device functionality compliance

Patent metadata

Field	Value
Publication number	US-11993274-B2
Application number	US-202017022948-A
Country	US
Kind code	B2
Filing date	Sep 16, 2020
Priority date	Sep 16, 2020
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

The present disclosure relates to a method performed by a luminance assessment system of a vehicle for monitoring of on-board vehicle image capturing device functionality compliance with a predeterminable requirement level. The luminance assessment system obtains at respective one or more time instants (t 1 , t 2 , t 3 ), a first image and a second image. A first section of the first images and a second section of the second images respectively cover a primary region. A first luminance value of the first section and a second luminance value of the second section are measured. A luminance deviation is determined by comparing at least one of the first luminance values to at least one of the second luminance values. A determination is made that, when the luminance deviation exceeds a deviation threshold, the first image capturing device or the second image capturing device failed to function according to the requirement level.

First claim

The invention claimed is:

1. A method performed by a luminance assessment system of a vehicle for monitoring of on-board vehicle image capturing device functionality compliance with a predeterminable requirement level, the vehicle comprising two or more image capturing devices configured to capture surroundings of the vehicle, a first image capturing device having a first field of view and a second image capturing device having a second field of view with a primary region thereof at least partly overlapping the first field of view, the method comprising:
obtaining at respective one or more time instants (t 1 , t 2 , t 3 ), a respective first image with support from the first image capturing device and a respective second image with support from the second image capturing device, a first section of the respective first image and a second section of the respective second image respectively covering the primary region;
measuring for respective first image a respective first luminance value based on measurements of a plurality of pixels of the first section, and for respective second image a respective second luminance value based on measurements of a plurality of pixels of the second section;
determining a luminance deviation exceeds a deviation threshold, the luminance deviation being based on a comparison of at least one of the respective first luminance values to at least one of the respective second luminance values;
determining, with support from a positioning system, a position of the vehicle;
localizing, based on the position of the vehicle, the vehicle in a digital map;
determining, based on the position of the vehicle in the digital map, that there is no presence of an image luminance affecting element within an image capturing device range of the vehicle, the image luminance affecting element being a light-obstructing element; and
determining, based on the luminance deviation exceeding the deviation threshold and there being no presence of the image luminance affecting element within the image capturing device range of the vehicle, that one of the first image capturing device and the second image capturing device fails to function according to the requirement level.

2. The method according to claim 1 , wherein determining a luminance deviation comprises comparing an average of two or more of the respective first luminance values to an average of two or more of the respective second luminance values.

3. The method according to claim 2 , wherein a third image capturing device has a third field of view with a secondary region thereof at least partly overlapping the first field of view, the method further comprising:
obtaining at respective one or more time instants (t 1 , t 2 , t 3 ), a respective third image with support from the third image capturing device, a third section of the respective third image and a first secondary section of the respective first image respectively covering the secondary region;
measuring for respective third image a respective third luminance value based on measurements of a plurality of pixels of the third section, and for respective first image a respective first secondary luminance value of the first secondary section;
determining a secondary luminance deviation based on comparing at least one of the respective third luminance values to at least one of the respective first secondary luminance values; and
identifying at least one of:
when the secondary luminance deviation exceeds a secondary deviation threshold, that it is the first image capturing device that fails to function according to the requirement level; and
when the secondary luminance deviation does not exceed the secondary deviation threshold, that it is the second image capturing device that fails to function according to the requirement level.

4. The method according to claim 1 , wherein a third image capturing device has a third field of view with a secondary region thereof at least partly overlapping the first field of view, the method further comprising:
obtaining at respective one or more time instants (t 1 , t 2 , t 3 ), a respective third image with support from the third image capturing device, a third section of the respective third image and a first secondary section of the respective first image respectively covering the secondary region;
measuring for respective third image a respective third luminance value based on measurements of a plurality of pixels of the third section, and for respective first image a respective first secondary luminance value based on measurements of a plurality of pixels of the first secondary section;
determining a secondary luminance deviation based on comparing at least one of the respective third luminance values to at least one of the respective first secondary luminance values; and
identifying at least one of:
when the secondary luminance deviation exceeds a secondary deviation threshold, that it is the first image capturing device that fails to function according to the requirement level; and
when the secondary luminance deviation does not exceed the secondary deviation threshold, that it is the second image capturing device that fails to function according to the requirement level.

5. The method according to claim 4 , wherein the determining a secondary luminance deviation comprises comparing an average of two or more of the respective third luminance values to an average of two or more of the respective first secondary luminance values.

6. The method according to claim 3 , wherein the determining a secondary luminance deviation comprises comparing an average of two or more of the respective third luminance values to an average of two or more of the respective first secondary luminance values.

7. A luminance assessment system for a vehicle for monitoring of on-board vehicle image capturing device functionality compliance with a predeterminable requirement level, the vehicle comprising two or more image capturing devices configured to capture surroundings of the vehicle, each of the two or more image capturing devices including a camera, a first image capturing device having a first field of view and a second image capturing device having a second field of view with a primary region thereof at least partly overlapping the first field of view, the luminance assessment system comprising:
a processor configured to:
obtain at respective one or more time instants (t 1 , t 2 , t 3 ), a respective first image with support from the first image capturing device and a respective second image with support from the second image capturing device, a first section of the respective first image and a second section of the respective second image respectively covering the primary region;
measure for respective first image a respective first luminance value based on measurements of a plurality of pixels of the first section, and for respective second image a respective second luminance value based on measurements of a plurality of pixels of the second section;
determine a luminance deviation exceeds a deviation threshold, the luminance deviation being based on a comparison of at least one of the respective first luminance values to at least one of the respective second luminance values;
determine, with support from a positioning system, a position of the vehicle;
localize, based on the vehicle position, the vehicle in a digital map;
determine, based on the vehicle position in the digital map, that there is no presence of an image luminance affecting element within an image capturing device range of the vehicle, the image luminance affecting element being a light-obstructing element; and
determine, based on the luminance deviation exceeding the deviation threshold and the processor determining there being no presence of an image lum

Assignees

• Zenuity Ab

Inventors

• Krutsch Robert
• Yaldo Valor

Classifications

• B60W50/0205 Primary

  Diagnosing or detecting failures; Failure detection models · CPC title

• B60W50/04

  Monitoring the functioning of the control system · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T7/97

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title

• B60W2050/021

  Means for detecting failure or malfunction · CPC title