Vision system and method for a motor vehicle

Patent metadata

Field	Value
Publication number	US-12277775-B2
Application number	US-201716322349-A
Country	US
Kind code	B2
Filing date	Jul 27, 2017
Priority date	Aug 2, 2016
Publication date	Apr 15, 2025
Grant date	Apr 15, 2025

Abstract

A vision system (10) for a motor vehicle includes an imaging apparatus (11) adapted to capture images from a surrounding of the motor vehicle, and a processing device (14) adapted to execute an object detection code (15) to detect objects in the surrounding of the motor vehicle by processing images captured by the imaging apparatus (11), and to calculate for each detected object its longitudinal distance relative to the ego vehicle. The vision system includes a gatekeeper code module (33) having a higher functional safety level than the object detection code (15). The gatekeeper code module (33) takes as input objects (32) detected by the object detection code (15) and evaluates if a depth or disparity image (35), in at least a part of an area covering the detected object (32), matches within given tolerances the longitudinal distance of the detected object (32) given by the object detection code (15).

First claim

The invention claimed is:

1. A vision system for a motor vehicle, comprising:
at least one processor configured to:
execute an object detection code having a first functional safety level and configured to: detect a plurality of objects in a surrounding of the motor vehicle by processing images; and calculate for a detected object its longitudinal distance relative to the vehicle;
execute a verification code having a higher functional safety level than the object detection code, wherein the verification code is configured to obtain a verified depth or disparity image by re-calculating original depth or disparity data from a depth or disparity image calculator for a subset of a depth or disparity image, compare the re-calculated depth or disparity data with an original of the depth or disparity data, and verify the depth or disparity image based on the comparison; and
execute a gatekeeper code module having a second functional safety level, the gatekeeper code module configured to:
divide an area associated with the detected object into a grid comprising a plurality of columns and a plurality of rows, wherein intersection of each column of the plurality of columns and each row of the plurality of rows comprises a respective box; and
compare a longitudinal distance calculated by the object detection code for a portion of the detected object within a box of the grid to one or more depth values in an area of the verified depth or disparity image corresponding to the portion of the detected object within the box to evaluate if the depth or disparity image, in at least a part of the area of the verified depth or disparity image, matches within given tolerances the longitudinal distance of the detected object given by the object detection code,

wherein the first functional safety level and the second functional safety level each correspond to ratings for at least one of reliability and risk reduction, and
wherein the second functional safety level has a higher rating than the first functional safety level for the at least one of reliability and risk reduction.

2. The vision system as claimed in claim 1 , further comprising the verification code is executed on a dedicated electronic processing device.

3. The vision system as claimed in claim 1 , wherein the gatekeeper code module is executed on a dedicated electronic processor independent from a processor of the at least one processor executing the object detection code.

4. The vision system as claimed in claim 1 , wherein the rating of the second functional safety level for the at least one of reliability and risk reduction includes the Automotive Safety Integrity Level (ASIL) risk classification scheme, and the second functional safety level of the gatekeeper code module is at least an ASIL-B functional safety level.

5. The vision system as claimed in claim 1 , wherein the first functional safety level is a non-critical functional safety level.

6. The vision system as claimed in claim 5 , wherein the non-critical safety level is a Quality Management (QM) functional safety level.

7. The vision system as claimed in claim 1 , wherein:
the object detection code is configured to output bounding boxes for the plurality of objects; and
to divide the area associated with the object into the grid, the gatekeeper code module is configured to divide a bounding box of the plurality of bounding boxes into the grid.

8. The vision system of claim 1 , wherein the subset of the depth or disparity image comprises a subset of rows of the depth or disparity image.

9. A vision method for a motor vehicle, comprising:
executing, by at least one processor, an object detection code having a first functional safety level to detect a plurality of objects in a surrounding of the motor vehicle by processing captured images, and to calculate for a detected object its longitudinal distance relative to the vehicle;
executing, by the at least one processor, a verification code having a higher functional safety level than the object detection code, the verification code obtaining a verified depth or disparity image by re-calculating original depth or disparity data from a depth or disparity image calculator for a subset of a depth or disparity image, comparing the re-calculated depth or disparity data with an original of the depth or disparity data, and verifying the depth or disparity image based on the comparison; and
executing, by the at least one processor, a gatekeeper code module having a second functional safety level, the gatekeeper code module;
dividing an area associated with the detected object into a grid comprising a plurality of columns and a plurality of rows, wherein intersection of each column of the plurality of columns and each row of the plurality of rows comprises a respective box; and
comparing a longitudinal distance calculated for a portion of the detected object within a box of the grid to one or more depth values in an area the verified depth or disparity image corresponding to a portion of the detected object within the box to evaluate if a depth or disparity image, in at least a part of the area of the verified depth or disparity image, matches within given tolerances the longitudinal distance of the detected object given by the object detection code,

wherein the first functional safety level and the second functional safety level each correspond to ratings for at least one of reliability and risk reduction, and
wherein the second functional safety level has a higher rating than the first functional safety level for the at least one of reliability and risk reduction.

10. The vision method as claimed in claim 9 , wherein the gatekeeper code module is executed on a dedicated electronic processor independent from a processor of the at least one processor executing the object detection code.

11. The vision method as claimed in claim 9 further comprising calculating the verified depth or disparity image by a depth or disparity image calculation code having a lower functional safety level than the gatekeeper code module, and calculating, by the verification code, a safety flag indicating a safe condition or a not safe condition.

12. An apparatus comprising:
a memory; and
at least one processor communicatively coupled to the memory and configured to:
detect a plurality of objects in a surrounding of the apparatus by processing images;
calculate for a detected object its longitudinal distance relative to the apparatus, wherein the detection is associated with a first functional safety level;
re-calculate original depth or disparity data for a subset of a depth or disparity image;
compare the re-calculated depth or disparity data with an original of the depth or disparity data;
verify the depth or disparity image based on the comparison to obtain a verified depth or disparity image, wherein the verification is associated with a higher functional safety level than the first functional safety level;
divide an area associated with the detected object into a grid comprising a plurality of columns and a plurality of rows, wherein intersection of each column of the plurality of columns and each row of the plurality of rows comprises a respective box; and
compare a longitudinal distance calculated for a portion of the detected object within box of the grid to one or more depth values in an area of the verified depth or disparity image corresponding to the portion of the detected object within the box to evaluate if the depth or disparity image, in at least a part of the area of the verified depth or disparity image, matches within given tolerances the longitudinal distance of the detected object based on the verified depth or dispa

Assignees

• Arriver Software Ab
• Qualcomm Inc

Inventors

• Forslund David
• Bjärkefur Jon

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V2201/07

  Target detection · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title