Technologies for detection of occlusions on a camera

Patent metadata

Field	Value
Publication number	US-2021192745-A1
Application number	US-201916718344-A
Country	US
Kind code	A1
Filing date	Dec 18, 2019
Priority date	Dec 18, 2019
Publication date	Jun 24, 2021
Grant date	—

Abstract

Technologies for detecting occlusions on a camera of a vehicle by a compute device are disclosed. The compute device may receive one or more images from the camera. The compute device may analyze the images using various algorithms such as optical flow calculations, blurriness detection for portions of the image, edge detection, and circular artifact detection. The analysis may be used to determine the presence of occlusions on the camera, such as water drops, mud, dirt, etc. The compute device may send a command to clean the camera and/or may use the determined presence of occlusions as part analyzing images from the camera for a driver assist system, such as by ignoring portions of the image that are occluded.

First claim

1 . A compute device for detection of occlusions on a camera of a vehicle, the compute device comprising:
the camera; an occlusion detection module to:
receive one or more images from the camera of the vehicle;
process the one or more images, wherein to process the one or more images comprises at least one of (i) determine a change in optical flow based on three images of the one or more images, (ii) determine one or more regions of at least one image of the one or more images that are out of focus, (iii) perform edge detection on at least one image of the one or more images, and (iv) detect one or more circular lens artifacts in at least one image of the one or more images; and
determine, based on the one or more images, one or more occlusions on the camera of the vehicle.

2 . The compute device of claim 1 , wherein to process the one or more images comprises to process one image,
wherein to determine, based on the one or more images, the one or more occlusions on the camera of the vehicle comprises to determine, based only on the one image from the camera, the one or more occlusions on the camera of the vehicle.

3 . The compute device of claim 1 , wherein to process the one or more images comprises to process no more than three images,
wherein to determine, based on the one or more images, the one or more occlusions on the camera of the vehicle comprises to determine, based only on no more than three images from the camera, the one or more occlusions on the camera of the vehicle.

4 . The compute device of claim 1 , wherein to process the one or more images comprises to determine the change in optical flow based on the three images of the one or more images, wherein to determine the change in optical flow based on the three images comprises:
determine a first optical flow between a first image of the three images and a second image of the three images; determine a second optical flow between the second image and a third image of the three images; and determine a difference in optical flow magnitude between the first optical flow and the second optical flow for each of a plurality of pixels, wherein to determine, based on the one or more images, the one or more occlusions on the camera of the vehicle comprises to determine, based on the difference in optical flow magnitude between the first optical flow and the second optical flow for each of a plurality of pixels, the one or more occlusions on the camera of the vehicle.

5 . The compute device of claim 4 , wherein to process the one or more images further comprises to determine a stationary score for each of the plurality of pixels based on the first optical flow, wherein the stationary score for each of the plurality of pixels indicates a magnitude of optical flow in the first optical flow for the corresponding pixel,
wherein to determine, based on the one or more images, the one or more occlusions on the camera of the vehicle comprises to determine, based on the stationary score for each of the plurality of pixels, the one or more occlusions on the camera of the vehicle.

6 . The compute device of claim 1 , wherein to process the one or more images comprises to determine the one or more regions of the at least one image that are out of focus, wherein to determine the one or more regions the at least one image that are out of focus comprises:
divide the at least one image into a plurality of subimages; and determine, for each of the plurality of subimages, whether the corresponding subimage is blurry, wherein to determine, based on the one or more images, the one or more occlusions on the camera of the vehicle comprises to determine, based on a determination of whether each of the plurality of subimages is blurry, the one or more occlusions on the camera of the vehicle.

7 . The compute device of claim 1 , wherein to process the one or more images comprises to perform edge detection on the at least one image of the one or more images, wherein to process the one or more images further comprises:
expand each of a plurality of edges identified in the at least one image during edge detection; and determine areas of at least one image without expanded edges, wherein to determine, based on the one or more images, the one or more occlusions on the camera of the vehicle comprises to determine, based on a determination of areas of the at least one image without expanded edges, the one or more occlusions on the camera of the vehicle.

8 . The compute device of claim 1 , wherein to process the one or more images comprises to detect one or more circular lens artifacts in the at least one image of the one or more images, wherein to detect one or more circular lens artifacts in the at least one image comprises:
perform edge detection of the at least one image; identify each of a plurality of contours defined by edge detection; and determine, for each of the plurality of contours, a ratio of a circle enclosing the corresponding contour to an area enclosed by the corresponding contour, wherein to determine, based on the one or more images, the one or more occlusions on the camera of the vehicle comprises to determine, based on a determination of the ratios for each of the plurality of contours.

9 . The compute device of claim 1 , further comprising selecting, based on one or more current conditions of the vehicle, one or more parameters of an occlusion detection algorithm.

10 . The compute device of claim 1 , further comprising applying a mask to the one or more images to block the sky.

11 . A method for detection of occlusions on a camera of a vehicle, the method comprising:
receiving, by a compute device, one or more images from the camera of the vehicle; processing, by the compute device, the one or more images, wherein processing the one or more images comprises at least one of (i) determining a change in optical flow based on three images of the one or more images, (ii) determining one or more regions of at least one image of the one or more images that are out of focus, (iii) performing edge detection on at least one image of the one or more images, and (iv) detecting one or more circular lens artifacts in at least one image of the one or more images; and determining, by the compute device and based on the one or more images, one or more occlusions on the camera of the vehicle.

12 . The method of claim 11 , wherein processing the one or more images comprises determining the change in optical flow based on the three images of the one or more images, wherein determining the change in optical flow based on the three images comprises:
determining, by the compute device, a first optical flow between a first image of the three images and a second image of the three images; determining, by the compute device, a second optical flow between the second image and a third image of the three images; and determining, by the compute device, a difference in optical flow magnitude between the first optical flow and the second optical flow for each of a plurality of pixels, wherein determining, based on the one or more images, the one or more occlusions on the camera of the vehicle comprises determining, based on the difference in optical flow magnitude between the first optical flow and the second optical flow for each of a plurality of pixels, the one or more occlusions on the camera of the vehicle.

13 . The method of claim 11 , wherein processing the one or more images comprises determining the one or more regions of the at least one image that are out of focus, wherein determining the one or more regions the at least one image that are out of focus comprises:
dividing, by the compute d

Assignees

• Clarion Co Ltd

Inventors

• Kahlbaum Robert

Classifications

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V10/993

  Evaluation of the quality of the acquired pattern · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title