Method, system, imaging device, movable device and program product for detecting static elements in video and image sources

Patent metadata

Field	Value
Publication number	US-9454703-B2
Application number	US-201514606540-A
Country	US
Kind code	B2
Filing date	Jan 27, 2015
Priority date	Jan 28, 2014
Publication date	Sep 27, 2016
Grant date	Sep 27, 2016

Abstract

The invention relates to a method for determining a static element in images captured by an imaging means mounted on a movable device. The method comprises steps of acquiring a first image and a second image captured by the imaging means. The first image and the second image are captured at capture times separated by a time difference and the method is characterized by the time difference being selected depending on motion parameters of the movable device. A determination measure for corresponding regions of the first and the second image for representing a similarity of the corresponding regions is calculated and a static element of the first and the second image is determined based on the calculated determination measure. An output signal comprising information on the determined static element is generated.

First claim

The invented claimed is:

1. Method for determining a static element in images captured by an imaging means mounted on a movable device, the method comprising steps of:
acquiring a first image and a second image captured by the imaging means, wherein the first image and the second image are captured at capture times separated by a time difference,
calculating a determination measure for corresponding regions of the first image and the second image for representing a similarity of the corresponding regions,
determining a static element of the first image and the second image based on the calculated determination measure,
generating an output signal comprising information on the determined static element, and wherein
the capture times of the first image and the second image that are acquired depends on at least one motion parameter of the movable device.

2. The method according to claim 1 , wherein
the time difference Δt is dependent on at least one motion parameter of the movable device.

3. The method according to claim 1 , wherein
the determination measure is calculated on the corresponding regions each of the first image and the second image for each pixel.

4. The method according to claim 1 , wherein
the determination measure is calculated as a normalized determination measure.

5. The method according to claim 1 , wherein
the determination measure is one of a normalized cross correlation, a summed normalized cross correlation, summed or squared difference measure for rank transformed images and a summed hamming distance of census transformed images.

6. The method according to claim 4 , wherein
for determining the static element an element in the first image and the second image is determined for which the calculated determination measure exceeds a threshold.

7. The method according to claim 1 , wherein
the determination measure is based on calculated optical flows between the first image and the second image.

8. The method according to claim 7 , wherein
for determining the static element an element in the first image and the second image is determined for which the optical flow is smaller than a second threshold or for which the optical flow is zero.

9. The method according to claim 1 , wherein
the imaging means is configured to capture a sequence of images.

10. The method according to claim 9 , wherein
the first image and the second image are selected from the sequence of images generated by the imaging device with a frame rate, and
that the frame rate differs from the time difference between the first image and the at least one second image.

11. The method according to claim 9 , wherein
plural determination measures are calculated for a plurality of pairs of images of the sequence of images.

12. The method according to claim 11 , wherein
an average determination measure for determining the static element is calculated from the calculated plural determination measures.

13. The method according to claim 1 , wherein
the motion parameters of the imaging means correspond to drive parameters of the movable device, the drive parameters comprising at least one of a speed and a turning angle of the movable device.

14. The method according to claim 1 , wherein
the first image and the second image are selected having the capturing times during a turning motion of the imaging means or of the movable device on which the imaging means is mounted.

15. The method according to claim 1 , wherein
the method for determining a static element is started when a turning movement of the imaging means or of the movable device is determined.

16. The method according to claim 1 , wherein
the method comprises a step of determining underexposed or overexposed areas in the first image and the second image,
wherein the underexposed or overexposed areas in the first image and the second image are excluded from determining static elements.

17. The method according to claim 1 , wherein
when a number of determined static elements exceeds a third threshold, the output signal is generated.

18. The method according to claim 17 , wherein
the output signal initiates performing a lens-cleaning procedure or reporting to a user of a determined imaging means state or stopping moving the movable device on which the imaging means is mounted or moving the movable device on which the imaging means is mounted to a cleaning position.

19. An image processing system configured to perform the method according to claim 1 .

20. An imaging device comprising the image processing system according to claim 19 .

21. A movable device equipped with an imaging device according to claim 20 , wherein
the movable device is one of an automobile device, an autonomous movable device, an autonomous lawn mower, a land vehicle, a car, a motorcycle, an aerial vehicle, a sea vehicle, an orbital vehicle, a spacecraft.

22. A computer program product, the computer program product comprising a non-transitory computer readable medium for performing the method according to claim 1 when executed on a computer or signal processor.

Assignees

• Honda Res Inst Europe Gmbh

Inventors

• Einecke Nils
• Deigmöller Jörg

Classifications

• G06T7/215 Primary

  Motion-based segmentation · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• H04N23/811

  by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G01N21/94

  Investigating contamination, e.g. dust (G01N21/85 takes precedence) · CPC title