Surrounding environment recognition device

Patent metadata

Field	Value
Publication number	US-9773177-B2
Application number	US-201415035812-A
Country	US
Kind code	B2
Filing date	Oct 15, 2014
Priority date	Nov 14, 2013
Publication date	Sep 26, 2017
Grant date	Sep 26, 2017

Abstract

A surrounding environment recognition device includes: an image acquisition unit that obtains a photographic image from a camera for capturing, via a camera lens, an image of a surrounding environment around a mobile object; an image recognition unit that recognizes an object image of an object present in the surrounding environment based upon the photographic image obtained by the image acquisition unit; an accumulation detection unit that detects accumulation settled on the camera lens based upon the photographic image obtained by the image acquisition unit; a suspension decision-making unit that makes a decision, based upon detection results provided by the accumulation detection unit, whether or not to suspend operation of the image recognition unit; a tracking unit that detects a characteristic quantity in a tracking target image from a specific area in an image captured by the image acquisition unit for a reference frame, determines through calculation an estimated area where the characteristic quantity should be detected in an image captured by the image acquisition unit for a later frame relative to the reference frame and makes a decision as to whether or not the characteristic quantity for the tracking target image is also present in the estimated area; and a resumption decision-making unit that makes a decision, based upon at least decision-making results provided by the tracking unit and indicating whether or not the characteristic quantity for the tracking target image is present, as to whether or not to resume the operation of the image recognition unit currently in a suspended state.

First claim

The invention claimed is:

1. A surrounding environment recognition device, comprising:
an image acquisition unit that obtains a photographic image from a camera for capturing, via a camera lens, an image of a surrounding environment around a mobile object;
an image recognition unit that recognizes an object image of an object present in the surrounding environment based upon the photographic image obtained by the image acquisition unit;
an accumulation detection unit that detects accumulation settled on the camera lens based upon the photographic image obtained by the image acquisition unit;
a suspension decision-making unit that makes a decision, based upon detection results provided by the accumulation detection unit, whether or not to suspend operation of the image recognition unit;
a tracking unit that detects a characteristic quantity in a tracking target image from a specific area in an image captured by the image acquisition unit for a reference frame, determines through calculation an estimated area where the characteristic quantity should be detected in an image captured by the image acquisition unit for a later frame relative to the reference frame and makes a decision as to whether or not the characteristic quantity for the tracking target image is also present in the estimated area; and
a resumption decision-making unit that makes a decision, based upon at least decision-making results provided by the tracking unit and indicating whether or not the characteristic quantity for the tracking target image is present, as to whether or not to resume the operation of the image recognition unit currently in a suspended state.

2. The surrounding environment recognition device according to claim 1 , wherein:
the resumption decision-making unit allows the operation of the image recognition unit currently in the suspended state to be resumed if the tracking unit judges that the characteristic quantity for the tracking target image is present.

3. The surrounding environment recognition device according to claim 1 , wherein:
the suspension decision-making unit decides that the operation of the image recognition unit is to be suspended based upon detection results provided by the accumulation detection unit and indicating that the camera lens is dirty enough to adversely affect image-capturing operation; and
the resumption decision-making unit makes a decision as to whether or not to resume the operation of the image recognition unit, having been suspended based upon the detection results provided by the accumulation detection unit upon deciding that the camera lens is dirty enough to adversely affect the image-capturing operation, if a dirt build-up condition at the camera lens improves to an extent at which the tracking unit is able to judge that the characteristic quantity for the tracking target image is present.

4. The surrounding environment recognition device according to claim 1 , wherein:
upon judging that the characteristic quantity for the tracking target image is present, the tracking unit increases an index pertaining to visibility through the camera lens over the specific area and the estimated area by a predetermined extent, whereas upon determining that the characteristic quantity for the tracking target image is not present, the tracking unit does not increase the index pertaining to the visibility; and
the resumption decision-making unit decides that the operation of the image recognition unit currently in the suspended state is to be resumed if a total sum of index values pertaining to a plurality of types of visibility each corresponding to one of a plurality of image areas in the photographic image is equal to or greater than a predetermined threshold value.

5. The surrounding environment recognition device according to claim 4 , wherein:
the resumption decision-making unit decides that the operation of the image recognition unit currently in the suspended state is to be resumed if a resumption condition set based upon decision-making results provided by the tracking unit indicating whether or not the characteristic quantity for the tracking target image is present, and a resumption condition set based upon detection results provided by the accumulation detection unit, all exist.

6. The surrounding environment recognition device according to claim 5 , wherein:
if an accumulation is detected in the photographic image obtained by the image acquisition unit, the accumulation detection unit increases an index indicating an accumulation condition in an image area where the accumulation is detected by a predetermined value;
the resumption decision-making unit obtains through calculation information related to a lens transmission factor of the camera lens and information related to an accumulation area over which an accumulation is present based upon the index indicating the accumulation condition; and
the resumption decision-making unit makes a decision as to whether or not an improvement is achieved at least either in the lens transmission factor or in the accumulation area at the camera lens as the resumption condition set based upon the detection results provided by the accumulation detection unit.

7. The surrounding environment recognition device according to claim 6 , wherein:
the accumulation detection unit individually detects a plurality of types of accumulation;
the surrounding environment recognition device further comprises:
a detection results integrating unit that generates, through calculation, an integrated detection result by integrating a plurality of sets of detection results based upon results obtained by detecting the plurality of types of accumulation; and
the resumption decision-making unit decides that the operation of the image recognition unit currently in the suspended state is to be resumed if the resumption condition set based upon the decision-making results provided by the tracking unit and indicating whether or not the characteristic quantity for the tracking target image is present, a resumption condition set based upon information pertaining to the lens transmission factor and the accumulation area at the camera lens and a resumption condition set based upon a change occurring over time in the integrated detection result, all exist.

8. The surrounding environment recognition device according to claim 7 , wherein:
the integrated detection result is indicated as coordinates on a coordinate system assuming a first coordinate axis set in relation to a lens transmission factor by the accumulation and a second coordinate axis set in relation to the accumulation area over which an accumulation is present at the camera lens;
along the first coordinate axis, a higher lens transmission factor is indicated at a point closer to an origin point of the coordinate system;
along the second coordinate axes, a smaller accumulation area is indicated at a point closer to the origin point; and
the resumption decision-making unit makes a decision as to whether or not coordinates indicated by the integrated detection result have moved closer to the origin point continuously over a predetermined length of time as the resumption condition set based upon a change occurring over time in the integrated detection result.

9. The surrounding environment recognition device according to claim 5 , wherein:
the accumulation detection unit individually detects a plurality of types of accumulation;
the surrounding environment recognition device further comprises:
a detection results integrating unit that generates, through calculation, an integrated detection result by integrating a plurality of sets of detection results based upon results obtained by detecting the plurality of types of accumulation; and

Assignees

• Clarion Co Ltd

Inventors

• Takemura Masayuki
• Kiyohara Masahiro
• Irie Kota
• Sakata Masao
• Uchida Yoshitaka

Classifications

• H04N23/811

  by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor · CPC title

• G06K9/00791

  Physics · mapped topic

• B60R2300/305

  merging camera image with lines or icons · CPC title

• H04N5/2171

  Electricity · mapped topic

• B60R2300/8093

  for obstacle warning · CPC title