Image processing apparatus and external environment recognition apparatus

Patent metadata

Field	Value
Publication number	US-11461595-B2
Application number	US-201716469770-A
Country	US
Kind code	B2
Filing date	Dec 13, 2017
Priority date	Dec 16, 2016
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

An image processing apparatus includes: an object detection section that performs convolution computation on an input image based on a captured image obtained by capturing the image with a camera, and that detects an object; a feature map validation section that performs feature map validation validating a likelihood that the input image contains the object on the basis of a feature map obtained by the convolution computation; a time series validation section that performs time series validation validating a result of the feature map validation performed by the feature map validation section in time series; and a detection result correction section that corrects a detection result about the object output by the object detection section on the basis of a result of the time series validation performed by the time series validation section.

First claim

The invention claimed is:

1. An image processing apparatus comprising:
a memory;
an input/output device; and
a processor communicatively coupled to the memory and the input/output device, wherein the processor is configured to:
perform convolution computation on an input image based on a captured image obtained by capturing the image with a camera, and that detects an object,
perform feature map validation validating a likelihood that the input image contains the object on a basis of a feature map obtained by the convolution computation,
perform time series validation validating in time series a result of the feature map validation,
correct a detection result about the object, the detection result being output by input/output device, when result of the time series validation does not satisfy a predetermined condition,
check timewise consecutiveness of an output about the result of the feature map validation, and correct the result of the feature map validation on a basis of a determination condition whether or not the output is consecutive a predetermined set times or more, and
correct the detection result about the object, the detection result being output by the input/out device, on a basis of a correction result about the result of the feature map validation, wherein

the feature map is configured with a plurality of blocks,
the processor determines whether or not each of the plurality of blocks configuring each of the feature maps indicates features of the object, and performs the feature map validation on a basis of placement of blocks determined to indicate the features of the object in the feature map and blocks determined not to indicate the features of the object in the feature map, and
the processor outputs a signal indicating that the object cannot be detected when a case in which the result of the time series validation does not satisfy the predetermined condition continues for a predetermined period of time.

2. The image processing apparatus according to claim 1 , wherein
the processor performs the convolution computation on the input image a plurality of times, and
the processor performs the feature map validation on each of a plurality of the feature maps obtained by the convolution computation performed the plurality of times.

3. The image processing apparatus according to claim 1 , wherein
the processor performs the feature map validation on a basis of a comparison result of comparison of the placement with a placement pattern stored in advance.

4. The image processing apparatus according to claim 1 , wherein
the image processing apparatus is mounted in an own vehicle, and
the object is an other vehicle present around the own vehicle.

5. The image processing apparatus according to claim 4 , wherein the processor is further configured to
set a direction in which the object is to be preferentially sensed on a basis of a driving state of the own vehicle,
perform the feature map validation on a basis of a comparison result of comparison of the placement with a placement pattern stored in advance, and
determine the placement pattern for use in the comparison, on a basis of the direction set.

6. The image processing apparatus according to claim 3 , wherein
the image processing apparatus is mounted in an own vehicle, and
the object is an other vehicle present around the own vehicle.

7. An external environment recognition apparatus comprising:
an image processing apparatus comprising:
a memory;
an input/output device; and
a processor communicatively coupled to the memory and the input/output device, wherein the processor is configured to:
perform convolution computation on an input image based on a captured image obtained by capturing the image with a camera, and that detects an object,
perform feature map validation validating a likelihood that the input image contains the object on a basis of a feature map obtained by the convolution computation,
perform time series validation validating in time series a result of the feature map validation,
correct a detection result about the object, the detection result being output by input/output device, when result of the time series validation does not satisfy a predetermined condition,
check timewise consecutiveness of an output about the result of the feature map validation, and correct the result of the feature map validation on a basis of a determination condition whether or not the output is consecutive a predetermined set times or more, and
correct the detection result about the object, the detection result being output by the input/out device, on a basis of a correction result about the result of the feature map validation, wherein

the feature map is configured with a plurality of blocks,
the processor determines whether or not each of the plurality of blocks configuring each of the feature maps indicates features of the object, and performs the feature map validation on a basis of placement of blocks determined to indicate the features of the object in the feature map and blocks determined not to indicate the features of the object in the feature map, and
the processor outputs a signal indicating that the object cannot be detected when a case in which the result of the time series validation does not satisfy the predetermined condition continues for a predetermined period of time,
the image processing apparatus is mounted in an own vehicle,
the object is an other vehicle present around the own vehicle, and
the external environment recognition apparatus outputs at least one of a warning signal for warning a driver of the own vehicle and a vehicle control signal for controlling an operation of the own vehicle, on a basis of a corrected sensing result about the other vehicle, the corrected detection result being corrected by the processor.

8. An image processing apparatus comprising:
a memory;
an input/output device; and
a processor communicatively coupled to the memory and the input/output device, wherein the processor is configured to:
perform convolution computation on an input image based on a captured image obtained by capturing the image with a camera, and that detects an object,
perform feature map validation validating a likelihood that the input image contains the object by comparing a feature map obtained by the convolution computation with a pattern for the feature map, the pattern being stored in advance and related to the object,
perform time series validation validating in time series a result of the feature map validation,
correct a detection result about the object, the detection result being output by the input/output device, when a result of the time series validation does not satisfy a predetermined condition,
check timewise consecutiveness of an output about the result of the feature map validation, and correct the result of the feature map validation on a basis of a determination condition whether or not the output is consecutive a predetermined set times or more, and
correct the detection result about the object, the detection result being output by the input/out device, on a basis of a correction result about the result of the feature map validation.

9. The image processing apparatus according to claim 1 , wherein
the predetermined condition is that a predetermined result of the feature map validation continues a predetermined number of times consecutively in a time series.

10. The image processing apparatus according to claim 8 , wherein
the predetermined condition is that a predetermined result of the feature map validation continues a predetermined number of times consecutively in a time series.

Assignees

• Clarion Co Ltd

Inventors

• Akiyama Yasuhiro
• Hamada Koichi

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06T1/00

  General purpose image data processing · CPC title