Obstacle detection device, obstacle detection method, and storage medium storing obstacle detection program

Patent metadata

Field	Value
Publication number	US-11763570-B2
Application number	US-202117355606-A
Country	US
Kind code	B2
Filing date	Jun 23, 2021
Priority date	Jul 2, 2020
Publication date	Sep 19, 2023
Grant date	Sep 19, 2023

Abstract

An obstacle detection device includes a stereo camera that captures IR images on the right side and left side; a point cloud data creation unit that creates point cloud data having three-dimensional position information from the IR images captured by the stereo camera; an obstacle detection unit that detects an obstacle according to point cloud data; an invalid area identification unit that identifies whether an invalid area with no point cloud is present in point cloud data; and a target recognition unit that recognizes a particular target in the invalid area with no point cloud according to an IR image. When a particular target is recognized by the target recognition unit, the obstacle detection unit does not determine the invalid area with no point cloud to be an obstacle.

First claim

What is claimed is:

1. An obstacle detection device comprises:
a stereo camera;
a creation means for creating point cloud data according to a stereo image captured by the stereo camera;
an obstacle detection means for detecting an obstacle according to the created point cloud data; and
a recognition means for recognizing a particular target through image recognition;
wherein the obstacle detection means detects an obstacle by using point cloud data excluding an area recognized by the recognition means as being a particular target and a confidence factor in image recognition by the recognition means.

2. The obstacle detection device according to claim 1 , wherein the recognition means recognizes a particular target in an area equivalent to an invalid area with no point cloud, the invalid area being included in the point cloud data.

3. The obstacle detection device according to claim 2 , wherein the recognition means includes an identification means for identifying an invalid area with no point cloud, the invalid area being included in the point cloud data.

4. The obstacle detection device according to claim 3 , wherein when a particular target is recognized by the recognition means, the obstacle detection means does not determine the invalid area to be an obstacle.

5. An obstacle detection device comprises:
a stereo camera;
a creation means for creating point cloud data according to a stereo image captured by the stereo camera;
an obstacle detection means for detecting an obstacle according to the created point cloud data; and
a recognition means for recognizing a particular target through image recognition;
wherein the obstacle detection means decides whether to detect an obstacle by using a result of recognition by the recognition means according to brightness information about an image used by the recognition means and a confidence factor in image recognition by the recognition means.

6. The obstacle detection device according to claim 5 , wherein:
when the brightness information is lower than or equal to a first threshold value, the obstacle detection means detects an obstacle by using only the point cloud data; and
when the brightness information exceeds the first threshold value, the obstacle detection means detects an obstacle by using both the point cloud data and the result of recognition by the recognition means.

7. The obstacle detection device according to claim 5 , wherein:
when the brightness information is lower than or equal to the first threshold value and the confidence factor is lower than or equal to a second threshold value, the obstacle detection means detects an obstacle by using only the point cloud data; and
when at least one of the first threshold value for the brightness information and the second threshold value for the confidence factor is exceeded, the obstacle detection means detects an obstacle by using both the point cloud data and the result of recognition by the recognition means.

8. The obstacle detection device according to claim 7 , wherein the first threshold value for the brightness information is set for each particular target.

9. The obstacle detection device according to claim 1 , wherein:
the particular target is a target that does not become an obstacle; and
the particular target is determined in advance.

10. The obstacle detection device according to claim 2 , wherein:
the point cloud data is three-dimensional information including distance information obtained by combining an image captured by the stereo camera; and
the invalid area with no point cloud is an area that lacks distance information.

11. The obstacle detection device according to claim 1 , wherein the recognition means recognizes a particular target according to an image captured before point cloud data is created.

12. The obstacle detection device according to claim 1 , wherein the recognition means recognizes a particular target according to an image captured by another camera different from the stereo camera, the image being synchronized with the stereo image captured by the stereo camera.

13. The obstacle detection device according to claim 1 , wherein the stereo camera is an infrared camera.

14. The obstacle detection device according to claim 1 , wherein the stereo camera captures a scene in front of a moving body.

15. An obstacle detection method that detects an obstacle by using a stereo image captured by a stereo camera, the method comprising:
creating point cloud data including distance information about a captured image, according to the stereo image captured by the stereo camera;
identifying whether an invalid area with no point cloud is present in the point cloud data;
recognizing, through image recognition, whether a particular target is present in an invalid area with no point cloud; and
detecting, when a particular target is recognized, an obstacle by using point cloud data excluding the invalid area with no point cloud and a confidence factor in the image recognition.

16. An obstacle detection method that detects an obstacle by using a stereo image captured by a stereo camera, the method comprising:
creating point cloud data including distance information about a captured target, according to the stereo image captured by the stereo camera;
identifying whether an invalid area with no point cloud is present in the point cloud data;
recognizing, through image recognition, whether a particular target is present in an invalid area with no point cloud; and
deciding whether to detect an obstacle by using a result of recognition in the recognizing according to brightness information about an image used in the recognizing and a confidence factor in the image recognition.

17. A non-transitory computer-readable storage medium that stores an obstacle detection program executed by an electronic device, the program comprising:
creating point cloud data including distance information about a captured image, according to a stereo image captured by a stereo camera;
identifying whether an invalid area with no point cloud is present in the point cloud data;
recognizing, through image recognition, whether a particular target is present in an invalid area with no point cloud; and
detecting, when a particular target is recognized, an obstacle by using point cloud data excluding the invalid area with no point cloud and a confidence factor in the image recognition.

18. A non-transitory computer-readable storage medium that stores an obstacle detection program executed by an electronic device, the program comprising:
creating point cloud data including distance information about a captured target, according to a stereo image captured by a stereo camera;
identifying whether an invalid area with no point cloud is present in the point cloud data;
recognizing, through image recognition, whether a particular target is present in an invalid area with no point cloud; and
deciding whether to detect an obstacle by using a result of recognition in the recognizing according to brightness information about an image used in the recognizing and a confidence factor in the image recognition.

Assignees

• Alps Alpine Co Ltd

Inventors

• Mashiko Kaoru
• Nakagawa Tomoki
• Matsui Tatsuya

Classifications

• G01C11/04 Primary

  Interpretation of pictures · CPC title

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G01C11/02

  Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures · CPC title

• G06T7/593

  from stereo images · CPC title

• G06T7/73

  using feature-based methods · CPC title