Environment Recognition System and Environment Recognition Method

Patent metadata

Field	Value
Publication number	US-2023282003-A1
Application number	US-202118017262-A
Country	US
Kind code	A1
Filing date	Apr 19, 2021
Priority date	Jul 31, 2020
Publication date	Sep 7, 2023
Grant date	—

Abstract

Provided is an external environment recognition system which complements a blind spot of an on-board camera occurring due to a moving obstacle such as a preceding vehicle with a photographed image obtained from an on-road camera, and can generate a detailed map for automatic traveling. To this end, the external environment recognition system composites a photographed image from an on-board camera with a photographed image from an external camera to recognize the environment outside the host vehicle, the external environment recognition system comprising: an on-board camera which photographs an area in front of the host vehicle; an on-board communication unit which communicates with the external camera; a front moving body detection unit which detects a moving body in the front area on the basis of the photographed image from the on-board camera; a blind spot extraction unit which extracts a blind spot caused by the moving body; an image conversion unit which converts the received photographed image from the external camera into a viewpoint of the on-board camera; an image compositing unit which composites the viewpoint conversion image from the image conversion unit with the blind spot of the photographed image from the on-board camera; and a detailed map generation unit which generates a detailed map in the traveling direction of the host vehicle on the basis of the composited image from the image compositing unit.

First claim

1 . An environment recognition system that recognizes an environment external to a subject vehicle by synthesizing an image captured by an onboard camera with an image captured by an external camera, the environment recognition system comprising:
the onboard camera that captures an image in front of the subject vehicle; an onboard communicating unit that communicates with the external camera; a front moving object detecting unit that detects a moving object in front, based on the image captured by the onboard camera; a blind spot extracting unit that extracts a blind spot created by the moving object; an image converting unit that converts an image captured by and received from the external camera into an image from a viewpoint of the onboard camera; an image synthesizing unit that synthesizes the image having the viewpoint converted by the image converting unit, to the blind spot included in the image captured by the onboard camera; and a detailed map generating unit that generates a detailed map of a direction ahead of the subject vehicle based on the image synthesized by the image synthesizing unit.

2 . An environment recognition system that recognizes an environment external to a subject vehicle by synthesizing an image captured by an onboard camera with an image captured by an external camera, the environment recognition system comprising:
the onboard camera that captures an image in front of the subject vehicle; an onboard communicating unit that communicates with the external camera; a front moving object detecting unit that detects a moving object in front, based on the image captured by the onboard camera; an image converting unit that requests transmission of a captured image corresponding to a blind spot created by the moving object from the external camera, and that converts an image captured by and received from the external camera into an image from a viewpoint of the onboard camera; an image synthesizing unit that synthesizes the image having the viewpoint converted by the image converting unit, to the blind spot included in the image captured by the onboard camera; and a detailed map generating unit that generates a detailed map of a direction ahead of the subject vehicle based on the image synthesized by the image synthesizing unit.

3 . The environment recognition system according to claim 1 , wherein
the onboard communicating unit receives a plurality of captured images captured at different points in time from the external camera, and the environment recognition system further comprises: a moving object recognizing unit that recognizes a moving object in the received plurality of captured images; a moving object deleting unit that deletes the moving object from the received plurality of captured images; and an external camera image synthesizing unit that synthesizes an image without the moving object by synthesizing the captured images resultant of deleting the moving object.

4 . The environment recognition system according to claim 1 , wherein, when there are a plurality of external cameras available, the image converting unit uses a captured image of an external camera closer to the viewpoint of the onboard camera as an image to be processed.

5 . The environment recognition system according to claim 3 , wherein, when there are a plurality of external cameras available, the moving object recognizing unit uses a captured image captured by an external camera closest to the viewpoint of the onboard camera as an image to be processed.

6 . An environment recognition method by which an external environment of a subject vehicle is recognized by synthesizing an image captured by an onboard camera with an image captured by an external camera, the environment recognition method comprising:
a step of detecting a moving object in front based on the image captured by the onboard camera; a step of extracting a blind spot created by the moving object; a step of converting the image captured by the external camera into an image from a viewpoint of the onboard camera; a step of synthesizing the image having the viewpoint converted, to the blind spot included in the image captured by the onboard camera; and a step of generating a detailed map of a direction ahead of the subject vehicle based on the synthesized image.

7 . The environment recognition system according to claim 2 , wherein, when there are a plurality of external cameras available, the image converting unit uses a captured image of an external camera closer to the viewpoint of the onboard camera as an image to be processed.

Assignees

• Hitachi Ltd

Inventors

• Matsuo Shigeru
• Kiyohara Masahiro
• Kido Hideaki

Classifications

• G01C21/3602 Primary

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title

• G01C21/3807

  characterised by the type of data · CPC title

• G01C21/3841

  Data obtained from two or more sources, e.g. probe vehicles · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title