Imaging system, driving assistance system, and program

Patent metadata

Field	Value
Publication number	US-12315107-B2
Application number	US-202117792053-A
Country	US
Kind code	B2
Filing date	Jan 6, 2021
Priority date	Jan 17, 2020
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

The driving assistance system includes an imaging device capable of capturing a first monochrome image in a vehicle traveling direction, a first neural network for segmentation processing, a second neural network for depth estimation processing, a determination portion determining a center of a portion cut off from the first monochrome image on the basis of the segmentation processing and the depth estimation processing, a third neural network for colorization processing of only a second cut-off monochrome image, and a display device for enlargement of the second monochrome image subjected to the colorization processing.

First claim

The invention claimed is:

1. A driving assistance method comprising the steps of:
driving a vehicle incorporating an imaging device;
capturing a monochrome image of a front of the vehicle during driving by the imaging device;
performing inference of regions of at least a sky, a car, and a road by performing segmentation processing on the monochrome image;
performing inference of a distant region to be extracted from the monochrome image by performing depth estimation processing on the monochrome image;
determining a center of the distant region to be extracted on the basis of the segmentation processing and the depth estimation processing;
measuring driving speed of the vehicle and determining size of the distant region to be extracted according to the driving speed of the vehicle;
extracting the distant region with a determined size from the monochrome image;
performing super-resolution processing to an extracted distant region;
after performing the super-resolution processing, performing colorization processing for highlighting of an object included in the extracted distant region; and
performing enlargement of the extracted distant region after the colorization processing.

2. The driving assistance method according to claim 1 ,
wherein the segmentation processing uses first neural network processing,
wherein the depth estimation processing uses second neural network processing,
wherein the super-resolution processing uses third neural network processing, and
wherein the colorization processing uses fourth neural network processing.

3. The driving assistance method according to claim 1 , wherein the imaging device does not include a color filter.

4. The driving assistance method according to claim 1 ,
wherein when the vehicle moves at a first driving speed, the size of the distant region is larger than when the vehicle moves at a second driving speed, and
wherein the first driving speed is faster than the second driving speed.

5. The driving assistance method according to claim 1 , wherein the distant region includes at least a road edge portion.

6. A computer program product comprising a non-transitory computer readable medium encoded with an information processing program for use in an information processing device, the program when executed performs the operations comprising:
driving a vehicle incorporating an imaging device;
capturing a monochrome image of a front of the vehicle during driving by the imaging device;
performing inference of regions of at least a sky, a car, and a road by performing segmentation processing on the monochrome image;
performing inference of a distant region to be extracted from the monochrome image by performing depth estimation processing on the monochrome image;
determining a center of the distant region to be extracted on the basis of the segmentation processing and the depth estimation processing;
measuring driving speed of the vehicle and determining size of the distant region to be extracted according to the driving speed of the vehicle;
extracting the distant region with a determined size from the monochrome image;
performing super-resolution processing on an extracted distant region;
after performing the super-resolution processing, performing colorization processing for highlighting of an object included in the extracted distant region; and
performing enlargement of the extracted distant region after the colorization processing.

Assignees

• Semiconductor Energy Lab

Inventors

• Akimoto Kengo
• Inoue Koki
• Koumura Yusuke

Classifications

• G06T3/4046

  using neural networks · CPC title

• H04N9/43

  Conversion of monochrome picture signals to colour picture signals for colour picture display · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10024

  Color image · CPC title

• G06V10/82

  using neural networks · CPC title