Coarse-to-fine attention networks for light signal detection and recognition

Patent metadata

Field	Value
Publication number	US-12087064-B2
Application number	US-202318230775-A
Country	US
Kind code	B2
Filing date	Aug 7, 2023
Priority date	Sep 4, 2020
Publication date	Sep 10, 2024
Grant date	Sep 10, 2024

Abstract

A vehicle light signal detection and recognition method, system, and computer program product include bounding, using a coarse attention module, one or more regions of an image of an automobile including at least one of a brake light and a signal light generated by automobile signals which include illuminated sections to generate one or more bounded region, removing, using a fine attention module, noise from the one or more bounded regions to generate one or more noise-free bounded regions, and identifying the at least one of the brake light and the signal light from the one or more noise-free bounded regions.

First claim

What is claimed is:

1. A computer-implemented vehicle light signal detection and recognition method, the method comprising:
bounding as a bounded region, using a coarse attention module with an input of a smaller feature map, self-luminous objects in one or more regions of an image of an automobile including at least one of a brake light and a signal light generated by automobile signals;
obtaining a first refined feature map, using the coarse attention module, by calculating an average of all previous features of a same category of the bounded region as a channel expected attention score (C-E score) of each category which is multiplied by the smaller feature map;
generating, using a fine attention module with an input of a bigger feature map, a second refined feature map to localize precise discriminative regions for the bounded region by using the bigger feature map, wherein the smaller feature map and the bigger feature map are generated by a Region of Interest (ROI) pooling with different Kernels in an original feature map; and
performing classification and localization by converting the first refined feature map and the second refined feature map to a value to calculate coordinates in the bounded region and categories of objects in the bounding regions.

2. The method of claim 1 , embodied in a cloud-computing environment.

3. A computer-implemented vehicle light signal detection and recognition method, the method comprising:
bounding as a bounded region, using a coarse attention module with an input of a smaller feature map, self-luminous objects in one or more regions of an image of an automobile including at least one of a brake light and a signal light generated by automobile signals,
wherein the coarse attention module includes an attention score branch and an expected score learning branch,
wherein, in the attention score learning branch, the coarse attention module converts the smaller feature map into an original feature vector through global average pooling (GAP) which is used by the coarse attention module to calculate a coarse attention score (C-A score), and
wherein, in the expected score learning branch, the coarse attention module calculates an average of all previous features of a same category of the bounded region as a channel expected attention score (C-E score) of each category, which is multiplied by the smaller feature map to obtain a first refined feature map;
generating, using a fine attention module with an input of a bigger feature map, a second refined feature map to localize precise discriminative regions for the bounded region by using the C-A score, wherein the smaller feature map and the bigger feature map are generated by a Region of Interest (ROI) pooling with different Kernels in an original feature map; and
performing classification and localization by converting the first refined feature map and the second refined feature map to a value to calculate coordinates in the bounded region and categories of objects in the bounding regions.

4. The method of claim 3 , embodied in a cloud-computing environment.

5. A computer program product, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
bounding as a bounded region, using a coarse attention module with an input of a smaller feature map, self-luminous objects in one or more regions of an image of an automobile including at least one of a brake light and a signal light generated by automobile signals;
obtaining a first refined feature map, using the coarse attention module, by calculating an average of all previous features of a same category of the bounded region as a channel expected attention score (C-E score) of each category which is multiplied by the smaller feature map;
generating, using a fine attention module with an input of a bigger feature map, a second refined feature map to localize precise discriminative regions for the bounded region by using the bigger feature map, wherein the smaller feature map and the bigger feature map are generated by a Region of Interest (ROI) pooling with different Kernels in an original feature map; and
performing classification and localization by converting the first refined feature map and the second refined feature map to a value to calculate coordinates in the bounded region and categories of objects in the bounding regions.

Assignees

• IBM

Inventors

• Wu Bo
• Gan Chuang
• Zhang Yang
• Wang Dakuo

Classifications

• G06V30/194

  References adjustable by an adaptive method, e.g. learning · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title