Ventral-dorsal neural networks: object detection via selective attention

Patent metadata

Field	Value
Publication number	US-10949666-B2
Application number	US-202017018611-A
Country	US
Kind code	B2
Filing date	Sep 11, 2020
Priority date	Sep 21, 2018
Publication date	Mar 16, 2021
Grant date	Mar 16, 2021

Abstract

Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.

First claim

The invention claimed is:

1. A computer-implemented method for object detection within a visual medium, comprising:
receiving a visual medium comprising a plurality of objects;
identifying, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
identifying, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
computing a measure of relevance for a plurality of pixel locations;
generating, based at least on the measure of relevance for the plurality of pixel locations, a visual mask comprising a data structure containing pixel values;
applying the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
identifying, via a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.

2. The computer-implemented method of claim 1 , wherein pixel intensity values associated with the one or more relevant visual regions are non-zero.

3. The computer-implemented method of claim 1 , wherein pixel intensity values associated with the one or more irrelevant visual regions are zero.

4. The computer-implemented method of claim 1 , wherein the first neural network is a deep convolutional attention based object detection neural network.

5. The computer-implemented method of claim 1 , wherein the second neural network is a supervised object detection neural network.

6. The computer-implemented method of claim 1 , wherein identifying, via the first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium further comprises:
extracting convolutional features from the visual medium and aggregating the extracted convolutional features into a Gestalt Total output.

7. The computer-implemented method of claim 1 , wherein the second neural network is utilized on the masked visual medium portion of the visual medium.

8. A non-transitory computer-readable storage medium having stored thereon instructions for causing at least one computer system to detect objects within a visual medium, the instructions comprising:
receiving a visual medium comprising a plurality of objects;
identifying, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
identifying, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
computing a measure of relevance for a plurality of pixel locations;
generating, based at least on the measure of relevance for the plurality of pixel locations, a visual mask comprising a data structure containing pixel values;
applying the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
identifying, via a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.

9. The non-transitory computer-readable storage medium of claim 8 , wherein pixel intensity values associated with the one or more relevant visual regions are non-zero.

10. The non-transitory computer-readable storage medium of claim 8 , wherein pixel intensity values associated with the one or more irrelevant visual regions are zero.

11. The non-transitory computer-readable storage medium of claim 8 , wherein the first neural network is a deep convolutional attention based object detection neural network.

12. The non-transitory computer-readable storage medium of claim 8 , wherein the second neural network is a supervised object detection neural network.

13. The non-transitory computer-readable storage medium of claim 8 , wherein identifying, via the first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium further comprises:
extracting convolutional features from the visual medium and aggregating the extracted convolutional features into a Gestalt Total output.

14. The non-transitory computer-readable storage medium of claim 8 , wherein the second neural network is only utilized on the masked visual medium portion of the visual medium.

15. A system for detecting objects within a visual medium, comprising:
one or more processors; and
a memory coupled with the one or more processors, the memory configured to store instructions that when executed by the one or more processors cause the one or more processors to:
receive a visual medium comprising a plurality of objects;
identify, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
identifying, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
computing a measure of relevance for a plurality of pixel locations;
generating, based at least on the measure of relevance for the plurality of pixel locations, a visual mask comprising a data structure containing pixel values;
apply the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
identify, via a second neural network, one or more objects of interest within the masked visual medium; and
output an identification of the one or more objects of interest.

16. The system of claim 15 , wherein pixel intensity values associated with the one or more relevant visual regions are non-zero.

17. The system of claim 15 , wherein pixel intensity values associated with the one or more irrelevant visual regions are zero.

18. The system of claim 15 , wherein the first neural network is a deep convolutional attention based object detection neural network.

19. The system of claim 15 , wherein the second neural network is a supervised object detection neural network.

20. The system of claim 15 , wherein identify, via the first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium further comprises:
extract convolutional features from the visual medium and aggregating the extracted convolutional features into a Gestalt Total output.

Assignees

• Ancestry Com Operations Inc

Inventors

• Ebrahimpour Mohammad K
• Yu Yen-Yun
• Li Jiayun
• Reese Jack
• Moghtaderi Azadeh

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06N3/08 Primary

  Learning methods · CPC title