Ventral-dorsal neural networks: object detection via selective attention

Patent metadata

Field	Value
Publication number	US-11475658-B2
Application number	US-202117178822-A
Country	US
Kind code	B2
Filing date	Feb 18, 2021
Priority date	Sep 21, 2018
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.

First claim

The invention claimed is:

1. A computer-implemented method for object detection within a visual medium, comprising:
receiving the visual medium comprising one or more objects;
identifying, using a first neural network, one or more irrelevant visual regions within the visual medium;
generating, based on the one or more irrelevant visual regions, a visual mask to be applied to the visual medium;
applying the visual mask to the visual medium to generate a masked visual medium, wherein applying the visual mask to the visual medium causes the one or more irrelevant visual regions to be filtered out from the visual medium;
identifying, using a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.

2. The computer-implemented method of claim 1 , wherein applying the visual mask to the visual medium includes modifying pixel intensity values associated with the one or more irrelevant visual regions.

3. The computer-implemented method of claim 2 , wherein pixel intensity values associated with one or more relevant visual regions are non-zero after applying the visual mask to the visual medium.

4. The computer-implemented method of claim 2 , wherein the pixel intensity values associated with the one or more irrelevant visual regions are zero after applying the visual mask to the visual medium.

5. The computer-implemented method of claim 1 , wherein the visual mask comprises a data structure containing pixel values.

6. The computer-implemented method of claim 1 , wherein the first neural network is a deep convolutional attention based object detection neural network.

7. The computer-implemented method of claim 1 , wherein the second neural network is a supervised object detection neural network.

8. A non-transitory computer-readable storage medium having stored thereon instructions for causing at least one computer system to detect objects within a visual medium, the instructions comprising:
receiving the visual medium comprising one or more objects;
identifying, using a first neural network, one or more irrelevant visual regions within the visual medium;
generating, based on the one or more irrelevant visual regions, a visual mask to be applied to the visual medium;
applying the visual mask to the visual medium to generate a masked visual medium, wherein applying the visual mask to the visual medium causes the one or more irrelevant visual regions to be filtered out from the visual medium;
identifying, using a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.

9. The non-transitory computer-readable storage medium of claim 8 , wherein applying the visual mask to the visual medium includes modifying pixel intensity values associated with the one or more irrelevant visual regions.

10. The non-transitory computer-readable storage medium of claim 9 , wherein pixel intensity values associated with one or more relevant visual regions are non-zero after applying the visual mask to the visual medium.

11. The non-transitory computer-readable storage medium of claim 9 , wherein the pixel intensity values associated with the one or more irrelevant visual regions are zero after applying the visual mask to the visual medium.

12. The non-transitory computer-readable storage medium of claim 8 , wherein the visual mask comprises a data structure containing pixel values.

13. The non-transitory computer-readable storage medium of claim 8 , wherein the first neural network is a deep convolutional attention based object detection neural network.

14. The non-transitory computer-readable storage medium of claim 8 , wherein the second neural network is a supervised object detection neural network.

15. A system for detecting objects within a visual medium, comprising:
one or more processors; and
a memory coupled with the one or more processors, the memory configured to store instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
receiving the visual medium comprising one or more objects;
identifying, using a first neural network, one or more irrelevant visual regions within the visual medium;
generating, based on the one or more irrelevant visual regions, a visual mask to be applied to the visual medium;
applying the visual mask to the visual medium to generate a masked visual medium, wherein applying the visual mask to the visual medium causes the one or more irrelevant visual regions to be filtered out from the visual medium;
identifying, using a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.

16. The system of claim 15 , wherein applying the visual mask to the visual medium includes modifying pixel intensity values associated with the one or more irrelevant visual regions.

17. The system of claim 16 , wherein pixel intensity values associated with one or more relevant visual regions are non-zero after applying the visual mask to the visual medium.

18. The system of claim 16 , wherein the pixel intensity values associated with the one or more irrelevant visual regions are zero after applying the visual mask to the visual medium.

19. The system of claim 15 , wherein the first neural network is a deep convolutional attention based object detection neural network.

20. The system of claim 15 , wherein the second neural network is a supervised object detection neural network.

Assignees

• Ancestry Com Operations Inc

Inventors

• Ebrahimpour Mohammad K
• Yu Yen-Yun
• Li Jiayun
• Reese Jack
• Moghtaderi Azadeh

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06N3/08 Primary

  Learning methods · CPC title