Method for processing information, information processing apparatus, and non-transitory computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-10796184-B2
Application number	US-201916394062-A
Country	US
Kind code	B2
Filing date	Apr 25, 2019
Priority date	Nov 9, 2016
Publication date	Oct 6, 2020
Grant date	Oct 6, 2020

Abstract

Inputting an image to a neural network, performing convolution on a current frame included in the image to calculate a current feature map, which is a feature map at a present time, combining a past feature map, which is obtained by performing convolution on a past frame included in the image, and the current feature map, estimating an object candidate area using the combined past feature map and current feature map, estimating positional information and identification information regarding the one or more objects included in the current frame using the combined past feature map and current feature map and the estimated object candidate area, and outputting the positional information and the identification information regarding the one or more objects included in the current frame of the image estimated in the estimating as object detection results are included.

First claim

What is claimed is:

1. A method for processing information achieved by a computer using a neural network, the method comprising:
inputting an image including one or more objects to the neural network;
causing a convolutional layer included in the neural network to perform convolution on a current frame included in the image to calculate a current feature map; which is a feature map at a present time;
causing a combiner for combining two or more feature maps into one feature map to combine a past feature map, which is a feature map obtained by causing the convolutional layer to perform convolution on a past frame included in the image and preceding the current frame, and the current feature map;
causing a region proposal network included in the neural network to estimate an object candidate area using the combined past feature map and current feature map, the region proposal network being used to estimate the object candidate area;
causing a region of interest pooling layer included in the neural network to estimate positional information and identification information regarding the one or more objects included in the current frame using the combined past feature map and current feature map and the estimated object candidate area, the region of interest pooling layer being used to perform class estimation; and
outputting the positional information and the identification information regarding the one or more objects included in the current frame of the image estimated in the causing as object detection results.

2. The method according to claim 1 ,
wherein the neural network includes three or more convolutional layers,
wherein one of the three or more convolutional layers is caused to perform convolution on the current frame included in the image to calculate the current feature map, and
wherein the corresponding ones of the three or more convolutional layers other than the foregoing convolution layer are caused to perform convolution on the past frame included in the image to calculate the past feature map.

3. The method according to claim 1 ,
wherein the neural network includes a convolutional layer,
wherein the convolutional layer is caused to perform convolution on the past frame included in the image to calculate the past feature map and store the past feature map in a memory, and
wherein, when the past feature map and the current feature map are combined with each other, the past feature map stored in the memory and the current feature map obtained by causing the convolutional layer to perform convolution on the current frame included in the image are combined with each other.

4. The method according to claim 1 ,
wherein the convolutional layer is a network model lighter than a certain network model.

5. The method according to claim 4 ,
wherein the lighter network model is a network model whose processing speed at which the computer performs the causing using the neural network is higher than 5 fps.

6. The method according to claim 4 ,
wherein the lighter network model is SqueezeNet including a plurality of fire modules, each of which includes a squeeze layer, which is a 1×1 filter, and an expand layer, in which a 1×1 filter and a 3×3 filter are arranged in parallel with each other.

7. A non-transitory computer-readable recording medium storing a program for causing a computer to perform operations comprising:
inputting an image including one or more objects to a neural network;
causing a convolutional layer included in the neural network to perform convolution on a current frame included in the image to calculate a current feature map, which is a feature map at a present time;
causing a combiner for combining two or more feature maps into one feature map to combine a past feature map, which is a feature map obtained by causing the convolutional layer to perform convolution on a past frame included in the image and preceding the current frame, and the current feature map;
causing a region proposal network included in the neural network to estimate an object candidate area using the combined past feature map and current feature map, the region proposal network being used to estimate the object candidate area;
causing a region of interest pooling layer included in the neural network to estimate positional information and identification information regarding the one or more objects included in the current frame using the combined past feature map and current feature map and the estimated object candidate area, the region of interest pooling layer being used to perform class estimation; and
outputting the positional information and the identification information regarding the one or more objects included in the current frame of the image estimated in the causing as object detection results.

8. An information processing apparatus achieved by a computer using a neural network, the information processing apparatus comprising:
an inputter that inputs an image including one or more objects to the neural network;
a processor that causes a convolutional layer included in the neural network to perform convolution on a current frame included in the image to calculate a current feature map, which is a feature map at a present time,
that causes a combiner for combining two or more feature maps into one feature map to combine a past feature map, which is a feature map obtained by causing the convolutional layer to perform convolution on a past frame included in the image and preceding the current frame, and the current feature map,
that causes a region proposal network included in the neural network to estimate an object candidate area using the combined past feature map and current feature map, the region proposal network being used to estimate the object candidate area, and
that causes a region of interest pooling layer included in the neural network to estimate positional information and identification information regarding the one or more objects included in the current frame using the combined past feature map and current feature map and the estimated object candidate area, the region of interest pooling layer being used to perform class estimation; and
an outputter that outputs the positional information and the identification information regarding the one or more objects included in the current frame of the image estimated by the processor as object detection results.

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Senay Gregory
• Tsukizawa Sotaro
• Kim Min Young
• Rigazio Luca

Classifications

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title