Image classification method, electronic device, and storage medium

Patent metadata

Field	Value
Publication number	US-10325181-B2
Application number	US-201715703027-A
Country	US
Kind code	B2
Filing date	Sep 13, 2017
Priority date	Dec 11, 2015
Publication date	Jun 18, 2019
Grant date	Jun 18, 2019

Abstract

An image classification method is provided. The method includes: inputting a to-be-classified image into a plurality of neural network models; obtaining data output by multiple non-input layers specified by each neural network model to generate a plurality of image features corresponding to the plurality of neural network models; respectively inputting the plurality of corresponding image features into linear classifiers, each of the linear classifiers being trained by one of the plurality of neural network models for determining whether an image belongs to a preset class; obtaining, using each neural network model, a corresponding probability that the to-be-classified image comprises an object image of the preset class; and determining, according to each obtained probability, whether the to-be-classified image includes the object image of the preset class.

First claim

What is claimed is:

1. An image classification method, comprising:
inputting a to-be-classified image into a plurality of neural network models;
obtaining data output by multiple non-input layers specified by each neural network model to generate a plurality of image features corresponding to the plurality of neural network models;
respectively inputting the plurality of corresponding image features into linear classifiers, each of the linear classifiers being trained by one of the plurality of neural network models for determining whether an image belongs to a preset class;
obtaining, using each neural network model, a corresponding probability that the to-be-classified image comprises an object image of the preset class; and
determining, according to each obtained probability, whether the to-be-classified image comprises the object image of the preset class.

2. The method according to claim 1 , wherein generating the plurality of corresponding image features further comprises:
obtaining vectors outputted by the multiple non-input layers specified among one or more intermediate layer and an output layer of each neural network model; and
combining vectors of the multiple non-input layers of each neural network model to obtain the plurality of image features corresponding to the plurality of the neural network model.

3. The method according to claim 2 , wherein:
inputting the to-be-classified image into the plurality of neural network model comprises:

respectively inputting the to-be-classified image in multiple scales into each neural network model; and
combining the vectors of the multiple non-input layers further comprises:
respectively combining the vectors of the multiple non-input layers of one of the neural network models corresponding to the to-be-classified image at each scale to obtain combined vectors corresponding to the multiple scales; and
averaging the combined vectors corresponding to the multiple scales, to obtain one of the plurality of image features corresponding to one of the plurality of the neural network models.

4. The method according to claim 1 , further comprising:
clearing up a coefficient of an output layer of a first neural network model trained by using a first training set, adjusting the output layer to adapt to a second training set, and performing retraining, by using the second training set, to obtain a retrained neural network model.

5. The method according to claim 4 , wherein obtaining the corresponding probability that the to-be-classified image comprises the object image of the preset class further comprises:
traversing the to-be-classified image by using a window to extract window images and scaling the window images to a same size;
inputting each window image into the retrained neural network model and obtaining data output by the non-input layers to generate a window image feature;
separately inputting each window image feature into a linear classifier corresponding to the retrained neural network model for determining the preset class; and
obtaining, according to a result output by the corresponding linear classifier, a probability that each window image comprises the object image of the preset class.

6. The method according to claim 5 , wherein determining, according to each obtained probability, whether the to-be-classified image comprises the object image of the preset class further comprises:
selecting a first probability with a maximum value from probabilities corresponding to the window images;
selecting a second probability with a maximum value from the first probability and a probability corresponding to the first neural network model;
calculating a weighted average of the second probability and the probability corresponding to the retrained neural network model; and
determining, according to a relationship between the weighted average and a probability threshold corresponding to the preset class, whether the to-be-classified image comprises the object image of the preset class.

7. An electronic device, comprising a memory and a processor, the memory storing instructions, which, when being executed by the processor, cause the processor to perform the following steps:
inputting a to-be-classified image into a plurality of neural network models;
obtaining data output by multiple non-input layers specified by each neural network model to generate a plurality of image features corresponding to the plurality of neural network models;
respectively inputting the plurality of corresponding image features into linear classifiers, each of the linear classifiers being trained by one of the plurality of neural network models for determining whether an image belongs to a preset class;
obtaining, using each neural network model, a corresponding probability that the to-be-classified image comprises an object image of the preset class; and
determining, according to each obtained probability, whether the to-be-classified image comprises the object image of the preset class.

8. The electronic device according to claim 7 , wherein generating the plurality of corresponding image features further comprises:
obtaining vectors outputted by the multiple non-input layers specified among one or more intermediate layer and an output layer of each neural network model; and
combining vectors of the multiple non-input layers of each neural network model to obtain the plurality of image features corresponding to the plurality of the neural network model.

9. The electronic device according to claim 8 , wherein:
inputting the to-be-classified image into the plurality of neural network model comprises:

respectively inputting the to-be-classified image in multiple scales into each neural network model; and
combining the vectors of the multiple non-input layers further comprises:
respectively combining the vectors of the multiple non-input layers of one of the neural network models corresponding to the to-be-classified image at each scale to obtain combined vectors corresponding to the multiple scales; and
averaging the combined vectors corresponding to the multiple scales, to obtain one of the plurality of image features corresponding to one of the plurality of the neural network models.

10. The electronic device according to claim 7 , wherein when being executed by the processor, the instructions further cause the processor to perform the following step:
clearing up a coefficient of an output layer of a first neural network model trained by using a first training set, adjusting the output layer to adapt to a second training set, and performing retraining, by using the second training set, to obtain a retrained neural network model.

11. The electronic device according to claim 10 , wherein obtaining the corresponding probability that the to-be-classified image comprises the object image of the preset class further comprises:
traversing the to-be-classified image by using a window to extract window images and scaling the window images to a same size;
inputting each window image into the retrained neural network model and obtaining data output by the non-input layers to generate a window image feature;
separately inputting each window image feature into a linear classifier corresponding to the retrained neural network model for determining the preset class; and
obtaining, according to a result output by the corresponding linear classifier, a probability that each window image comprises the object image of the preset class.

12. The electronic device according to claim 11 , wherein determining, according to each obtained probability, whether the to-be-classified image comprises the object image of

Assignees

• Tencent Tech Shenzhen Co Ltd
• Univ Tsinghua

Inventors

• Xu Kun
• Guo Xiaowei
• Huang Feiyue
• Zhang Ruixin
• Wang Juhong
• Hu Shimin
• Liu Bin

Classifications

• G06V30/1916

  Validation; Performance evaluation · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/809

  of classification results, e.g. where the classifiers operate on the same input data · CPC title

• G06F18/24

  Classification techniques · CPC title