Image classification method, neural network training method, and apparatus

Patent metadata

Field	Value
Publication number	US-12314343-B2
Application number	US-202117538640-A
Country	US
Kind code	B2
Filing date	Nov 30, 2021
Priority date	May 30, 2019
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

An image classification method, a neural network training method, and an apparatus are provided, and relate to the field of artificial intelligence, and specifically, to the field of computer vision. The image classification method includes: obtaining a to-be-processed image; and obtaining a classification result of the to-be-processed image based on a pre-trained neural network model, where the classification result includes a class or a superclass to which the to-be-processed image belongs. When the neural network model is trained, not only labels of a plurality of training images but also class hierarchy information of the plurality of training images is used. That is, more abundant information of the training images is used. Therefore, images can be better classified.

First claim

What is claimed is:

1. An image classification method, comprising:
obtaining a to-be-processed image; and
classifying the to-be-processed image based on a preset neural network model, to obtain a classification result of the to-be-processed image,
wherein the classification result comprises a class or a superclass to which the to-be-processed image belongs,
wherein the neural network model is obtained by performing training based on a plurality of training images, a label of each training image of the plurality of training images, and class hierarchy information of the plurality of training images,
wherein the class hierarchy information of the plurality of training images comprises at least one superclass to which each of a plurality of classes belongs, and the plurality of classes comprise one or more classes to which the plurality of training images belong, and
wherein the at least one superclass to which each of the plurality of classes belongs is pre-labeled.

2. The method according to claim 1 , wherein the classifying the to-be-processed image based on the preset neural network model comprises:
extracting a feature vector of the to-be-processed image;
determining, based on the feature vector of the to-be-processed image, confidence that the to-be-processed image belongs to each of a plurality of candidate classes; and
determining the classification result of the to-be-processed image from the plurality of candidate classes based on the confidence that the to-be-processed image belongs to each of the plurality of candidate classes.

3. The method according to claim 2 , wherein the determining the classification result of the to-be-processed image from the plurality of candidate classes based on the confidence that the to-be-processed image belongs to each of the plurality of candidate classes comprises:
determining a first candidate class in the plurality of candidate classes as the classification result of the to-be-processed image, wherein the first candidate class is a class with highest confidence in the plurality of candidate classes.

4. The method according to claim 3 , wherein the method further comprises:
determining a first candidate superclass in a plurality of candidate superclasses as the classification result of the to-be-processed image, wherein the confidence of the first candidate class is less than a first confidence threshold, and confidence of the first candidate superclass is greater than or equal to a second confidence threshold.

5. The method according to claim 1 , wherein the classifying the to-be-processed image based on the preset neural network model comprises:
obtaining reference images of the to-be-processed image, wherein the reference images comprise a plurality of image classes, and the to-be-processed image belongs to one of the plurality of image classes;
extracting a feature vector of the to-be-processed image and a feature vector of each image class in the plurality of image classes;
determining, based on a difference between the feature vector of the to-be-processed image and the feature vector of each image class in the plurality of image classes, confidence that the to-be-processed image belongs to each image class in the plurality of image classes; and
determining the classification result of the to-be-processed image from the plurality of image classes based on the confidence that the to-be-processed image belongs to each image class in the plurality of image classes.

6. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable medium stores program code to be executed by a device, and the program code is used to perform the method according to claim 1 .

7. A chip, wherein the chip comprises a processor and a data interface, and the processor reads, through the data interface, instructions stored in a memory, to perform the method according to claim 1 .

8. An image classification method, comprising:
obtaining a to-be-processed image; and
classifying the to-be-processed image based on a preset first neural network model, to obtain a classification result of the to-be-processed image,
wherein the classification result of the to-be-processed image comprises a class or a superclass to which the to-be-processed image belongs,
wherein the first neural network model is obtained by performing training based on a plurality of first feature vectors, labels of a plurality of first training images, and semantic description information of the plurality of first training images, wherein the semantic description information of each first training image of the plurality of first training images is a semantic description of an image feature of the first training image; and
wherein the plurality of first feature vectors are feature vectors obtained by performing feature extraction on the plurality of first training images by a second neural network model, wherein the second neural network model is obtained by performing training based on a plurality of second training images, a label of each of the plurality of second training images, and class hierarchy information of the plurality of second training images, and wherein the class hierarchy information of the plurality of second training images comprises one or more classes to which the plurality of second training images belong and at least one superclass to which each of the plurality of second training images belongs.

9. The method according to claim 8 , wherein the classifying the to-be-processed image based on the preset first neural network model comprises:
extracting a feature vector of the to-be-processed image based on the second neural network model;
processing the feature vector of the to-be-processed image based on the first neural network model, to obtain a semantic vector of the to-be-processed image; and
comparing the semantic vector of the to-be-processed image with a candidate semantic vector, to obtain the classification result of the to-be-processed image.

10. A neural network training method, comprising:
obtaining a plurality of training images;
extracting image features of the plurality of training images based on a feature extraction network of a neural network;
processing the image features of the plurality of training images based on a hierarchical prediction network of the neural network, to obtain classification results of the plurality of training images, wherein a classification result of each training image of the plurality of training images comprises a class and a superclass to which the training image belongs; and
determining a parameter of a neural network model based on the classification results of the plurality of training images and labeled classes of the plurality of training images.

11. An image classification apparatus, comprising:
a memory configured to store a program; and
a processor configured to execute the program stored in the memory, wherein when the program stored in the memory is executed by the processor, the processor is configured to:
obtain a to-be-processed image; and
classify the to-be-processed image based on a preset neural network model, to obtain a classification result of the to-be-processed image,
wherein the classification result comprises a class or a superclass to which the to-be-processed image belongs,
wherein the neural network model is obtained by performing training based on a plurality of training images, a label of each training image of the plurality of training images, and class hierarchy information of the plurality of training images,
wherein the class hierarchy information of the plurality of training images comprises at least one superclass to which each of a plurality

Assignees

• Huawei Tech Co Ltd
• Univ Beijing

Inventors

• Huang Weiran
• Li Aoxue
• Li Zhenguo
• Luo tiange
• Qian Li
• Wang Liwei

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06V10/95

  structured as a network, e.g. client-server architectures · CPC title