Network optimization method and apparatus, image processing method and apparatus, and storage medium

Patent metadata

Field	Value
Publication number	US-11416703-B2
Application number	US-202017037654-A
Country	US
Kind code	B2
Filing date	Sep 29, 2020
Priority date	Jan 15, 2019
Publication date	Aug 16, 2022
Grant date	Aug 16, 2022

Abstract

The present disclosure relates to a network optimization method and apparatus, an image processing method and apparatus, and a storage medium. The network optimization method includes: obtaining an image sample group; obtaining a first feature and a second feature of an image in the image sample group, and obtaining a first classification result by using the first feature of the image; performing feature exchange processing on an image pair in the image sample group to obtain a new image pair; obtaining a first loss value of the first classification result, a second loss value of the new image pair, and a third loss value of first features and second features of the new image pair in a preset manner; and adjusting parameters of a neural network at least according to the first loss value, the second loss value, and the third loss value until a preset requirement is met.

First claim

The invention claimed is:

1. A network optimization method for optimizing a neural network, comprising:
obtaining an image sample group, wherein the image sample group comprises an image pair formed by images of a same object and an image pair formed by images of different objects;
obtaining a first feature and a second feature of an image in the image sample group, and obtaining a first classification result by using the first feature of the image, wherein the first feature comprises an identity feature, and the second feature comprises an attribute feature;
performing feature exchange processing on an image pair in the image sample group to obtain a new image pair, wherein the feature exchange processing is to generate a new first image by using a first feature of a first image and a second feature of a second image in the image pair, and to generate a new second image by using a second feature of the first image and a first feature of the second image;
obtaining a first loss value of the first classification result, a second loss value of the new image pair, and a third loss value of first features and second features of the new image pair in a preset manner; and
adjusting parameters of the neural network at least according to the first loss value, the second loss value, and the third loss value until a preset requirement is met.

2. The method according to claim 1 , wherein obtaining the first feature and the second feature of the image in the image sample group comprises:
inputting two images in the image pair to an identity coding network module and an attribute coding network module of the neural network; and
obtaining first features of the two images in the image pair by using the identity coding network module, and obtaining second features of the two images in the image pair by using the attribute coding network module.

3. The method according to claim 2 , wherein obtaining the first loss value of the first classification result, the second loss value of the new image pair, and the third loss value of the first features and the second features of the new image pair in the preset manner comprises:
obtaining the first classification result of the first features obtained by means of the identity coding network module; and
obtaining the first loss value in a first preset manner according to the first classification result and a real classification result corresponding to the image in the image sample group.

4. The method according to claim 2 , wherein before inputting the two images in the image pair to the identity coding network module, the method further comprises:
adding noise to image areas of objects in the two images in the image pair.

5. The method according to claim 1 , wherein performing feature exchange processing on the image pair in the image sample group to obtain the new image pair comprises:
inputting a first feature and a second feature of an image in the image pair in the image sample group to a generation network module of the neural network; and
performing the feature exchange processing on the image pair in the image sample group by means of the generation network module to obtain the new image pair.

6. The method according to claim 1 ,
wherein if an input image pair comprises images of a same object,
performing feature exchange processing on the image pair in the image sample group to obtain the new image pair comprises: performing the feature exchange processing on the images in the image pair once to obtain the new image pair, and
performing the feature exchange processing on the images in the image pair once to obtain the new image pair comprises: generating a new first image by using the first feature of the first image and the second feature of the second image in the image pair, and generating a new second image by using the second feature of the first image and the first feature of the second image, and/or

wherein if the input image pair are images of different objects,
performing feature exchange processing on the image pair in the image sample group to obtain the new image pair comprises: performing the feature exchange processing on the images in the image pair twice to obtain the new image pair, and
performing the feature exchange processing on the images in the image pair twice to obtain the new image pair comprises:
generating a first intermediate image by using the first feature of the first image and the second feature of the second image in the image pair, and generating a second intermediate image by using the second feature of the first image and the first feature of the second image; and
generating a new first image by using a first feature of the first intermediate image and a second feature of the second intermediate image, and generating a new second image by using a second feature of the first intermediate image and a first feature of the second intermediate image.

7. The method according to claim 5 , wherein obtaining the first loss value of the first classification result, the second loss value of the new image pair, and the third loss value of the first features and the second features of the new image pair in the preset manner comprises:
obtaining, in a second preset manner, the second loss value of the new image pair obtained by means of the network generation module relative to an original image pair, wherein the original image pair corresponds to the new image pair.

8. The method according to claim 1 , wherein obtaining the first loss value of the first classification result, the second loss value of the new image pair, and the third loss value of the first features and the second features of the new image pair in the preset manner comprises:
obtaining the third loss value of the first features and the second features of the new image pair in a third preset manner based on the first features and the second features of the new image pair as well as first features and second features of the original image pair, wherein the original image pair corresponds to the new image pair.

9. The method according to claim 1 , wherein after performing feature exchange processing on the image pair in the image sample group to obtain the new image pair, the method further comprises:
inputting the generated new image pair to a discrimination network module of the neural network to obtain a label feature representing reality of the new image pair; and
obtaining a fourth loss value of the new image pair in a fourth preset manner based on the label feature.

10. The method according to claim 9 , wherein adjusting the parameters of the neural network at least according to the first loss value, the second loss value, and the third loss value until the preset requirement is met comprises:
obtaining a loss value of the neural network by using the first loss value, the second loss value, the third loss value, and the fourth loss value; and
adjusting the parameters of the neural network by using the loss value of the neural network until the preset requirement is met.

11. The method according to claim 10 , wherein obtaining the loss value of the neural network by using the first loss value, the second loss value, the third loss value, and the fourth loss value comprises:
if the image sample group input to the neural network is the image pair of the same object, obtaining a first network loss value of the neural network in a fifth preset manner based on the first loss value, the second loss value, the third loss value, and the fourth loss value;
if the image sample group input to the neural network is the image pair of different objects, obtaining a second network loss value of the neural network in a sixth preset manner based on the f

Assignees

• Beijing Sensetime Tech Development Co Ltd

Inventors

• Ge Yixiao
• Shen Yantao
• Chen Dapeng
• Wang Xiaogang
• Li Hongsheng

Classifications

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title