Image processing method and apparatus using neural network

Patent metadata

Field	Value
Publication number	US-12014529-B2
Application number	US-202117153018-A
Country	US
Kind code	B2
Filing date	Jan 20, 2021
Priority date	Jan 21, 2020
Publication date	Jun 18, 2024
Grant date	Jun 18, 2024

Abstract

An image processing method and apparatus using a neural network are provided. The image processing method includes generating a plurality of augmented features by augmenting an input feature, and generating a prediction result based on the plurality of augmented features.

First claim

What is claimed is:

1. An image processing method comprising:
extracting an input feature from an input image;
generating augmented features by augmenting the input feature; and
generating a prediction result based on the augmented features,
wherein the generating of the augmented features is performed using an inference operation of a trained machine learning model.

2. The image processing method of claim 1 , wherein the generating of the augmented features comprises generating a first augmented feature based on executing a neural network model, of the machine learning model, based on the input feature and a first transformation code.

3. The image processing method of claim 2 , wherein the neural network model comprises an encoding model and a decoding model, and
the generating of the first augmented feature comprises:
encoding the input feature to a latent feature using the encoding model;
combining the latent feature and the first transformation code to determine a combined feature; and
decoding the combined feature to the first augmented feature using the decoding model.

4. The image processing method of claim 2 , wherein the generating of the augmented features comprises:
generating a second augmented feature based on another executing of the neural network model based on the input feature and a second transformation code.

5. The image processing method of claim 4 , wherein the first transformation code and the second transformation code correspond to different transformations.

6. The image processing method of claim 4 , wherein the generating of the prediction result comprises generating the prediction result based on a fusion of a first partial prediction result according to the first augmented feature and a second partial prediction result according to the second augmented feature.

7. The image processing method of claim 1 , wherein the generating of the prediction result comprises:
generating a plurality of partial prediction results based on the plurality of augmented features; and
generating the prediction result by fusing the plurality of partial prediction results.

8. The image processing method of claim 1 , wherein the generating of the augmented features comprises augmenting the input feature based on transformation parameters corresponding to different transformations.

9. The image processing method of claim 8 , wherein the transformations comprise any one or any combination of two or more of scaling, cropping, flipping, padding, rotation, translation, color transformation, brightness transformation, contrast transformation, and noise addition.

10. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the image processing method of claim 1 .

11. An apparatus comprising:
a processor configured to:
extract an input feature from an input image;
generate augmented features by augmenting the input feature; and
generate a prediction result based on the augmented features,

wherein the generation of the augmented features is performed using an inference operation of a trained machine learning model.

12. The apparatus of claim 11 , wherein the processor is further configured to generate a first augmented feature based on an execution of a neural network model, of the machine learning model, based on the input feature and a first transformation code.

13. The apparatus of claim 12 , wherein the processor is further configured to generate a second augmented feature based on another execution of the neural network model based on the input feature and a second transformation code.

14. The apparatus of claim 13 , wherein the processor is further configured to generate the prediction result based on a fusion of a first partial prediction result according to the first augmented feature and a second partial prediction result according to the second augmented feature.

15. The apparatus of claim 11 , wherein the processor is further configured to perform multiple augments of the input feature based on transformation parameters corresponding to different transformations, and
wherein, for the generating of the prediction result, the processor is configured to generate the prediction result dependent on results of the multiple augments.

16. The apparatus of claim 15 , wherein the transformation parameters correspond to at least one of scaling, cropping, flipping, padding, rotation, translation, color transformation, brightness transformation, contrast transformation, or noise addition, or correspond to the scaling, the cropping, the flipping, the padding, the rotation, the translation, the color transformation, the brightness transformation, the contrast transformation, and the noise addition.

17. The apparatus of claim 11 , wherein, for the generating of the augmented features, the processor is configured to:
encode the input feature using an encoding model, as a first machine learning model; and
decode a generated feature, dependent on the encoded input feature and a first transformation code, into a first augmented feature using a decoding model, as a second machine learning model.

18. The apparatus of claim 17 ,
wherein the processor is further configured to generate a second augmented feature using the decoding model based on the encoded feature and a second transformation code corresponding to a second transformation information that is different from a first transformation information which corresponds to the first transformation code,
wherein the first transformation information is first information with respect to at least one of scaling, cropping, flipping, padding, rotation, translation, color transformation, brightness transformation, contrast transformation, or noise addition, and the second transformation information is second information with respect to at least one of a corresponding scaling, a corresponding cropping, a corresponding flipping, a corresponding padding, a corresponding rotation, a corresponding translation, a corresponding color transformation, a corresponding brightness transformation, a corresponding contrast transformation, or a corresponding noise addition, and
wherein, for the generating of the prediction result, the processor is configured to generate the prediction result dependent on the first augmented feature and the second augmented feature.

19. An electronic apparatus comprising:
a camera configured to generate the input image; and
the apparatus of claim 17 .

20. An image processing apparatus comprising:
a processor configured to:
extract an input feature from an input image;
generate augmented features by augmenting the input feature; and

generate a prediction result based on the augmented features,
wherein the processor is further configured to generate a first augmented feature based on an execution of a neural network model based on the input feature and a first transformation code, and
wherein the neural network model comprises an encoding model and a decoding model, and the processor is further configured to:
encode the input feature to a latent feature using the encoding model;
combine the latent feature and the first transformation code to determine a combined feature; and
decode the combined feature to the first augmented feature using the decoding model.

21. An electronic apparatus comprising:
a camera configured to generate an input image; and
a processor configured to:
extract an input feature from the input image

Assignees

• Samsung Electronics Co Ltd

Inventors

• Guo Tianchu
• Zhang Hui
• Liu Xiabing
• Yoo Byungin
• Choi Changkyu
• Han Jaejoon
• Kwak Youngjun
• Liu Yongchao

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06T3/10

  Selection of transformation methods according to the characteristics of the input images · CPC title

• G06V10/82

  using neural networks · CPC title