Method, electronic device, and computer program product for processing an image

Patent metadata

Field	Value
Publication number	US-11688143-B2
Application number	US-202117179988-A
Country	US
Kind code	B2
Filing date	Feb 19, 2021
Priority date	Jan 29, 2021
Publication date	Jun 27, 2023
Grant date	Jun 27, 2023

Abstract

Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for processing an image. The method includes acquiring an image about an augmented reality scene. The method further includes determining a target image part corresponding to a target object from the image. The method further includes using a machine learning model to augment information about the target object in the target image part to obtain an augmented target image part. The method further includes displaying the augmented target image part. Through the method, augmentation of an augmented reality image may be quickly achieved, the image quality is improved, the use of hardware resources is reduced, and user experience is improved.

First claim

What is claimed is:

1. A method for processing an image, comprising:
acquiring an image about an augmented reality scene;
determining a target image part corresponding to a target object from the image;
using a machine learning model to augment information about the target object in the target image part to obtain an augmented target image part; and
displaying the augmented target image part;
wherein the machine learning model is configured to generate a complete image of a given object based on a partial image of the given object;
wherein the target image part comprises an unobscured part of an obscured image of the target object, the obscured image of the target object further comprising an obscured part that is obscured by at least a portion of an image of an additional object different than the target object; and
wherein using a machine learning model to augment information about the target object in the target image part comprises:
extracting the unobscured part of the target image part of the image; and
inputting the unobscured part of the target image part to the machine learning model to obtain a complete unobscured image of the target object as the augmented target image part.

2. The method according to claim 1 , wherein determining the target image part comprises:
extracting an image part corresponding to the target object from the image;
zooming the image part to a predetermined display size of the target object;
determining a ratio of the number of feature points of the target object in the zoomed image part to the number of standard feature points of the target object; and
determining the zoomed image part as the target image part in response to the ratio being less than a threshold.

3. The method according to claim 2 , wherein zooming the image part to a predetermined display size of the target object comprises:
zooming in the image part to the predetermined display size of the target object if it is determined that a size of the image part is smaller than the predetermined display size of the target object; and
zooming out the image part to the predetermined display size of the target object if it is determined that the size of the image part is larger than the predetermined display size of the target object.

4. The method according to claim 1 , wherein using a machine learning model to augment information about the target object in the target image part comprises:
inputting the target image part to the machine learning model for increasing the resolution of the target image part, wherein the machine learning model is configured to increase the resolution of an image.

5. The method according to claim 1 , wherein the target image part is a 2-dimensional image;
wherein using a machine learning model to augment information about the target object in the target image part comprises:
inputting the target image part to the machine learning model to obtain a 3-dimensional image of the target object, wherein the machine learning model is configured to convert a 2-dimensional image into a 3-dimensional image.

6. The method according to claim 1 , further comprising:
acquiring an original image about a reality scene through a camera;
acquiring feature point information in the reality scene; and
adding information about a virtual scene to the original image based on the feature point information to obtain the image about the augmented reality scene.

7. The method according to claim 1 , wherein the machine learning model is a generative adversarial network model.

8. An electronic device, comprising:
at least one processor; and
a memory coupled to the at least one processor and having instructions stored thereon, wherein the instructions, when executed by the at least one processor, cause the electronic device to execute actions comprising:
acquiring an image about an augmented reality scene;
determining a target image part corresponding to a target object from the image;
using a machine learning model to augment information about the target object in the target image part to obtain an augmented target image part; and
displaying the augmented target image part;
wherein the machine learning model is configured to generate a complete image of a given object based on a partial image of the given object;
wherein the target image part comprises an unobscured part of an obscured image of the target object, the obscured image of the target object further comprising an obscured part that is obscured by at least a portion of an image of an additional object different than the target object; and
wherein using a machine learning model to augment information about the target object in the target image part comprises:
extracting the unobscured part of the target image part of the image; and
inputting the unobscured part of the target image part to the machine learning model to obtain a complete unobscured image of the target object as the augmented target image part.

9. The electronic device according to claim 8 , wherein determining the target image part comprises:
extracting an image part corresponding to the target object from the image;
zooming the image part to a predetermined display size of the target object;
determining a ratio of the number of feature points of the target object in the zoomed image part to the number of standard feature points of the target object; and
determining the zoomed image part as the target image part in response to the ratio being less than a threshold.

10. The electronic device according to claim 9 , wherein zooming the image part to a predetermined display size of the target object comprises:
zooming in the image part to the predetermined display size of the target object if it is determined that a size of the image part is smaller than the predetermined display size of the target object; and
zooming out the image part to the predetermined display size of the target object if it is determined that the size of the image part is larger than the predetermined display size of the target object.

11. The electronic device according to claim 8 , wherein using a machine learning model to augment information about the target object in the target image part comprises:
inputting the target image part to the machine learning model for increasing the resolution of the target image part, wherein the machine learning model is configured to increase the resolution of an image.

12. The electronic device according to claim 8 , wherein the target image part is a 2-dimensional image;
wherein using a machine learning model to augment information about the target object in the target image part comprises:
inputting the target image part to the machine learning model to obtain a 3-dimensional image of the target object, wherein the machine learning model is configured to convert a 2-dimensional image into a 3-dimensional image.

13. The electronic device according to claim 8 , wherein the actions further comprise:
acquiring an original image about a reality scene through a camera;
acquiring feature point information in the reality scene; and
adding information about a virtual scene to the original image based on the feature point information to obtain the image about the augmented reality scene.

14. The electronic device according to claim 8 , wherein the machine learning model is a generative adversarial network model.

15. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions, wherein the machine-executable instructions, when executed by a machine, cause the machine t

Assignees

• Emc Ip Holding Co Llc
• EMC IP Holdinq Company LLC

Inventors

• Sha Danqing
• Chen Qiang
• Jia Zhen
• Liu Jinpeng

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G06V10/40

  Extraction of image or video features · CPC title