Image processing method and apparatus, storage medium, and electronic device

Patent metadata

Field	Value
Publication number	US-12182971-B2
Application number	US-202117524387-A
Country	US
Kind code	B2
Filing date	Nov 11, 2021
Priority date	Nov 26, 2019
Publication date	Dec 31, 2024
Grant date	Dec 31, 2024

Abstract

A plurality of groups of portrait components and a region in which the plurality of groups of portrait components are located are recognized from a first image, each group of portrait components corresponding to one human body, and a target region that includes a human face is determined in the region in which the plurality of groups of portrait components are located, to blur regions other than the target region in the first target image. A target group of portrait components including the human face is recognized from the first image, so that the target region in which the target group of portrait components is located is determined as a foreground region, and limbs of other people without a human face are determined as a background region. The disclosed system and method improve the accuracy of recognizing a foreground person and reducing incorrect detection in portrait recognition.

First claim

What is claimed is:

1. An image processing method comprising:
recognizing, from a first image, a plurality of groups of portrait components and a region in which the plurality of groups of portrait components are located, each group of portrait components corresponding to one human body;
determining, by processing circuitry of an electronic device, in the recognized region in which the plurality of groups of portrait components are located, which of the plurality of groups of portrait components includes a respective human face; determining a target region in the recognized region that is separated from other regions in the recognized region based on which of the plurality of groups of portrait components is determined to include the respective human face, the target region including each of the plurality of groups of portrait components that includes the respective human face; and
blurring regions of the first image other than the target region, to obtain a second image.

2. The method according to claim 1 , wherein the determining the target region comprises:
determining M groups of portrait components comprising a human face from N groups of portrait components, the N groups of portrait components being the plurality of groups of portrait components, N≥M≥1; and
determining, in a region in which the M groups of portrait components are located, the target region in which the respective human face of each of the plurality of groups of portrait components including the respective human face is located is larger than or equal to a first threshold, or the target region in which each of the plurality of groups of portrait components including the respective human face is located being larger than or equal to a second threshold.

3. The method according to claim 2 , wherein the first threshold and the second threshold are positively correlated to a size of the first image.

4. The method according to claim 2 , wherein the determining, in the region in which the M groups of portrait components are located, comprises:
setting pixel values of pixels corresponding to the M groups of portrait components to a first pixel value, and setting pixel values of pixels in the first image other than the pixels corresponding to the M groups of portrait components to a second pixel value, to obtain a binary image, the first pixel value being different from the second pixel value; and
performing a region recognition on the binary image, to obtain the target region, the target region comprising pixels of the respective human face of each of the plurality of groups of portrait components determined to include the respective human face.

5. The method according to claim 1 , wherein the recognizing comprises:
processing the first image by using a recognition model, and determining the plurality of groups of portrait components and the recognized region in which the plurality of groups of portrait components are located.

6. The method according to claim 1 , wherein, before the recognizing, the method further comprises:
acquiring a first group of training images, a second group of training images, a group of region division results, and a group of training recognition results, the first group of training images corresponding one to one to the group of region division results, each region division result representing a known portrait region in an image in the first group of training images, the second group of training images corresponding one to one to the group of training recognition results, each training recognition result representing a known portrait component in an image in the second group of training images; and
training an initial recognition model based on the first group of training images and the second group of training images, to obtain a trained recognition model, an error between an estimated portrait region recognized from the first group of training images by using the trained recognition model and the known portrait region in the group of region division results satisfying a first convergence condition, an error between an estimated portrait component recognized from the second group of training images by using the trained recognition model and the known portrait component in the group of training recognition results satisfying a second convergence condition,
the trained recognition model comprising:
an encoding network configured to encode an image to obtain encoded data,
a portrait region recognition network configured to recognize a portrait region according to the encoded data, and
a portrait component recognition network configured to recognize a portrait component according to the encoded data.

7. The method according to claim 6 , wherein the training comprises:
selecting a first training image from the first group of training images, and selecting a second training image from the second group of training images;
inputting the first training image and the second training image into the initial recognition model, the initial recognition model comprising an initial encoding network, an initial portrait region recognition network, and an initial portrait component recognition network, the initial encoding network comprising cascaded first convolutional layers, the initial portrait region recognition network comprising cascaded second convolutional layers, the initial portrait component recognition network comprising cascaded third convolutional layers; and
receiving, by a first convolutional layer in the initial encoding network, encoded data obtained after a cascaded previous first convolutional layer encodes the first training image and the second training image, and transmitting the encoded data to a corresponding second convolutional layer, third convolutional layer, and cascaded next first convolutional layer;
receiving, by the initial portrait region recognition network, encoded data transmitted by a corresponding first convolutional layer and cascaded previous second convolutional layer, and performing a portrait region recognition on the received encoded data; and
receiving, by the initial portrait component recognition network, encoded data transmitted by a corresponding first convolutional layer and cascaded previous third convolutional layer, and performing a portrait component recognition on the received encoded data.

8. The method according to claim 1 , wherein, in response to a determination that the first image is a video frame image in a video, after the blurring the regions other than the target region in the first image, the method further comprises:
replacing the first image in the video with the second image; and
playing the second image in a process of playing the video.

9. An image processing apparatus, comprising:
processing circuitry configured to
recognize, from a first image, a plurality of groups of portrait components and a region in which the plurality of groups of portrait components are located, each group of portrait components corresponding to one human body;
determine, in the recognized region in which the plurality of groups of portrait components are located, which of the plurality of groups of portrait components includes a respective human face;
determine a target region in the recognized region that is separated from other regions in the recognized region based on which of the plurality of groups of portrait components is determined to include the respective human face, the target region including each of the plurality of groups of portrait components that includes the respective human face; and
blur regions of the first image other than the target region, to obtain a second image.

10. The apparatus according to claim 9 , wherein the processing circuitry perfo

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Yu Xiaoming
• Yi Yang
• Li Feng
• Cai Kaijie

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title