Image recognition method and apparatus, electronic device, and readable storage medium using an update on body extraction parameter and alignment parameter

Patent metadata

Field	Value
Publication number	US-11417095-B2
Application number	US-201916685526-A
Country	US
Kind code	B2
Filing date	Nov 15, 2019
Priority date	Dec 14, 2017
Publication date	Aug 16, 2022
Grant date	Aug 16, 2022

Abstract

An image recognition method is provided. The method includes obtaining predicted locations of joints of a target person in a to-be-recognized image based on a joint prediction model, where the joint prediction model is pre-constructed by: obtaining a plurality of sample images; inputting training features of the sample images and a body model feature to a neural network and obtaining predicted locations of joints in the sample images outputted by the neural network; updating a body extraction parameter and an alignment parameter; and inputting the training features of the sample images and the body model feature to the neural network to obtain the joint prediction model.

First claim

What is claimed is:

1. An image recognition method, applied to an electronic device, and comprising:
obtaining a to-be-recognized feature of a to-be-recognized image, the to-be-recognized image comprising a target person;
obtaining a preset body model feature of a body frame image, the preset body model feature comprising locations of joints in the body frame image;
inputting the to-be-recognized feature and the preset body model feature to a pre-constructed joint prediction model, the joint prediction model is pre-constructed by:
obtaining a plurality of positive sample images and a plurality of negative sample images respectively, to obtain a plurality of sample images, the positive sample image comprising a person, and the negative sample image comprising no person;
obtaining training features of the sample images respectively;
inputting the training features of the sample images and the preset body model feature to a neural network and obtaining predicted locations of joints in the sample images outputted by the neural network;
comparing the predicted locations of the joints with real locations of the joints respectively, to obtain comparison results;
updating a body extraction parameter and an alignment parameter based on the comparison results, the body extraction parameter being used to extract the person from a background environment in the sample images and the alignment parameter being used to represent respective correspondences between locations of joints in the preset body model feature and the predicted locations of the joints in the sample images;
in response to determining the plurality of positive sample images reflects a sitting posture, assigning a higher weight ratio to lower body joint location data than to upper body joint location data; and
inputting the training features of the sample images and the preset body model feature to the neural network to obtain the joint prediction model; and

obtaining predicted locations of joints of the target person in the to-be-recognized image based on the joint prediction model.

2. The image recognition method according to claim 1 , further comprising:
obtaining a body posture of the target person in the to-be-recognized image outputted by the joint prediction model, the body posture comprising the predicted locations of joints of the target person in the to-be-recognized image.

3. The image recognition method according to claim 1 , wherein updating the body extraction parameter and the alignment parameter comprises:
updating the body extraction parameter in response to determining any of the predicted locations of the joints is located in a background environment area of the sample images; and
updating the alignment parameter in response to determining any of the predicted locations of the joints is different from its corresponding real location.

4. The image recognition method according to claim 1 , wherein obtaining the to-be-recognized feature comprises:
obtaining multi-frame video images in a video and using a frame of the video image as the to-be-recognized image, to obtain the to-be-recognized feature of the to-be-recognized image.

5. The image recognition method according to claim 4 , further comprising:
obtaining tracking information of the target person based on the predicted locations of joints in the multi-frame video images.

6. The image recognition method according to claim 1 , wherein the to-be-recognized feature includes pixel values of color channels.

7. The image recognition method according to claim 1 , wherein a resolution of the to-be-recognized image is M1×N1, both M1 and N1 are positive integers, and a pixel value of a pixel in the i th row and the j th column in the to-be-recognized image is (R ij , G ij , B ij ), a matrix representation of the to-be-recognized feature of the to-be-recognized image is:

[

(

R

1
,
1

,

G

1
,
1

,

B

1
,
1

)

…

(

R

1
,

N
⁢

⁢
1

,

G

1
,

N
⁢

⁢
1

,

B

1
,

N
⁢

⁢
1

)

⋮

⋱

⋮

(

R

M
⁢

⁢
1

,

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Zhu Xiaolong
• Huang Kaining
• Luo Jingmin
• Mei Lijian
• Huang Shenghui
• Zheng Yongsen
• Wang Yitong
• Huang Haozhi

Classifications

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V20/40 Primary

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title