Method and apparatus for positioning key point, device, and storage medium

Patent metadata

Field	Value
Publication number	US-11610389-B2
Application number	US-202117201665-A
Country	US
Kind code	B2
Filing date	Mar 15, 2021
Priority date	Jun 12, 2020
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

A method and apparatus for positioning a key point, a device, and a storage medium are provided. The method may include: extracting a first feature map and a second feature map of a to-be-positioned image, the first feature map and the second feature map being different feature maps; determining, based on the first feature map, an initial position of a key point in the to-be-positioned image; determining, based on the second feature map, an offset of the key point; and adding the initial position of the key point with the offset of the key point to obtain a final position of the key point.

First claim

What is claimed is:

1. A method for positioning a key point, comprising:
extracting a first feature map and a second feature map of a to-be-positioned image, the first feature map and the second feature map being different feature maps;
determining, based on the first feature map, an initial position of a key point in the to-be-positioned image;
determining, based on the second feature map, an offset of the key point; and
adding the initial position of the key point with the offset of the key point to obtain a final position of the key point,
wherein the extracting the first feature map and the second feature map of the to-be-positioned image comprises:
inputting a to-be-positioned feature map into a main network to output an initial feature map of the to-be-positioned image; and
inputting the initial feature map into a first sub-network and a second sub-network respectively to output the first feature map and the second feature map, wherein the first sub-network and the second sub-network are two different branches of the main network.

2. The method according to claim 1 , wherein the determining, based on the first feature map, the initial position of the key point in the to-be-positioned image comprises:
generating, based on the first feature map, a heat map of the key point in the to-be-positioned image; and
determining, based on a heat value of a point on the heat map, the initial position of the key point.

3. The method according to claim 2 , wherein the generating, based on the first feature map, the heat map of the key point in the to-be-positioned image comprises:
performing 1×1 convolution on the first feature map to obtain the heat map, wherein channels of the heat map correspond to key points one to one.

4. The method according to claim 1 , wherein the determining, based on the second feature map, the offset of the key point comprises:
extracting, based on the initial position of the key point, a feature from a corresponding position of the second feature map; and
performing offset regression by using the feature to obtain the offset of the key point.

5. An electronic device, comprising:
one or more processors; and
a storage apparatus storing one or more programs thereon,
the one or more programs, when executed by the one or more processors, causing the one or more processors to perform operations comprising:
extracting a first feature map and a second feature map of a to-be-positioned image, the first feature map and the second feature map being different feature maps;
determining, based on the first feature map, an initial position of a key point in the to-be-positioned image;
determining, based on the second feature map, an offset of the key point; and
adding the initial position of the key point with the offset of the key point to obtain a final position of the key point,
wherein the extracting the first feature map and the second feature map of the to-be-positioned image comprises:
inputting a to-be-positioned feature map into a main network to output an initial feature map of the to-be-positioned image; and
inputting the initial feature map into a first sub-network and a second sub-network respectively to output the first feature map and the second feature map, wherein the first sub-network and the second sub-network are two different branches of the main network.

6. The electronic device according to claim 5 , wherein the determining, based on the first feature map, the initial position of the key point in the to-be-positioned image comprises:
generating, based on the first feature map, a heat map of the key point in the to-be-positioned image; and
determining, based on a heat value of a point on the heat map, the initial position of the key point.

7. The electronic device according to claim 6 , wherein the generating, based on the first feature map, the heat map of the key point in the to-be-positioned image comprises:
performing 1×1 convolution on the first feature map to obtain the heat map, wherein channels of the heat map correspond to key points one to one.

8. The electronic device according to claim 5 , wherein the determining, based on the second feature map, the offset of the key point comprises:
extracting, based on the initial position of the key point, a feature from a corresponding position of the second feature map; and
performing offset regression by using the feature to obtain the offset of the key point.

9. A non-transitory computer readable medium, storing a computer program thereon, the computer program, when executed by a processor, causing the processor to perform operations comprising:
extracting a first feature map and a second feature map of a to-be-positioned image, the first feature map and the second feature map being different feature maps;
determining, based on the first feature map, an initial position of a key point in the to-be-positioned image;
determining, based on the second feature map, an offset of the key point; and
adding the initial position of the key point with the offset of the key point to obtain a final position of the key point,
wherein the extracting the first feature map and the second feature map of the to-be-positioned image comprises:
inputting a to-be-positioned feature map into a main network to output an initial feature map of the to-be-positioned image; and
inputting the initial feature map into a first sub-network and a second sub-network respectively to output the first feature map and the second feature map, wherein the first sub-network and the second sub-network are two different branches of the main network.

10. The non-transitory computer readable medium according to claim 9 , wherein the determining, based on the first feature map, the initial position of the key point in the to-be-positioned image comprises:
generating, based on the first feature map, a heat map of the key point in the to-be-positioned image; and
determining, based on a heat value of a point on the heat map, the initial position of the key point.

11. The non-transitory computer readable medium according to claim 10 , wherein the generating, based on the first feature map, the heat map of the key point in the to-be-positioned image comprises:
performing 1×1 convolution on the first feature map to obtain the heat map, wherein channels of the heat map correspond to key points one to one.

12. The non-transitory computer readable medium according to claim 9 , wherein the determining, based on the second feature map, the offset of the key point comprises:
extracting, based on the initial position of the key point, a feature from a corresponding position of the second feature map; and
performing offset regression by using the feature to obtain the offset of the key point.

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Wang Jian
• Lu Zipeng
• Sun Hao
• Zhang Hongwu
• Wen Shilei
• Ding Errui

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/462 Primary

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T2207/30196

  Human being; Person · CPC title