Apparatus and method for neck and shoulder landmark detection

Patent metadata

Field	Value
Publication number	US-9569661-B2
Application number	US-201514719079-A
Country	US
Kind code	B2
Filing date	May 21, 2015
Priority date	May 21, 2015
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

A device is configured to perform a method for neck and shoulder detection. The method includes receiving an image that includes a face. The method also includes performing a neck localization operation on the image. The neck localization operation is performed using results from a pre-trained regression model. The method further includes performing a shoulder localization operation on the image. The method still further includes estimating a plurality of shoulder and neck keypoints using results of the neck localization operation and the shoulder localization operation.

First claim

What is claimed is:

1. A method for neck and shoulder detection, the method comprising:
receiving, by one or more microprocessors, an image that includes a face;
performing, by the one or more microprocessors, a neck localization operation on the image, the neck localization operation performed using results from a pre-trained regression model;
performing, by the one or more microprocessors, a shoulder localization operation on the image; and
estimating, by the one or more microprocessors, a plurality of shoulder and neck keypoints using results of the neck localization operation and the shoulder localization operation.

2. The method of claim 1 , wherein the neck localization operation comprises:
detecting, by the one or more microprocessors, at least one face in the image;
detecting, by the one or more microprocessors, a plurality of facial landmarks in the detected at least one face; and
determining, by the one or more microprocessors, a candidate neck position based on the detected plurality of facial landmarks and at least one predetermined relationship between facial landmark positions and a corresponding neck position, the predetermined relationship associated with the pre-trained regression model.

3. The method of claim 2 , wherein the neck localization operation further comprises:
performing, by the one or more microprocessors, skin color based image segmentation and skin color analysis on the candidate neck position.

4. The method of claim 2 , wherein the shoulder localization operation comprises:
detecting, by the one or more microprocessors, one or more shoulder contour edges using the detected at least one face and locations of the detected plurality of facial landmarks;
determining, by the one or more microprocessors, a contour of the shoulders by performing one or more shoulder contour segmentations; and
fitting, by the one or more microprocessors, a quadratic curve along the shoulder contour using the one or more shoulder contour edges.

5. The method of claim 4 , wherein the one or more shoulder contour segmentations are performed using a watershed algorithm.

6. The method of claim 1 , wherein the pre-trained regression model is developed over time using a plurality of images having manually labelled neck regions and a support vector regression technique.

7. An apparatus for neck and shoulder detection, the apparatus comprising:
at least one memory that stores instructions; and
at least one microprocessor coupled to the at least one memory, the at least one microprocessor configured by the instructions to perform operations comprising:
receiving an image that includes a face;
performing a neck localization operation on the image, the neck localization operation performed using results from a pre-trained regression model;
performing a shoulder localization operation on the image; and
estimating a plurality of shoulder and neck keypoints using results of the neck localization operation and the shoulder localization operation.

8. The apparatus of claim 7 , wherein the neck localization operation comprises:
detecting at least one face in the image;
detecting a plurality of facial landmarks in the at least one detected face; and
determining a candidate neck position based on the detected plurality of facial landmarks and at least one predetermined relationship between facial landmark positions and a corresponding neck position, the predetermined relationship associated with the pre-trained regression model.

9. The apparatus of claim 8 , wherein the neck localization operation further comprises:
performing skin color based image segmentation and skin color analysis on the candidate neck position.

10. The apparatus of claim 8 , wherein the shoulder localization operation comprises:
detecting one or more shoulder contour edges using the detected at least one face and locations of the detected plurality of facial landmarks;
determining a contour of the shoulders by performing one or more shoulder contour segmentations; and
fitting a quadratic curve along the shoulder contour using the one or more shoulder contour edges.

11. The apparatus of claim 10 , wherein the performing of the one or more shoulder contour segmentations comprises using a watershed algorithm.

12. The apparatus of claim 7 , wherein the pre-trained regression model is developed over time using a plurality of images having manually labelled neck regions and a support vector regression technique.

13. A non-transitory computer readable medium that stores instructions which, when executed by one or more microprocessors, cause the one or more microprocessors to perform operations comprising:
receiving an image that includes a face;
performing a neck localization operation on the image, the neck localization operation performed using results from a pre-trained regression model;
performing a shoulder localization operation on the image; and
estimating a plurality of shoulder and neck keypoints using results of the neck localization operation and the shoulder localization operation.

14. The non-transitory computer readable medium of claim 13 , wherein the neck localization operation comprises:
detecting at least one face in the image;
detecting a plurality of facial landmarks in the at least one detected face; and
determining a candidate neck position based on the detected plurality of facial landmarks and at least one predetermined relationship between facial landmark positions and a corresponding neck position, the predetermined relationship associated with the pre-trained regression model.

15. The non-transitory computer readable medium of claim 14 , wherein the neck localization operation further comprises:
performing skin color based image segmentation and skin color analysis on the candidate neck position.

16. The non-transitory computer readable medium of claim 14 , wherein the shoulder localization operation comprises:
detecting one or more shoulder contour edges using the detected at least one face and locations of the detected plurality of facial landmarks;
determining a contour of the shoulders by performing one or more shoulder contour segmentations; and
fitting a quadratic curve along the shoulder contour using the one or more shoulder contour edges.

17. The non-transitory computer readable medium of claim 16 , wherein the one or more shoulder contour segmentations are performed using a watershed algorithm.

18. The non-transitory computer readable medium of claim 13 , wherein the pre-trained regression model is developed over time using a plurality of images having manually labelled neck regions and a support vector regression technique.

Assignees

• Futurewei Technologies Inc

Inventors

• Xu Jingsong
• Cui Ying
• Wu Qiang
• Zhang Jian
• Zhang Chen-Xiong
• Liu Haibo
• Fang Kui

Classifications

• G06V10/40

  Extraction of image or video features · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06T7/75 Primary

  involving models · CPC title

• G06T2207/30201

  Face · CPC title

• G06T2207/20081

  Training; Learning · CPC title