Apparatus and method for neck and shoulder landmark detection

Patent metadata

Field	Value
Publication number	US-2016342831-A1
Application number	US-201514719079-A
Country	US
Kind code	A1
Filing date	May 21, 2015
Priority date	May 21, 2015
Publication date	Nov 24, 2016
Grant date	—

Abstract

A device is configured to perform a method for neck and shoulder detection. The method includes receiving an image that includes a face. The method also includes performing a neck localization operation on the image. The neck localization operation is performed using results from a pre-trained regression model. The method further includes performing a shoulder localization operation on the image. The method still further includes estimating a plurality of shoulder and neck keypoints using results of the neck localization operation and the shoulder localization operation.

First claim

1 . A method for neck and shoulder detection, the method comprising:
receiving, by one or more microprocessors, an image that includes a face; performing, by the one or more microprocessors, a neck localization operation on the image, the neck localization operation performed using results from a pre-trained regression model; performing, by the one or more microprocessors, a shoulder localization operation on the image; and estimating, by the one or more microprocessors, a plurality of shoulder and neck keypoints using results of the neck localization operation and the shoulder localization operation.

2 . The method of claim 1 , wherein the neck localization operation comprises:
detecting, by the one or more microprocessors, at least one face in the image; detecting, by the one or more microprocessors, a plurality of facial landmarks in the detected at least one face; and determining, by the one or more microprocessors, a candidate neck position based on the detected plurality of facial landmarks and at least one predetermined relationship between facial landmark positions and a corresponding neck position, the predetermined relationship associated with the pre-trained regression model.

3 . The method of claim 2 , wherein the neck localization operation further comprises:
performing, by the one or more microprocessors, skin color based image segmentation and skin color analysis on the candidate neck position.

4 . The method of claim 2 , wherein the shoulder localization operation comprises:
detecting, by the one or more microprocessors, one or more shoulder contour edges using the detected at least one face and locations of the detected plurality of facial landmarks; determining, by the one or more microprocessors, a contour of the shoulders by performing one or more shoulder contour segmentations; and fitting, by the one or more microprocessors, a quadratic curve along the shoulder contour using the one or more shoulder contour edges.

5 . The method of claim 4 , wherein the one or more shoulder contour segmentations are performed using a watershed algorithm.

6 . The method of claim 1 , wherein the pre-trained regression model is developed over time using a plurality of images having manually labelled neck regions and a support vector regression technique.

7 . An apparatus for neck and shoulder detection, the apparatus comprising:
at least one memory that stores instructions; and at least one microprocessor coupled to the at least one memory, the at least one microprocessor configured by the instructions to perform operations comprising:
receiving an image that includes a face;
performing a neck localization operation on the image, the neck localization operation performed using results from a pre-trained regression model;
performing a shoulder localization operation on the image; and
estimating a plurality of shoulder and neck keypoints using results of the neck localization operation and the shoulder localization operation.

8 . The apparatus of claim 7 , wherein the neck localization operation comprises:
detecting at least one face in the image; detecting a plurality of facial landmarks in the at least one detected face; and determining a candidate neck position based on the detected plurality of facial landmarks and at least one predetermined relationship between facial landmark positions and a corresponding neck position, the predetermined relationship associated with the pre-trained regression model.

9 . The apparatus of claim 8 , wherein the neck localization operation further comprises:
performing skin color based image segmentation and skin color analysis on the candidate neck position.

10 . The apparatus of claim 8 , wherein the shoulder localization operation comprises:
detecting one or more shoulder contour edges using the detected at least one face and locations of the detected plurality of facial landmarks; determining a contour of the shoulders by performing one or more shoulder contour segmentations; and fitting a quadratic curve along the shoulder contour using the one or more shoulder contour edges.

11 . The apparatus of claim 10 , wherein the performing of the one or more shoulder contour segmentations comprises using a watershed algorithm.

12 . The apparatus of claim 7 , wherein the pre-trained regression model is developed over time using a plurality of images having manually labelled neck regions and a support vector regression technique.

13 . A non-transitory computer readable medium that stores instructions which, when executed by one or more microprocessors, cause the one or more microprocessors to perform operations comprising:
receiving an image that includes a face; performing a neck localization operation on the image, the neck localization operation performed using results from a pre-trained regression model; performing a shoulder localization operation on the image; and estimating a plurality of shoulder and neck keypoints using results of the neck localization operation and the shoulder localization operation.

14 . The non-transitory computer readable medium of claim 13 , wherein the neck localization operation comprises:
detecting at least one face in the image; detecting a plurality of facial landmarks in the at least one detected face; and determining a candidate neck position based on the detected plurality of facial landmarks and at least one predetermined relationship between facial landmark positions and a corresponding neck position, the predetermined relationship associated with the pre-trained regression model.

15 . The non-transitory computer readable medium of claim 14 , wherein the neck localization operation further comprises:
performing skin color based image segmentation and skin color analysis on the candidate neck position.

16 . The non-transitory computer readable medium of claim 14 , wherein the shoulder localization operation comprises:
detecting one or more shoulder contour edges using the detected at least one face and locations of the detected plurality of facial landmarks; determining a contour of the shoulders by performing one or more shoulder contour segmentations; and fitting a quadratic curve along the shoulder contour using the one or more shoulder contour edges.

17 . The non-transitory computer readable medium of claim 16 , wherein the one or more shoulder contour segmentations are performed using a watershed algorithm.

18 . The non-transitory computer readable medium of claim 13 , wherein the pre-trained regression model is developed over time using a plurality of images having manually labelled neck regions and a support vector regression technique.

Assignees

• Futurewei Technologies Inc

Inventors

• Xu Jingsong
• Cui Ying
• Wu Qiang
• Zhang Jian
• Zhang Chen-Xiong
• Liu Haibo
• Fang Kui

Classifications

• G06V10/40

  Extraction of image or video features · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06T7/75 Primary

  involving models · CPC title

• G06T7/12

  Edge-based segmentation · CPC title

• G06T2207/30201

  Face · CPC title