Deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition

Patent metadata

Field	Value
Publication number	US-10860837-B2
Application number	US-201615746237-A
Country	US
Kind code	B2
Filing date	Jul 20, 2016
Priority date	Jul 20, 2015
Publication date	Dec 8, 2020
Grant date	Dec 8, 2020

Abstract

Various image processing may benefit from the application deep convolutional neural networks. For example, a deep multi-task learning framework may assist face detection, for example when combined with landmark localization, pose estimation, and gender recognition. An apparatus can include a first module of at least three modules configured to generate class independent region proposals to provide a region. The apparatus can also include a second module of the at least three modules configured to classify the region as face or non-face using a multi-task analysis. The apparatus can further include a third module configured to perform post-processing on the classified region.

First claim

We claim:

1. An apparatus, comprising:
a first module of at least three modules, wherein the first module configured to generate class independent region proposals to provide a region;
a second module of the at least three modules is configured to classify the region as face or non-face using a multi-task analysis,
wherein the second module comprises a five convolutional layers with three fully connected layers, a network configured to fuse the three fully connected layers, separate networks for face detection, landmark detection, visibility determination, pose estimation, and gender determination; and
a third module of the at least three modules is configured to perform post-processing on the classified region.

2. The apparatus of claim 1 , wherein the third module comprises at least one of an iterative region proposal or landmark-based non-maximum suppression.

3. An apparatus, comprising:
at least one processor; and
at least one memory including computer program instructions,
wherein the at least one memory and the computer program instructions are configured to
select a set of data for facial analysis; and
apply the set of data to a network comprising at least three modules,
wherein a first module of the at least three modules is configured to generate class independent region proposals to provide a region,
wherein a second module of the at least three modules is configured to classify the region as face or non-face using a multi-task analysis,
wherein the second module comprises a five convolutional layers with three fully connected layers, a network configured to fuse the three fully connected layers, and separate networks for face detection, landmark detection, visibility determination, pose estimation, and gender determination; and
wherein a third module of the at least three modules is configured to perform post-processing on the classified region.

4. The apparatus of claim 3 , wherein the third module comprises at least one of an iterative region proposal or landmark-based non-maximum suppression.

5. A method, comprising:
selecting a set of data for facial analysis; and
applying the set of data to a network comprising at least three modules,
wherein a first module of the at least three modules is configured to generate class independent region proposals to provide a region,
wherein a second module of the at least three modules is configured to classify the region as face or non-face using a multi-task analysis, wherein the second module comprises a five convolutional layers with three fully connected layers, a network configured to fuse the three fully connected layers, and separate networks for face detection, landmark detection, visibility determination, pose estimation, and gender determination, and
wherein a third module of the at least three modules is configured to perform post-processing on the classified region.

6. The method of claim 5 , wherein the third module comprises at least one of an iterative region proposal or landmark-based non-maximum suppression.

7. An apparatus, comprising:
means for selecting a set of data for facial analysis; and
means for applying the set of data to a network comprising at least three modules,
wherein a first module of the at least three modules is configured to generate class independent region proposals to provide a region,
wherein a second module of the at least three modules is configured to classify the region as face or non-face using a multi-task analysis,
wherein the second module comprises a five convolutional layers with three fully connected layers, a network configured to fuse the three fully connected layers, and separate networks for face detection, landmark detection, visibility determination, pose estimation, and gender determination, and
wherein a third module of the at least three modules is configured to perform post-processing on the classified region.

8. The apparatus of claim 7 , wherein the third module comprises at least one of an iterative region proposal or landmark-based non-maximum suppression.

Assignees

• Univ Maryland

Inventors

• Ranjan Rajeev
• Patel Vishal M
• Chellappa Ramalingam
• Castillo Carlos D

Classifications

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V40/165 Primary

  using facial parts and geometric relationships · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V10/993

  Evaluation of the quality of the acquired pattern · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title