Identification using depth-based head-detection data

Patent metadata

Field	Value
Publication number	US-9754154-B2
Application number	US-201414559757-A
Country	US
Kind code	B2
Filing date	Dec 3, 2014
Priority date	Feb 15, 2013
Publication date	Sep 5, 2017
Grant date	Sep 5, 2017

Abstract

A candidate human head is found in depth video using a head detector. A head region of light intensity video is spatially resolved with a three-dimensional location of the candidate human head in the depth video. Facial recognition is performed on the head region of the light intensity video using a face recognizer.

First claim

The invention claimed is:

1. On a computing system, a method comprising:
receiving a depth video;
receiving a light intensity video at least partially spatially-registered to the depth video;
finding a candidate human head in the depth video using a head detector;
spatially resolving a head region of the light intensity video with a three-dimensional location of the candidate human head in the depth video, the head region defining a limited portion of the light intensity video; and
performing facial recognition on only the head region of the light intensity video using a face recognizer.

2. The method of claim 1 , further comprising:
performing skeletal modeling on a body region of the depth video using a body tracker to produce a skeletal model, the body region being spatially contiguous with the three-dimensional location of the candidate human head, and the body region defining a limited portion of the depth video.

3. The method of claim 2 , wherein the skeletal modeling and the facial recognition are performed in parallel.

4. The method of claim 2 , further comprising:
responsive to the face recognizer producing a positive identification of a human face in the head region of the light intensity video, associating the skeletal model with the human face.

5. The method of claim 2 , wherein the body tracker is configured to constrain a head joint of the skeletal model to the three-dimensional location of the candidate human head as identified by the head detector.

6. The method of claim 1 , wherein the head detector is configured to classify depth pixels of the depth video by producing for each depth pixel a probability that the depth pixel corresponds to a human head without producing a probability that the depth pixel corresponds to another body part, and wherein the candidate human head includes a contiguous region of depth pixels each having a probability this is greater than a threshold.

7. The method of claim 1 , wherein the face recognizer is configured to repeatedly scan the head region inside a bounding rectangle, a size of the bounding rectangle changing each scan.

8. The method of claim 7 , wherein the bounding rectangle is scaled as a function of a depth of the candidate human head.

9. A computing system, comprising:
a logic machine; and
a storage machine holding instruction executable by the logic machine to:
receive a depth video;
receive an infrared video at least partially spatially-registered to the depth video;
find a candidate human head in the depth video using a previously-trained, machine-learning head detector;
spatially resolve a head region of the infrared video with a three-dimensional location of the candidate human head in the depth video, the head region defining a limited portion of the infrared video;
perform facial recognition on only the head region of the infrared video using a previously-trained, machine-learning face recognizer; and
perform skeletal modeling on a body region of the depth video using a previously-trained, machine-learning body tracker to produce a skeletal model, the body region being spatially contiguous with the three-dimensional location of the candidate human head, and the body region defining a limited portion of the depth video.

10. The computing system of claim 9 , wherein the skeletal modeling and the facial recognition are performed in parallel.

11. The computing system of claim 9 , wherein the storage machine further holds instructions executable by the logic machine to:
responsive to the previously-trained, machine-learning face recognizer producing a positive identification of a human face in the head region of the infrared video, associate the skeletal model with the human face.

12. The computing system of claim 9 , wherein the previously-trained, machine-learning body tracker is configured to constrain a head joint of the skeletal model to the three-dimensional location of the candidate human head as identified by the previously-trained, machine-learning head detector.

13. The computing system of claim 9 , wherein the previously-trained, machine-learning head detector is configured to classify depth pixels of the depth video by producing for each depth pixel a probability that the depth pixel corresponds to a human head without producing a probability that the depth pixel corresponds to another body part, and wherein the candidate human head includes a contiguous region of depth pixels each having a probability this is greater than a threshold.

14. The computing system of claim 9 , wherein the previously-trained, machine-learning face recognizer is configured to repeatedly scan the head region inside a bounding rectangle, a size of the bounding rectangle changing each scan.

15. The computing system of claim 14 , wherein the bounding rectangle is scaled as a function of a depth of the candidate human head.

16. A computing system, comprising:
a logic machine; and
a storage machine holding instruction executable by the logic machine to:
receive a depth video;
receive an infrared video at least partially spatially-registered to the depth video;
find a candidate human head in the depth video using a previously-trained, machine-learning head detector;
spatially resolve a head region of the infrared video with a three-dimensional location of the candidate human head in the depth video, the head region defining a limited portion of the infrared video;
perform facial recognition on only the head region of the infrared video using a previously-trained, machine-learning face recognizer;
perform skeletal modeling on a body region of the depth video using a previously-trained, machine-learning body tracker to produce a skeletal model, the previously-trained, machine-learning body tracker being configured to constrain a head joint of the skeletal model to the three-dimensional location of the candidate human head as identified by the previously-trained, machine-learning head detector, the body region being spatially contiguous with the three-dimensional location of the candidate human head, and the body region defining a limited portion of the depth video.

17. The computing system of claim 16 , wherein the storage machine further holds instructions executable by the logic machine to:
responsive to the previously-trained, machine-learning face recognizer producing a positive identification of a human face in the head region of the infrared video, associate the skeletal model with the human face.

18. The computing system of claim 16 , wherein the previously-trained, machine-learning head detector is configured to classify depth pixels of the depth video by producing for each depth pixel a probability that the depth pixel corresponds to a human head without producing a probability that the depth pixel corresponds to another body part, and wherein the candidate human head includes a contiguous region of depth pixels each having a probability this is greater than a threshold.

19. The computing system of claim 16 , wherein the previously-trained, machine-learning face recognizer is configured to repeatedly scan the head region inside a bounding rectangle, a size of the bounding rectangle changing each scan, and wherein the bounding rectangle is scaled as a function of a depth of the candidate human head.

20. The computing system of claim 16 , wherein finding the candidate human head in the depth video is performed prior to performing facial recognition on the infrared video, and wherein the skeletal modeling and the facial recognition are per

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Craig Robert M
• Tankovich Vladimir
• Peeper Craig
• Dalal Ketan
• Dedhia Bhaven
• Meekhof Casey

Classifications

• G06F21/32 Primary

  using biometric data, e.g. fingerprints, iris scans or voiceprints · CPC title

• G06T2210/12

  Bounding box · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T2207/30201

  Face · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title