Methods and systems for determining and tracking extremities of a target

Patent metadata

Field	Value
Publication number	US-9659377-B2
Application number	US-201414570005-A
Country	US
Kind code	B2
Filing date	Dec 15, 2014
Priority date	Oct 7, 2009
Publication date	May 23, 2017
Grant date	May 23, 2017

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may also be removed to isolate one or more voxels associated with a foreground object such as a human target. A location or position of one or more extremities of the isolated human target may then be determined.

First claim

What is claimed:

1. A method for determining an orientation of one or more extremities comprising:
receiving a depth image;
generating a grid of voxels based on the depth image;
removing a background included in the grid of voxels to isolate one or more voxels associated with a human target;
defining a head-to-center vector based on locations or positions of a head and a center of the human target;
creating an extremity volume box based on a displacement along the head-to-center vector; and
determining the orientation of the one or more extremities based on a line fit of depth values within the extremity volume box.

2. The method of claim 1 , wherein the one or more extremities comprises at least one of the following: a head, a centroid, a shoulder, a hip, a leg, an arm, a hand, an elbow, a knee, and a foot.

3. The method of claim 1 , further comprising estimating a center of the human target, wherein estimating the center of the human target comprises calculating an average position of the voxels in the grid associated with the human target.

4. The method of claim 1 , further comprising determining a core volume of the human target.

5. The method of claim 1 , further comprising:
creating a torso volume;
identifying voxels outside of the torso volume; and
labeling voxels outside the torso volume as being associated with the one or more extremities.

6. The method of claim 1 , further comprising processing the one or more extremities.

7. The method of claim 1 , further comprising:
determining a location or position of one or more extremities of the human target; and
refining the location or position of the one or more extremities based on pixels associated with the one or more extremities in the depth image.

8. A system for determining an orientation of one or more extremities comprising:
a capture device, wherein the capture device comprises a camera component to receive a depth image; and
a computing device operable to communicate with the capture device, wherein the computing device comprises a processor operable to perform operations comprising:
generating a grid of voxels based on the depth image;
removing a background included in the grid of voxels to isolate one or more voxels associated with a human target;
defining a head-to-center vector based on locations or positions of a head and a center of the human target;
creating an extremity volume box based on a displacement along the head-to-center vector; and
determining the orientation of the one or more extremities based on a line fit of depth values within the extremity volume box.

9. The system of claim 8 , wherein the one or more extremities comprises at least one of the following: a head, a centroid, a shoulder, a hip, a leg, an arm, a hand, an elbow, a knee, and a foot.

10. The system of claim 8 , wherein the operations further comprise estimating a center of the human target, wherein estimating the center of the human target comprises calculating an average position of the voxels in the grid associated with the human target.

11. The system of claim 8 , wherein the operations further comprise determining a core volume of the human target.

12. The system of claim 8 , wherein the operations further comprise:
creating a torso volume;
identifying voxels outside of the torso volume; and
labeling voxels outside the torso volume as being associated with the one or more extremities.

13. The system of claim 8 , wherein the operations further comprise processing the one or more extremities.

14. The system of claim 8 , wherein the operations further comprise:
determining a location or position of one or more extremities of the human target; and
refining the location or position of the one or more extremities based on pixels associated with the one or more extremities in the depth image.

15. A computer-readable storage device having stored thereon computer-executable instructions, the computer-executable instructions comprising instructions for:
receiving a depth image;
generating a grid of voxels based on the depth image;
removing a background included in the grid of voxels to isolate one or more voxels associated with a human target;
defining a head-to-center vector based on locations or positions of a head and a center of the human target;
creating an extremity volume box based on a displacement along the head-to-center vector; and
determining an orientation of the one or more extremities based on a line fit of depth values within the extremity volume box.

16. The computer-readable storage device of claim 15 , wherein the one or more extremities comprises at least one of the following: a head, a centroid, a shoulder, a hip, a leg, an arm, a hand, an elbow, a knee, and a foot.

17. The computer-readable storage device of claim 15 , wherein the computer-executable instructions further comprise instructions for estimating a center of the human target, wherein estimating the center of the human target comprises calculating an average position of the voxels in the grid associated with the human target.

18. The computer-readable storage device of claim 15 , wherein the computer-executable instructions further comprise instructions for determining a core volume of the human target.

19. The computer-readable storage device of claim 15 , wherein the computer-executable instructions further comprise instructions for:
creating a torso volume;
identifying voxels outside of the torso volume; and
labeling voxels outside the torso volume as being associated with the one or more extremities.

20. The computer-readable storage device of claim 15 , wherein the computer-executable instructions further comprise instructions for processing the one or more extremities.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Lee Johnny Chung
• Leyvand Tommer
• Stachniak Szymon Piotr
• Peeper Craig
• Liu Shao

Classifications

• G06T2219/2004

  Aligning objects, relative positioning of parts · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title