Method and system for modeling subjects from a depth map

Patent metadata

Field	Value
Publication number	US-9330470-B2
Application number	US-201314135388-A
Country	US
Kind code	B2
Filing date	Dec 19, 2013
Priority date	Jun 16, 2010
Publication date	May 3, 2016
Grant date	May 3, 2016

Abstract

A method for modeling and tracking a subject using image depth data includes locating the subject's trunk in the image depth data and creating a three-dimensional (3D) model of the subject's trunk. Further, the method includes locating the subject's head in the image depth data and creating a 3D model of the subject's head. The 3D models of the subject's head and trunk can be exploited by removing pixels from the image depth data corresponding to the trunk and the head of the subject, and the remaining image depth data can then be used to locate and track an extremity of the subject.

First claim

What is claimed is:

1. A system comprising:
a first site having a first sensor to acquire image depth data; and
a processor communicatively coupled to:
a background manager to separate a background of an image from a foreground of the image in the image depth data to create a model of the image background;
a subject manager to determine from the background image a subset of the image depth data that corresponds to a subject and to send image depth data that does not correspond to the subject, wherein the background manager dynamically updates the image background based on second image depth data received from the first sensor and the image depth data that does not correspond to the subject

a subject tracking engine to create a three-dimensional (3D) model of a torso and a head of the subject based on the model of the image background and the subset of the image depth data corresponding to the subject and locate an extremity of the subject by using the 3D model of the torso and the head of the subject and the subset of the image depth data, wherein locating the extremity comprises generating an approximate position of the extremity upon a determination that data corresponding to the extremity is not included in the subset of the image depth data corresponding to the subject.

2. The system of claim 1 further comprising a memory to store the 3D model.

3. The system of claim 2 further comprising a second site, coupled to the processor, having a second sensor to acquire second image depth data.

4. The system of claim 3 wherein the background manager creates a second model of an image background from the second image depth data, the subject manager determines a subset of the second image depth data that corresponds to a second subject, and the subject tracking engine creates a 3D model of a second torso and a head of the second subject based on the model of the second image background and the subset of the second image depth data and locates an extremity of the second subject by using the 3D model of the torso and the head of the second subject and the second subset of the image depth data.

5. The system of claim 1 , wherein the subject manager uses remaining image depth data to locate the extremity of the subject by detecting a blob from the remaining image depth data that corresponds to an arm, determining whether the blob corresponds to a right arm or a left arm; and calculating where a hand and an elbow are located based on the blob.

6. The system of claim 5 wherein the processor further recognizes a gesture performed by the subject.

7. The system of claim 6 further comprising a gesture database, coupled to the processor, wherein recognizing the gesture comprises the processor storing a plurality of locations of the subject and comparing the plurality of locations of the subject to gestures in a gesture database.

8. The system of claim 6 wherein the first site further comprises a display, to display the gesture to a user.

9. The system of claim 1 wherein the subject is a human.

10. A computer generated method comprising:
receiving image depth data from an image sensor;
separating a background of an image from a foreground of the image in the image depth data;
creating a model of the image background from the image depth data;
determining from the background image a subset of the image depth data that corresponds to a subject;
dynamically updating the image background model using second image depth data received from the image sensor and the image depth data that does not correspond to the subject;
creating a three dimensional (3D) model of a torso and a head of the subject based on the updated model of the image background and the subset of the image data corresponding to the subject; and
locating an extremity of the subject using the 3D model of the torso and the head of the subject of the image depth data, including generating an approximate position of the extremity upon a determination that data corresponding to the extremity is not included in the subset of the image depth data corresponding to the subject.

11. The method of claim 10 wherein the image depth data is received via a network.

12. The method of claim 10 further comprising:
acquiring depth data for a plurality of sequential images; and
tracking a torso location of the subject in the sequential images.

13. The method of claim 12 further comprising determining a pelvis location of the subject based on the torso location from the two-dimensional torso tracking engine.

14. The method of claim 13 further comprising creating a 3D model of the torso of the subject in the sequential images based on the image depth data and the torso location.

15. The method of claim 10 wherein the 3D model of the torso of the subject is a parametric cylinder model.

16. At least one non-transitory computer readable medium having instructions, which when executed causes a processor to perform:
receiving image depth data from an image sensor;
separating a background of an image from a foreground of the image in the image depth data;
creating a model of the image background from the image depth data;
determining from the background image a subset of the image depth data that corresponds to a subject;
dynamically updating the image background model using second image depth data received from the image sensor and the image depth data that does not correspond to the subject;
creating a three dimensional (3D) model of a torso and a head of the subject based on the updated model of the image background and the subset of the image data corresponding to the subject; and
locating an extremity of the subject using the 3D model of the torso and the head of the subject of the image depth data, including generating an approximate position of the extremity upon a determination that data corresponding to the extremity is not included in the subset of the image depth data corresponding to the subject.

17. The computer readable medium of claim 16 wherein the image depth data is received via a network.

18. The computer readable medium of claim 17 having further instructions, which when executed causes the processor to perform determining a pelvis location of the subject based on the torso location from the two-dimensional torso tracking engine.

19. The computer readable medium of claim 18 having further instructions, which when executed causes the processor to perform creating a tracking the 3D model of the torso of the subject in the sequential images based on the image depth data and the torso location.

20. The computer readable medium of claim 16 having further instructions, which when executed causes the processor to perform:
acquiring depth data for a plurality of sequential images, and the subject tracking engine; and
tracking a torso location of the subject in the sequential images.

Assignees

• Kutliroff Gershom
• Bleiweiss Amit
• Glazer Itamar
• Madmoni Maoz
• Intel Corp

Inventors

• Kutliroff Gershom
• Bleiweiss Amit
• Glazer Itamar
• Madmoni Maoz

Classifications

• G06V40/10 Primary

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/2046

  Physics · mapped topic

• G06T7/0051 Primary

  Physics · mapped topic