Detection, tracking, and pose estimation of an articulated body

Patent metadata

Field	Value
Publication number	US-2016335486-A1
Application number	US-201514749303-A
Country	US
Kind code	A1
Filing date	Jun 24, 2015
Priority date	May 13, 2015
Publication date	Nov 17, 2016
Grant date	—

Abstract

Techniques related to pose estimation for an articulated body are discussed. Such techniques may include extracting, segmenting, classifying, and labeling blobs, generating initial kinematic parameters that provide spatial relationships of elements of a kinematic model representing an articulated body, and refining the kinematic parameters to provide a pose estimation for the articulated body.

First claim

What is claimed is:

1 . A method for generating a pose estimation for an articulated body comprising:
classifying a segmented blob as a hand blob; generating finger labels for the hand blob; generating, based on the labeled hand blob, initial kinematic model parameters that provide spatial relationships of elements of a kinematic model representing an articulated body; and applying a kinematic model refinement to the initial kinematic model parameters based on matching the kinematic model to target positions of the hand blob to generate resultant kinematic model parameters.

2 . The method of claim 1 , further comprising:
extracting the segmented blob from input depth image data, wherein extracting the segmented blob comprises an over-segmentation of the input depth image data to generate super-pixels and merging or splitting the super-pixels to generated the segmented blob.

3 . The method of claim 1 , wherein classifying the segmented blob comprises applying a random forest classifier to a plurality of invariant features associated with the segmented blob.

4 . The method of claim 1 , further comprising:
matching the hand blob to a previous frame hand blob, wherein the hand blob is associated with a current frame and the previous frame hand blob is associated with a previous frame, and wherein generating the finger labels for the hand blob comprises copying finger labels from a previous hand model associated with the previous frame hand blob.

5 . The method of claim 1 , further comprising:
applying a rigid transformation to a second kinematic hand model, wherein the hand blob is associated with a current frame and the second kinematic hand model is associated with a previous frame, and wherein the second kinematic hand model comprises second finger labels; generating, based on the transformed second kinematic hand model, second initial kinematic model parameters; and applying a second kinematic model refinement to the second initial kinematic model parameters.

6 . The method of claim 1 , further comprising:
generating, based on the labeled hand blob, second initial kinematic model parameters; applying a second kinematic model refinement to the second initial kinematic model parameters to generate second resultant kinematic model parameters; and outputting at least one of the first or second resultant kinematic model parameters based on a scoring.

7 . The method of claim 6 , wherein the scoring comprises a comparison of a first error and a second error, wherein the first error is based on comparing the kinematic model implementing the resultant kinematic model parameters with the hand blob and the second error is based on comparing the kinematic model implementing the second resultant kinematic model parameters with the hand blob.

8 . The method of claim 1 , wherein generating the finger labels comprises:
applying edge detection to the hand blob; locating bases and tips of a plurality of portions of the hand blob; generating one or more invariant feature vectors associated with the portions of the hand blob; and classifying the hand blob as representing a left hand or a right hand based on the one or more invariant feature vectors.

9 . The method of claim 8 , further comprising:
providing one or more finger labels to each of the portions of the hand blob; and generating, based on the one or more finger labels, a plurality of sets of initial kinematic model parameters based on a plurality of finger label permutations associated with the hand blob.

10 . The method of claim 1 , wherein generating the finger labels comprises:
applying edge detection to the hand blob; locating a base and a tip of a first portion of the hand blob; generating an invariant feature vector associated with the first portion of the hand blob; and providing a finger label to the first portion of the hand blob based on applying a classifier to the invariant feature vector.

11 . The method of claim 1 , wherein the kinematic model refinement comprises at least one of a particle swarm optimization technique, a Levenberg Marquardt technique based on a numerical Jacobian, a partial Levenberg Marquardt technique, or an inverse kinematics based iterative closest point technique.

12 . The method of claim 1 , wherein applying the kinematic model refinement comprises:
selecting, based on the hand blob, the target positions for matching the kinematic model; generating a plurality of virtual end-effectors corresponding to the target positions based on the plurality of target positions and the kinematic model; generating an inverse kinematics problem comprising a Jacobian matrix based on the initial kinematic model parameters, the target positions, and the virtual end-effectors; determining a change in the kinematic model parameters based on the inverse kinematics problem; and repeating the selecting the plurality of target positions, generating the plurality of virtual end-effectors, generating the inverse kinematics problem, and determining the change in the kinematic model parameters until a convergence is attained to generate the resultant kinematic model parameters.

13 . The method of claim 12 , wherein the Jacobian matrix comprises at least one of an element having a target weighting parameter associated with a first target position of the plurality of target positions, an element having a joint weighting parameter associated with a first joint of the elements of the kinematic model, or an element having a repulsive target functionality associated with a first target position of the plurality of target positions.

14 . The method of claim 12 , wherein the inverse kinematics model comprises at least one first kinematic model parameter comprising a feasibility set such that the first kinematic model parameter must be within the feasibility set.

15 . A system for generating a pose estimation for an articulated body comprising:
a memory to store image data; and a central processor coupled to the memory, the central processor to classify a segmented blob as a hand blob, generate finger labels for the hand blob, generate, based on the labeled hand blob, initial kinematic model parameters that provide spatial relationships of elements of a kinematic model representing an articulated body, and apply a kinematic model refinement to the initial kinematic model parameters based on matching the kinematic model to target positions of the hand blob to generate resultant kinematic model parameters.

16 . The system of claim 15 , wherein the central processor is further to apply a rigid transformation to a second kinematic hand model, wherein the hand blob is associated with a current frame and the second kinematic hand model is associated with a previous frame, and wherein the second kinematic hand model comprises second finger labels, generate, based on the transformed second kinematic hand model, second initial kinematic model parameters, and apply a second kinematic model refinement to the second initial kinematic model parameters.

17 . The system of claim 15 , wherein the central processor is further to generate, based on the labeled hand blob, second initial kinematic model parameters, apply a second kinematic model refinement to the second initial kinematic model parameters to generate second resultant kinematic model parameters, and output at least one of the first or second resultant kinematic model parameters based on a scoring.

18 . The system of claim 15 , wherein the kinematic model refinement comprises at least one of a partic

Assignees

• Intel Corp

Inventors

• Fleishman Shahar
• Kliger Mark
• Lerner Alon

Classifications

• G06T7/143

  involving probabilistic approaches, e.g. Markov random field [MRF] modelling · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G06T2219/2016

  Rotation, translation, scaling · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/75

  involving models · CPC title