Method and apparatus for estimating pose

Patent metadata

Field	Value
Publication number	US-9697609-B2
Application number	US-201414289915-A
Country	US
Kind code	B2
Filing date	May 29, 2014
Priority date	Jun 3, 2013
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

A method and apparatus for estimating a pose that estimates a pose of a user using a depth image is provided, the method including, recognizing a pose of a user from a depth image, and tracking the pose of the user using a user model exclusively of one another to enhance precision of estimating the pose.

First claim

What is claimed is:

1. A method for pose estimating, the method comprising:
recognizing, by way of a processor, a pose of a user from a depth image by classifying a body part of the user from the depth image by respectively determining probabilities for visible body parts and probabilities for hidden body parts using a classification tree;
estimating the pose of the user based on a result of the recognition of the pose;
recognizing a next pose of the user from a subsequent depth image when a confidence of the result of the recognizing of the pose satisfies a predetermined condition;
tracking the pose of the user, using a user model when the confidence of the result of the recognizing of the pose fails to satisfy the predetermined condition; and
estimating the pose of the user based on a result of the tracking,
wherein the user model corresponding to the recognition of the pose is selected from a plurality of candidate kinematic models,
wherein the tracking includes initializing the user model using body parts derived from result of recognition of previous depth image.

2. The method of claim 1 , further comprising:
continuing the tracking of the pose of the user with respect to a second subsequent depth image using the user model when the confidence of the result of the tracking of the pose satisfies a second predetermined condition, while otherwise re-performing the recognizing of the pose of the user from the second subsequent depth image.

3. The method of claim 1 , further comprising:
generating the user model corresponding to a user, based on a user area displayed in the depth image and the result of the recognition of the pose.

4. The method of claim 3 , wherein the generating of the user model comprises:
applying joint information derived from the result of the recognition of the pose to a body part included in the user model; and
generating the user model through connecting adjacent body parts to which the joint information is applied.

5. The method of claim 3 , wherein the generating of the user model comprises:
randomly sampling a user area displayed in the depth image; and
merging body parts of the user derived from the result of the recognition of the pose into predetermined body parts.

6. The method of claim 1 , further comprising:
selecting a user model corresponding to the user from among a plurality of predetermined user models, based on a similarity between a user area displayed in the depth image and the plurality of predetermined user models.

7. The method of claim 6 , wherein the selecting of the user model corresponding to the user further comprises:
extracting candidate user models from among the plurality of predetermined user models based on user information identified through the user area; and
selecting, to be the user model corresponding to the user, a candidate user model having a highest similarity between the extracted candidate user models and the user area.

8. The method of claim 1 , wherein the tracking of the pose of the user comprises:
searching for a corresponding relationship between a point included in body parts of the user model and a point included in a depth image;
determining a rotation matrix and a translation matrix, based on the corresponding relationship; and
updating the user model, using the rotation matrix and the translation matrix.

9. The method of claim 8 , wherein the updating of the user model comprises:
updating the user model by applying a result of updating of a body part of an upper level from among the body parts included in the user model to a body part of a lower level.

10. The method of claim 1 , wherein the confidence of the result of the tracking is determined based on a similarity between a user area displayed in the depth image and the result of the tracking and an average distance of points corresponding between a user model and a depth image.

11. The method of claim 1 , wherein the tracking of the pose of the user comprises:
adjusting a position of a user model, based on a difference between a center coordinate of the user model and a center coordinate of the user area displayed in the depth image.

12. The method of claim 1 :
wherein the initializing the user model includes applying the result of the recognition of the pose to the user model.

13. The method of claim 12 , wherein the initializing of the user model comprises:
updating a position of a body part, based on a corresponding relationship between a joint inserted into a body part in the user model and a joint derived from the result of the recognition of the pose.

14. The method of claim 1 , wherein each of plural nodes of the classification tree include separate information on a visible body part and information on a hidden body part, such that the probabilities for visible body parts are determined using information on a visible body part of a node of the classification tree and the probabilities for hidden body parts are determined using information on a hidden body part of the node of the classification tree.

15. The method of claim 1 , wherein, in the recognizing of the pose of the user, a classification of a body part includes combined respective considerations of visible probabilities and hidden probabilities from plural classification trees.

16. The method of claim 1 ,
wherein the user model is a determined user kinematic model to track body parts, including body parts that are occluded in the depth image, selected from plural different body type candidate user kinematic models, and
wherein tracking of the pose includes initializing the determined user kinematic model by updating the determined user kinematic model using merged different body parts recognized in at least one previous recognizing of a pose of the user from a previous depth image using the classification tree.

17. The method of claim 16 , wherein the initializing of the determined kinematic model further includes updating centers of body parts of the determined user kinematic model based on to determined centers of body parts derived from the at least one previous recognizing of the pose of the user from the previous depth image.

18. A hybrid pose recognition and tracking method comprising:
recognizing, by way of a processor, a pose of a user from a depth by classifying a body part of the user from the depth image by respectively determining probabilities for visible body parts and probabilities for hidden body parts using a classification tree;
estimating the pose of the user, based on a result of the recognizing of the pose; and
tracking the pose of the user in a subsequent depth image using a user model when a confidence of a result of the recognizing of the pose of the user fails to satisfy a predetermined condition,
wherein the user model corresponding to the recognition of the pose is selected from a plurality of candidate kinematic models,
wherein the tracking includes initializing the user model using body parts derived from result of recognition of previous depth image.

19. The hybrid pose recognition and tracking method of claim 18 , wherein the user model selected from among the plurality of candidate kinematic models is selected based on a similarity between a user area displayed in the depth image and the selected user model.

20. A hybrid pose recognition and tracking method comprising:
recognizing, by way of a processor, a pose of a user from a depth image;
estimating the pose of the user, based on a result of the recognizing; and
tracking the pose of the us

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Won Jun
• Yoo Byung In
• Choi Chang Kyu

Classifications

• G06K9/00369

  Physics · mapped topic

• G06T7/0046 Primary

  Physics · mapped topic

• G06T7/75 Primary

  involving models · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title