Estimator training method and pose estimating method using depth image

Patent metadata

Field	Value
Publication number	US-9449392-B2
Application number	US-201414280990-A
Country	US
Kind code	B2
Filing date	May 19, 2014
Priority date	Jun 5, 2013
Publication date	Sep 20, 2016
Grant date	Sep 20, 2016

Abstract

An estimator training method and a pose estimating method using a depth image are disclosed, in which the estimator training method may train an estimator configured to estimate a pose of an object, based on an association between synthetic data and real data, and the pose estimating method may estimate the pose of the object using the trained estimator.

First claim

What is claimed is:

1. A method of estimating a pose, the method comprising:
obtaining, using at least one processor, a plurality of vector sets corresponding to a plurality of patches included in an input image using an estimator trained based on an association between synthetic data in which an object is synthesized and real data in which the object is photographed; and
estimating, using the at least one processor, a pose of an input object included in the input image based on the plurality of vector sets by generating a plurality of mixture models corresponding to a plurality of joints included in the input object based on the plurality of vector sets.

2. The method of claim 1 , wherein each of the plurality of vector sets comprises a plurality of vectors respectively indicating the plurality of joints included in the input object.

3. The method of claim 1 , wherein the estimating comprises:
generating a plurality of 2-part Gaussian mixture models (GMMs) corresponding to a plurality of joints included in the input object based on the plurality of vector sets; and
calculating three-dimensional (3D) coordinates of the plurality of joints included in the input object based on the plurality of 2-part GMMs.

4. The method of claim 3 , wherein the calculating comprises:
comparing an average value of a first Gaussian component to an average value of a second Gaussian component included in each of the plurality of 2-part GMMs;
detecting a 2-part GMM having a difference between the average value of the first Gaussian component and the average value of the second Gaussian component less than a threshold value, among the plurality of 2-part GMMs; and
calculating 3D coordinates of a joint corresponding to the detected 2-part GMM based on an average value of a Gaussian component having a greater weight between a first Gaussian component and a second Gaussian component included in the detected 2-part GMM.

5. The method of claim 3 , wherein the calculating comprises:
comparing an average value of a first Gaussian component to an average value of a second Gaussian component included in each of the plurality of 2-part GMMs;
detecting first GMMs, each having a difference between the average value of the first Gaussian component and the average value of the second Gaussian component less than a threshold value, among the plurality of 2-part GMMs;
detecting a second GMM having the difference between the average value of the first Gaussian component and the average value of the second Gaussian component greater than or equal to the threshold value;
detecting a Gaussian component closest to the first GMMs among N Gaussian components included in an N-part GMM corresponding to a view of the second GMM, N being an integer greater than “2”;
selecting one of a first Gaussian component and a second Gaussian component included in the second GMM based on the closest Gaussian component; and
calculating 3D coordinates of a joint corresponding to the second GMM based on the closest Gaussian component and the selected Gaussian component.

6. The method of claim 5 , wherein the calculating of the 3D coordinates of the plurality of joints further comprises selecting the N-part GMM from a plurality of N-part GMMs corresponding to a plurality of views based on the view of the second GMM,
wherein the plurality of N-part GMMs is generated in advance using a dataset comprising information related to the plurality of joints included in the object.

7. An apparatus for estimating a pose, the apparatus comprising:
at least one processor configured to:
obtain a plurality of vector sets corresponding to a plurality of patches included in an input image using an estimator trained based on an association between synthetic data in which an object is synthesized and real data in which the object is photographed; and
estimate a pose of an input object included in the input image based on the plurality of vector sets, by generating a plurality of mixture models corresponding to a plurality of joints included in the input object based on the plurality of vector sets.

Assignees

• Samsung Electronics Co Ltd
• Imp Innovations Ltd

Inventors

• Han Jae Joon
• Tang Danhang
• Kim Tae Kyun
• Han Seung Ju
• Yoo Byung In
• Choi Chang Kyu
• Tejani Alykhan
• Chang Hyung Jin

Classifications

• G06V10/7625

  Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2155

  characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling · CPC title

• G06V40/11 Primary

  Hand-related biometrics; Hand pose recognition · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title