Pose estimation method, method of displaying virtual object using estimated pose, and apparatuses performing the same

Patent metadata

Field	Value
Publication number	US-10937189-B2
Application number	US-201916238785-A
Country	US
Kind code	B2
Filing date	Jan 3, 2019
Priority date	Jan 18, 2018
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

Disclosed are a pose estimation methods and apparatuses of displaying a virtual object using an estimated pose. The pose estimation method includes receiving an input image and estimating pose information of an object from the input image based on local information of the object.

First claim

What is claimed is:

1. A pose estimation method comprising:
receiving an input image;
estimating pose information of an object from the input image, using a neural network, based on local information of the object; and
concurrently estimating keypoint information of the object while estimating the pose information of the object,
wherein the local information is based on the keypoint information, and
wherein a task of the estimating of the pose information and a task of the estimating of the keypoint information share parameters of a base layer in the neural network.

2. The pose estimation method of claim 1 , wherein the estimating of the pose information of the object comprises:
correcting the pose information using the keypoint information.

3. The pose estimation method of claim 1 , wherein the task of the estimating of the pose information and the task of the estimating of the keypoint information are connected in the neural network through one of a parallel mode and a cascade mode,
wherein the keypoint information is input to the task of estimating the pose information in the cascade mode.

4. The pose estimation method of claim 1 , wherein the neural network comprises:
a first path comprising the base layer and one or more convolution layers to estimate the keypoint information; and
a second path comprising the base layer and one or more fully-connected layers to estimate the pose information.

5. The pose estimation method of claim 1 , wherein the neural network comprises:
a first path comprising the base layer and one or more convolution layers to estimate the keypoint information; and
a second path comprising the base layer, one or more convolution layers, and one or more fully-connected layers to estimate the pose information,
wherein an output of one of the convolution layers in the first path is connected to an output of one of the convolution layers in the second path to be input to the fully-connected layers.

6. The pose estimation method of claim 1 , wherein the neural network comprises:
a first path comprising the base layer and one or more convolution layers to estimate the keypoint information; and
a second path comprising the base layer, the one or more convolution layers in the first path, and one or more fully-connected layers to estimate the pose information,
wherein outputs of two or more of the convolution layers in the first path are connected and input to one of the fully-connected layers in the second path.

7. The pose estimation method of claim 1 , further comprising:
concurrently estimating class information of the object from the input image while estimating the pose information and the keypoint information of the object.

8. The pose estimation method of claim 7 , wherein the task of estimating the pose information, the task of estimating the keypoint information, and a task of estimating the class information share the parameters of the base layer in the neural network.

9. The pose estimation method of claim 8 , wherein the neural network further comprises:
a third path comprising the base layer and one or more connected layers to estimate the class information.

10. An apparatus comprising:
a processor configured to:
receive an input image;
estimate pose information of an object from the input image, through a neural network, based on local information of the object; and
concurrently estimating keypoint information of the object, through the neural network, while estimating the pose information of the object,
wherein the local information is based on the keypoint information, and
wherein a task of the estimating of the pose information and a task of the estimating of the keypoint information share parameters of a base layer in the neural network.

11. The apparatus of claim 10 , wherein the processor is further configured to:
correct the pose information using the keypoint information.

12. The apparatus of claim 10 , wherein the neural network comprises:
a first path comprising the base layer and one or more convolution layers to estimate the keypoint information; and
a second path comprising the base layer and one or more fully-connected layers to estimate the pose information.

13. The apparatus of claim 10 , wherein the neural network comprises:
a first path comprising the base layer and one or more convolution layers to estimate the keypoint information; and
a second path comprising the base layer, one or more convolution layers, and one or more fully-connected layers to estimate the pose information,
wherein an output of one of the convolution layers in the first path is connected to an output of one of the convolution layers in the second path to be input to the fully-connected layers.

14. The apparatus of claim 10 , wherein the neural network comprises:
a first path comprising the base layer and one or more convolution layers to estimate the keypoint information; and
a second path comprising the base layer, the one or more convolution layers in the first path, and one or more fully-connected layers to estimate the pose information,
wherein outputs of two or more of the convolution layers in the first path are connected and input to one of the fully-connected layers in the second path.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Li Weiming
• Wang Zairan
• Yu Dongdong
• Wang Hao
• Ahn Minsu
• Wang Qiang
• Hong Sunghoon
• Liu Yang
• Kao Yueying

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title