Methods and systems for generating a three dimensional representation of a subject

Patent metadata

Field	Value
Publication number	US-9646408-B2
Application number	US-201414270427-A
Country	US
Kind code	B2
Filing date	May 6, 2014
Priority date	Jun 21, 2013
Publication date	May 9, 2017
Grant date	May 9, 2017

Abstract

According to one embodiment, a method of generating a three dimensional representation of a subject from a depth image, comprises comparing a depth image of the subject with a plurality of representative images, wherein each representative image is associated with a respective parametric model of a subject; identifying a representative image of the plurality of representative images as a closest representative image to the depth image of the subject; selecting the parametric model associated with the closest representative image to the depth image; and generating a three dimensional representation of the subject by fitting the selected parametric model to the depth image of the subject.

First claim

The invention claimed is:

1. A method of generating a three dimensional representation of the shape of a subject from a depth image, the method comprising
comparing a depth image of the subject with a plurality of representative images, wherein each representative image is associated with a respective parametric model for the shape and pose of a subject, each respective parametric model including numerical variables and the representative images being representations of a mean value of said variables for a cluster of possible shapes and poses of the subject;
identifying a representative image of the plurality of representative images as a closest representative image to the depth image of the subject;
selecting the parametric model for the shape and pose of the subject associated with the closest representative image to the depth image; and
generating a three dimensional representation of the subject by fitting the selected parametric model to the depth image of the subject.

2. The method according to claim 1 , wherein fitting the selected parametric model to the depth image of the subject comprises minimising an energy function, wherein the energy function comprises a distance term which depends on the distance between points on the depth image of the subject and corresponding points on the selected parametric model.

3. The method according to claim 2 , wherein the distance term gives a preference to points on the model being inside a volume defined by points on the depth image.

4. The method according to claim 2 , wherein the distance term comprises a gravity function dependant on the normal to a surface of the selected model and a gravity direction.

5. The method according to claim 1 , wherein each representative image is a depth image calculated from the respective parametric model.

6. The method according to claim 1 , wherein each representative image is a weighted combination of a depth image and an alpha image calculated from the respective parametric model.

7. The method according to claim 1 , further comprising normalising the orientation of the subject and/or the depth of the subject in the depth image before comparing the depth image of the subject with the plurality of representative images.

8. The method according to claim 1 , further comprising reducing the resolution of the depth image before comparing the depth image of the subject with the plurality of representative images.

9. The method according to claim 1 , further comprising calculating a size indicator and/or a measurement of the subject from the three dimensional representation.

10. A non-transitory computer readable medium carrying processor executable instructions which when executed on a processor cause the processor to carry out the method according to claim 1 .

11. A system for generating a three dimensional representation of the shape of a subject from a depth image, the system comprising
storage for a plurality of parametric models for the shape and pose of the subject and a plurality of representative images, each representative image being associated with a respective parametric model, each respective parametric model including numerical variables and the representative images being representations of a mean value of said variables for a cluster of possible shapes and poses of the subject;
a selection module operable to
compare the depth image of the subject with each of the plurality of representative images;
identify a representative image of the plurality of representative images as a closest representative image to the depth image of the subject; and
select the parametric model for the shape and pose of the subject associated with the closest representative image to the depth image; and

a fitting module operable to generate a three dimensional representation of the subject by fitting the selected parametric model to the depth image of the subject.

12. The system according to claim 11 , wherein the fitting module is operable to fit the selected parametric model to the depth image of the subject by minimising an energy function, wherein the energy function comprises a distance term which depends on the distance between points on the depth image of the subject and corresponding points on the selected parametric model.

13. The system according to claim 12 , wherein the distance term gives a preference to points on the model being inside a volume defined by points on the depth image.

14. The system according to claim 12 , wherein the distance term comprises a gravity function dependant on the normal to a surface of the selected model and a gravity direction.

15. The system according to claim 11 , wherein each representative image is a depth image calculated from the respective parametric model.

16. The system according to claim 11 , wherein each representative image is a weighted combination of a depth image and an alpha image calculated from the respective parametric model.

17. The system according to claim 11 , wherein each representative image as a resolution of less than 100×100 pixels.

18. The system according to claim 11 , wherein the selection module is operable to normalise the orientation of the subject and/or the depth of the subject in the depth image before comparing the depth image of the subject with the plurality of representative images.

19. The system according to claim 11 , being further operable to calculate a size indicator and/or a measurement of the subject from the three dimensional representation.

20. The system according to claim 11 , further comprising a depth image capture device configured to capture the depth image of the subject.

Assignees

• Toshiba Kk

Inventors

• Johnson Sam
• Perbet Frank
• Stenger Bjorn
• Pham Minh-Tri
• Woodford Oliver
• Gherardi Riccardo
• Cipolla Roberto

Classifications

• G06T17/20 Primary

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T15/00 Primary

  Three-dimensional [3D] image rendering · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title