Method and arrangement for image model construction

Patent metadata

Field	Value
Publication number	US-9324191-B2
Application number	US-201214122143-A
Country	US
Kind code	B2
Filing date	Jun 4, 2012
Priority date	Jun 20, 2011
Publication date	Apr 26, 2016
Grant date	Apr 26, 2016

Abstract

A method for constructing an image model (M 1 ; M) from at least one image data input (IV 1 ; IV 1 -IVn), comprises the steps of, in an iterative way,
determining at least one state (PS 1 ; PS 1 -PSn) of said at least one image data input (IV 1 ; IV 1 -IVn), and a state (PSMF) of an intermediate learning model (MF; MIF) determining a target state (TSP) from said at least one state (PS 1 ; PS 1 -PSn) of said at least one image data input, and from the state (PSMF) of said intermediate learning model (MF; MIF), performing at least one transformation in accordance with the determined target state (TSP) on said at least one image data input (IV 1 ; IV 1 -IVn), thereby generating at least one transformed image (IV 1 T; IV 1 T-IVnT), aggregating said at least one transformed image (IV 1 T; IV 1 T-IVnt) with intermediate learning model (MF; MIF; MIT; MFT) information, thereby generating an updated estimate of said image model (M 1 ; M), providing said updated estimate of said image model (M 1 ; M) as said image model (M 1 ; M) while also providing said updated estimate of said image model (M 1 ; M) in a feedback loop to a model object learning module ( 500 ) for deriving an update of said intermediate learning model (MF, MIF).

First claim

The invention claimed is:

1. A method for constructing an image model of an object from at least one image data input, the method comprising the steps of:
extracting at least one state comprising a configuration of object features, the features being represented by a set of values of the at least one image data input, and a state comprising a configuration of object features, the features being represented by a set of values of an intermediate learning model;
determining a target state by performing a weighted combination of the at least one state of the at least one image data input, and of the state of the intermediate learning model, with the weights reflecting confidences of the states, the confidences being determined during the state extraction;
performing at least one image transformation by using the determined target state on the at least one image data input, thereby generating at least one transformed image;
aggregating the at least one transformed image with intermediate learning model information into a single two-dimensional or three-dimensional image, thereby generating an updated estimate of the image model;
providing the updated estimate of the image model as the image model to an output while also
providing the updated estimate of the image model in a feedback loop to a model object learning module for deriving an update of the intermediate learning model; and
repeating the previous steps in an iterative process.

2. The method of claim 1 , wherein the at least one image data input comprises:
a first image data input comprising a video sequence of an object; and
a second image data input comprising a 3D image of the object, wherein at least one state of the second image data input comprises values representing position and morphing parameters of the object in 3D, such that successive updated estimates of the image model in 3D are provided as the image model.

3. The method of claim 2 , wherein the 3D image of the object shows the object with at least one different feature with respect to the video sequence images of the object, and wherein the image model in 3D of the object shows the at least one different feature.

4. The method of claim 2 further comprising the step of:
projecting the updated estimates of the image model in 3D onto a 2D plane to generate a projected updated image model, wherein the projected updated image model is provided as the image model.

5. The method of claim 1 , wherein the target state is further determined based on additional information related to an object for which the image model is to be generated.

6. The method of claim 1 , wherein the intermediate learning model is further derived from externally provided model information.

7. The method of claim 1 , further comprising the step of:
performing a transformation on the intermediate learning model using the determined target state to generate an intermediate transformed model, wherein during the aggregating step the intermediate transformed model is aggregated with the at least one transformed image to generate the updated estimate of the image model.

8. The method of claim 7 , wherein the at least one image data input comprises:
a first image data input comprising a video sequence of an object in 2D or 2D+z format; and
a second image data input comprising a 3D image of the object, wherein at least one state of the second image data input comprises a combination of values representing position and morphing parameters of the object in 3D, such that successive updated estimates of the image model in 3D are provided as the image model.

9. The method of claim 8 , wherein the 3D image of the object shows the object with at least one different feature with respect to the video sequence images of the object, and wherein the image model in 3D of the object shows the at least one different feature.

10. The method of claim 8 further comprising the step of:
projecting the updated estimates of the 3D image model onto a 2D plane, and providing the projections in 2D of the updated estimates as the image model.

11. An apparatus for constructing an image model of an object from at least one image data input, comprising one or more processors configured to:
extract at least one state comprising a configuration of object features, the features being represented by a set of values of the at least one image data input provided to at least one input of the apparatus, and of an intermediate learning model;
determine a target state by performing a weighted combination of the at least one state of the at least one image data input, and of the state of the intermediate learning model, with the weights reflecting confidences of the states, the confidences being determined during the state extraction;
perform at least one image transformation by using the determined target state on the at least one image data input, thereby generating at least one transformed image;
aggregate the at least one transformed image with intermediate learning model information into a single two-dimensional or three-dimensional image, thereby generating an updated estimate of the image model;
provide the updated estimate of the image model as the image model to an output of the apparatus; and to
provide the updated estimate of the image model in a feedback loop to a model object learning module of the apparatus, the model object learning module being adapted to derive from the update estimate of the image model an update of the intermediate learning model; and
such that the two-dimensional or three-dimensional image model is generated in an iterative process.

12. The apparatus of claim 11 , wherein the at least one image data input comprises:
a first image data input comprising a video sequence of an object; and
a second image data input comprising a 3D image of the object, wherein at least one state of the second image data input comprises values representing position and morphing parameters of the object in 3D, such that successive updated estimates of the image model in 3D are provided as the image model.

13. The apparatus of claim 11 , wherein the target state is further determined based on additional information related to an object for which the image model is to be generated.

14. The apparatus of claim 11 , further being adapted to perform a transformation on the intermediate learning model using the determined target state to generate an intermediate transformed model, wherein the at least one transformed image is aggregated with the at least one transformed image to generate the updated estimate of the image model.

15. The apparatus of claim 14 , wherein the at least one image data input comprises:
a first image data input comprising a video sequence of an object in 2D or 2D+z format; and
a second image data input comprising a 3D image of the object, wherein at least one state of the second image data input comprises a combination of values representing position and morphing parameters of the object in 3D, the apparatus being adapted to generate successive updated estimates of the image model in 3D as the image model.

16. A non-transitory computer readable medium encoding a machine-executable program of instructions to perform steps, comprising:
extracting at least one state comprising a configuration of object features, the features being represented by a set of values of the at least one image data input, and a state comprising a configuration of object features, the features being represented by a set of values of an intermediate learning model;
determining a target state by performing a weighted combinatio

Assignees

• Tytgat Donny
• Six Erwin
• Lievens Sammy
• Aerts Maarten
• Alcatel Lucent

Inventors

• Tytgat Donny
• Six Erwin
• Lievens Sammy
• Aerts Maarten

Classifications

• G06T7/251 Primary

  involving models · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/20121

  Active appearance model [AAM] · CPC title

• G06T2207/30201

  Face · CPC title

• G06T7/2046

  Physics · mapped topic