Adjusting a digital representation of a head region

Patent metadata

Field	Value
Publication number	US-10740985-B2
Application number	US-201816057566-A
Country	US
Kind code	B2
Filing date	Aug 7, 2018
Priority date	Aug 8, 2017
Publication date	Aug 11, 2020
Grant date	Aug 11, 2020

Abstract

Methods and devices for generating reference data for adjusting a digital representation of a head region, and methods and devices for adjusting the digital representation of a head region are disclosed. In some arrangements, training data are received. A first machine learning algorithm generates first reference data using the training data. A second machine learning algorithm generates second reference data using the same training data and the first reference data generated by the first machine learning algorithm.

First claim

The invention claimed is:

1. A method of generating reference data for adjusting a digital representation of a head region, the method comprising:
receiving training data comprising:
a set of input patches, each input patch comprising a target feature of a digital representation of a head region prior to adjustment of the digital representation of the head region, wherein the target feature is the same for each input patch; and
a set of output patches in one-to-one correspondence with the input patches, each output patch comprising the target feature of the digital representation of the head region after adjustment of the digital representation of the head region;

using a first machine learning algorithm to generate first reference data using the training data, the first reference data comprising editing instructions for adjusting the digital representation of the head region for a range of possible digital representations of the head region; and
using a second machine learning algorithm to generate second reference data using the same training data as the first machine learning algorithm and the first reference data generated by the first machine learning algorithm, the second reference data comprising editing instructions for adjusting the digital representation of the head region for a range of possible digital representations of the head region,
wherein the first reference data comprise first editing instructions for a range of possible configurations of the target feature and first selection instructions for selecting editing instructions for a particular input patch from the first editing instructions based on the configuration of the target feature of the input patch; and
wherein the second reference data comprise second editing instructions for a range of possible configurations of the target feature and second selection instructions for selecting editing instructions for a particular input patch from the second image editing instructions based on the configuration of the target feature of the input patch.

2. The method of claim 1 , wherein the configuration of the target feature of each input patch is represented by a feature vector derived from plural local descriptors of the input patch, and the first and second selection instructions define how the feature vector is used to select editing instructions for the input patch.

3. The method of claim 1 , wherein the editing instructions comprise a displacement vector field defining how the input patch is to be transformed.

4. The method of claim 1 , wherein the editing instructions comprise a filter field, a brightness adjustment field, or a texture blending field.

5. The method of claim 1 , wherein:
a first editing algorithm is used by the first machine learning algorithm to define how the first editing instructions are to be applied to an input patch to derive an output patch; and
a second editing algorithm is used by the second machine learning algorithm to define how the second editing instructions are to be applied to an input patch to derive an output patch.

6. The method of claim 5 , wherein:
the second editing instructions are principle component analysis components of a principle component analysis of the first editing instructions; and
the second image editing algorithm is configured to transform the second editing instructions into the first editing instructions by inverse principle component analysis transform.

7. The method of claim 5 , wherein:
the second editing instructions are wavelet components of the first editing instructions; and
the second editing algorithm is configured to transform the second editing instructions into the first editing instructions by inverse wavelet transform.

8. The method of claim 1 , wherein the first selection instructions for the first reference data are able to select between a larger number of alternative editing instructions than the second selection instructions for the second reference data.

9. The method of claim 1 , wherein the first machine learning algorithm is of a different machine learning algorithm type than the second machine learning algorithm.

10. The method of claim 1 , wherein the first machine learning algorithm comprises one or more of the following: a neural network; a support vector machine;
and a generative adversarial network (GAN).

11. The method of claim 1 , wherein the second machine learning algorithm comprises one or more of the following: a regression forest; regression ferns, cluster centres, a lookup table, and separable filter banks.

12. The method of claim 1 , wherein the target feature comprises one or more of the following: an eye region comprising at least part of an eye, a nose region comprising at least part of a nose, a mouth region comprising at least part of a mouth, a chin region comprising at least part of a chin, a neck region comprising at least part of a neck, and a hair region comprising hair.

13. A method of adjusting a digital representation of a head region, the method comprising
receiving a digital representation of a head region; and
using reference data comprising editing instructions to adjust the digital representation of the head region, wherein the reference data comprises the second reference data generated by the method of claim 1 .

14. A non-transitory computer readable storage medium storing a computer program capable of execution by a processor and arranged on execution to cause the processor to perform a method according to claim 1 .

15. A method of training a machine learning algorithm to adjust a digital representation of a head region, the method comprising:
receiving training data comprising a set of input digital representations of a head region;
training a first machine learning algorithm using the training data to perform an adjustment of a digital representation of a head region;
using the trained first machine learning algorithm to generate first reference data, the first reference data comprising an adjusted digital representation of the head region for each of at least a subset of the input digital representations, each adjusted digital representation being obtained by performed the adjustment that the first machine learning algorithm was trained to perform; and
training a second machine learning algorithm using at least a subset of the training data used to train the first machine learning algorithm and the first reference data to perform the same adjustment of a digital representation of a head region as the first machine learning algorithm, wherein the adjustment of the digital representation comprises converting from a two-dimensional digital representation to a three-dimensional digital representation, and wherein the training of the first machine learning algorithm comprises iteratively using a rendering process to generate a two-dimensional digital representation from a three-dimensional digital representation generated by the first machine learning algorithm and comparing the generated digital representation with a corresponding digital representation in the training data.

16. A device for generating reference data for adjusting a digital representation of a head region, the device comprising a data processing unit arranged to:
receive training data comprising:
a set of input patches, each input patch comprising information about a target feature of a digital representation of the head region prior to adjustment of the digital representation of the head region, wherein the target feature is the same for each input patch; and
a set of output patches in one-to-one correspondence with the input p

Assignees

• Reald Spark Llc

Inventors

• Sommerlade Eric
• Neophytou Alexandros

Classifications

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title