Avatar facial expression representation in multidimensional space

Patent metadata

Field	Value
Publication number	US-11527031-B2
Application number	US-202117385621-A
Country	US
Kind code	B2
Filing date	Jul 26, 2021
Priority date	Apr 23, 2018
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

Examples of the disclosed systems and methods may provide for improved and more realistic rendering of virtual characters and a more realistic interaction between a user and virtual characters. For example, the systems and methods describe techniques for mathematically generating a map used for animating facial expressions in a multidimensional animation blendspace. As another example, the systems and methods describe a transition system for dynamically transitioning facial expressions across a face of the virtual character. As another example, realistic physical movements can be added to a virtual character's facial expressions to provide interactivity with other virtual characters.

First claim

What is claimed is:

1. A system for transitioning a first expression to a second expression for a virtual character, the system comprising:
non-transitory storage medium storing vector values for facial expressions of a virtual character in multidimensional vector space;
wherein an angular difference between vectors in the multidimensional vector space corresponds to a difference in emotion type between facial expressions associated with the vectors; and
wherein a difference in magnitude between the vectors corresponds to a difference in intensity of emotion between the facial expressions associated with the vectors; and
a hardware processor programmed to:
detect a trigger for an expression change from a first facial expression to a second facial expression;
determine first values of parameters indicating a starting vector in the multidimensional vector space associated with the first facial expression, an ending vector in the multidimensional vector space associated with the second facial expression, and a sweep direction comprising a direction of propagation of expression change from the first facial expression to the second facial expression on a face of the virtual character; and
animate a transition from the first facial expression to the second facial expression based on the first values of parameters.

2. The system of claim 1 , wherein the trigger comprises: an interaction of a user with the virtual character or an event in an environment of the virtual character.

3. The system of claim 1 , wherein the first facial expression comprises a first portion of the face having a first expression.

4. The system of claim 1 , wherein the hardware processor is further programmed to calculate a rate of change of expressions.

5. The system of claim 1 , wherein a facial expression of the virtual character's face is controlled by a face vector comprising a plurality of dimensions with each vector value corresponding to a dimension of the plurality of dimensions.

6. The system of claim 1 , wherein at least one of the first or second facial expression can be determined from a wheel shaped map comprising a set of expressions projected on to a plurality of arms of the wheel shaped map wherein similar expressions are located in the same directions on the wheel shaped map.

7. The system of claim 6 , wherein a movement from the first facial expression to the second facial expression is associated with an expression change trajectory on the wheel shaped map.

8. A method for transitioning a first expression to a second expression for a virtual character, the method comprising:
under control of a hardware processor:
determining a first starting facial expression and a first ending facial expression for a first time period during a transition from the first expression to the second expression;
updating vector values of a face vector for changing a virtual character's facial expression from the first starting facial expression to the first ending facial expression at the first time period,
wherein an angular difference between vector values corresponds to a difference in emotion type between facial expressions associated with the vectors; and
wherein a difference in magnitude between vector values corresponds to a difference in intensity of emotion between the facial expressions associated with the vectors; and

determining a second starting facial expression and a second ending facial expression for a second time period to transition to a third expression; and
updating the vector values for changing the virtual character's facial expression from the second starting facial expression to the second ending facial expression at the second time period, wherein the second time period occurs during the transition between the first expression to the second expression.

9. The method of claim 8 , wherein the second ending facial expression comprises a first portion of the face having a first facial expression associated with the second facial expression and a second portion of the face having a second expression associated with the third expression.

10. The method of claim 9 , the second ending facial expression comprises the first and the second expressions in response to a determination that a rate of change of expressions is faster than a sweep speed associated with transitioning from the first facial expression to the second facial expression.

11. The method of claim 8 , wherein a facial expression of the virtual character's face is controlled by a face vector comprising a plurality of dimensions with each vector value corresponding to a dimension of the plurality of dimensions.

12. The method of claim 8 , wherein the second starting facial expression is the same as the first ending facial expression.

13. The method of claim 8 , wherein the first facial expression is the same as the third facial expression.

14. The method of claim 8 , wherein at least one of the first, second, or third facial expression can be determined from a wheel shaped map comprising a set of expressions projected on to a plurality of arms of the wheel shaped map wherein similar expressions are located in the same directions on the wheel shaped map.

Assignees

• Magic Leap Inc

Inventors

• Miller Iv Thomas Marshall
• Quach The Hung
• Lin Jeffrey

Classifications

• G06V40/174

  Facial expression recognition · CPC title

• G06V40/107

  Static hand or arm · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title