Augmented facial animation

Patent metadata

Field	Value
Publication number	US-11587279-B2
Application number	US-202217682862-A
Country	US
Kind code	B2
Filing date	Feb 28, 2022
Priority date	Dec 23, 2014
Publication date	Feb 21, 2023
Grant date	Feb 21, 2023

Abstract

Examples of systems and methods for augmented facial animation are generally described herein. A method for mapping facial expressions to an alternative avatar expression may include capturing a series of images of a face, and detecting a sequence of facial expressions of the face from the series of images. The method may include determining an alternative avatar expression mapped to the sequence of facial expressions, and animating an avatar using the alternative avatar expression.

First claim

What is claimed is:

1. At least one storage device comprising instructions that, when executed, cause processor circuitry to:
identify an augmented facial appearance based on user input, the augmented facial appearance to be presented on a display based on one or more images of a face captured by a camera, the augmented facial appearance corresponding to an expression beyond an ability of a human to express;
detect a particular movement of the face in successive ones of the one or more images; and
in response to detection of the particular movement, generate augmented images based on the successive images, the augmented images including the augmented facial appearance.

2. The at least one storage device of claim 1 , wherein the particular movement includes a mouth opening.

3. The at least one storage device of claim 1 , wherein the augmented facial appearance includes a mouth that is larger than humanly possible relative to other facial features in the augmented facial appearance.

4. The at least one storage device of claim 1 , wherein the instructions cause the processor circuitry to cause the camera to capture the one or more images of the face.

5. The at least one storage device of claim 1 , wherein the instructions cause the processor circuitry to detect a hand in the one or more images.

6. The at least one storage device of claim 1 , wherein the instructions cause the processor circuitry to detect the particular movement based on detection of movement of a particular landmark point on the face.

7. The at least one storage device of claim 6 , wherein the instructions cause the processor circuitry to detect the movement of the particular landmark point using dynamic time warping.

8. The at least one storage device of claim 1 , wherein the augmented facial appearance is one of a plurality of augmented facial appearances, different ones of the plurality of augmented facial appearances selectable by a user.

9. A method comprising:
identifying an augmented facial animation based on user input, the augmented facial animation to be presented on a display based on one or more images of a face captured by a camera, the augmented facial animation including an appearance beyond an ability of a human face to replicate;
detecting a particular movement of the face in successive ones of the one or more images; and
in response to detection of the particular movement, generating, by executing an instruction with processor circuitry, augmented images based on the successive images, the augmented images including the augmented facial animation.

10. The method of claim 9 , wherein the particular movement includes a mouth opening.

11. The method of claim 9 , wherein the augmented facial animation includes a mouth that is larger than humanly possible relative to other facial features in the augmented facial animation.

12. The method of claim 9 , further including causing the camera to capture the one or more images of the face.

13. The method of claim 9 , further including detecting a hand in the one or more images.

14. The method of claim 9 , further including detecting the particular movement based on detection of movement of a particular landmark point on the face.

15. The method of claim 14 , further including detecting the movement of the particular landmark point using dynamic time warping.

16. The method of claim 9 , wherein the augmented facial animation is one of a plurality of augmented facial animations, different ones of the plurality of augmented facial animations selectable by a user.

17. At least one storage device comprising instructions that, when executed, cause processor circuitry to:
identify a facial feature to be changed based on user input, the changed facial feature to be presented on a display based on one or more images of a face captured by a camera, the facial feature to be changed to produce a surreal facial appearance;
detect a particular movement of the face in successive ones of the one or more images; and
in response to detection of the particular movement, generate updated images based on the successive images, the updated images including the surreal facial appearance.

18. The at least one storage device of claim 17 , wherein the particular movement includes a mouth opening.

19. The at least one storage device of claim 17 , wherein the surreal facial appearance corresponds to a mouth that is larger than humanly possible relative to other facial features.

20. The at least one storage device of claim 17 , wherein the processor circuitry is to cause the camera to capture the one or more images of the face.

21. The at least one storage device of claim 17 , wherein the processor circuitry is to detect a hand in the one or more images.

22. The at least one storage device of claim 17 , wherein the processor circuitry is to detect the particular movement based on detection of movement of a particular landmark point on the face.

23. The at least one storage device of claim 22 , wherein the processor circuitry is to detect the movement of the particular landmark point using dynamic time warping.

24. The at least one storage device of claim 17 , wherein the facial feature to be changed is one of a plurality of facial features to be changed, different ones of the plurality of facial features selectable by a user.

25. An apparatus comprising
memory;
instructions; and
processor circuitry to execute the instructions to:
identify an augmented facial image based on user input, the augmented facial image to be presented on a display based on one or more images of a face captured by a camera, the augmented facial image corresponding to a surreal facial appearance;
detect a particular movement of the face in successive ones of the one or more images; and
in response to detection of the particular movement, generate updated images based on the successive images, the updated images including the surreal facial appearance.

26. The apparatus of claim 25 , wherein the particular movement includes a mouth opening.

27. The apparatus of claim 25 , wherein the surreal facial appearance corresponds to a mouth that is larger than humanly possible relative to other facial features.

28. The apparatus of claim 25 , wherein the processor circuitry is to cause the camera to capture the one or more images of the face.

29. The apparatus of claim 25 , wherein the processor circuitry is to detect a hand in the one or more images.

30. The apparatus of claim 25 , wherein the processor circuitry is to detect the particular movement based on detection of movement of a particular landmark point on the face.

31. The apparatus of claim 30 , wherein the processor circuitry is to detect the movement of the particular landmark point using dynamic time warping.

32. The apparatus of claim 25 , wherein the surreal facial appearance is one of a plurality of surreal facial appearances, different ones of the plurality of surreal facial appearances selectable by a user.

Assignees

• Intel Corp

Inventors

• Fang Yikai
• Du Yangzhou
• Li Qiang Eric
• Tong Xiaofeng
• Li Wenlong
• Park Minje
• Ju Myung-Ho
• Yi Jihyeon Kate
• Kim Tae-Hoon Pete

Classifications

• G06V40/176

  Dynamic expression · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06V40/166

  using acquisition arrangements · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title