Virtual avatar animation based on facial feature movement

Patent metadata

Field	Value
Publication number	US-12482161-B2
Application number	US-202117461014-A
Country	US
Kind code	B2
Filing date	Aug 30, 2021
Priority date	Jan 18, 2019
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

The present disclosure generally relates to displaying visual effects such as virtual avatars. An electronic device having a camera and a display apparatus displays a virtual avatar that changes appearance in response to changes in a face in a field of view of the camera. In response to detecting changes in one or more physical features of the face in the field of view of the camera, the electronic device modifies one or more features of the virtual avatar.

First claim

What is claimed is:

1 . An electronic device, comprising:
a display apparatus; one or more cameras; one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
displaying, via the display apparatus, a virtual avatar having:
a respective avatar feature that moves independently of detected changes in pose of a face in a field of view of the one or more cameras; and
a plurality of avatar features that change appearance in response to detected changes in pose of the face in the field of view of the one or more cameras, the plurality of avatar features including:
a first avatar feature;
a second avatar feature different from the first avatar feature; and
a third avatar feature different from the first and second avatar features;

while displaying the respective avatar feature moving independently of changes in pose of the face in the field of view of the one or more cameras and while the face is detected in the field of view of the one or more cameras, the face including a plurality of detected facial features including a first facial feature and a second facial feature different from the first facial feature, detecting movement of one or more facial features of the face; and
in response to detecting the movement of the one or more facial features:
in accordance with a determination that the detected movement of the one or more facial features includes movement of the first facial feature in a first direction:
moving the first avatar feature in the first direction based on the detected movement of the first facial feature in the first direction;
moving the second avatar feature and the third avatar feature in a second direction different from the first direction and based on the detected movement of the first facial feature in the first direction; and
continuing to display the respective avatar feature moving independently of changes in pose of the face in the field of view of the one or more cameras; and

in accordance with a determination that the detected movement of the one or more facial features includes movement of the second facial feature:
moving the second avatar feature based on at least one of a direction and a magnitude of movement of the second facial feature; and
continuing to display the respective avatar feature moving independently of changes in pose of the face in the field of view of the one or more cameras.

2 . The electronic device of claim 1 , the one or more programs further including instructions for:
in response to detecting the movement of the one or more facial features and in accordance with a determination that the detected movement of the one or more facial features includes movement of the second facial feature without more than a first threshold amount of movement of the first facial feature:
moving the second avatar feature based on at least one of the direction and the magnitude of movement of the second facial feature;
forgoing moving the first and third avatar features; and
continuing to display the respective avatar feature moving independently of changes in pose of the face in the field of view of the one or more cameras.

3 . The electronic device of claim 1 , the one or more programs further including instructions for:
in response to detecting the movement of the one or more facial features and in accordance with a determination that the detected movement of the one or more facial features includes movement of the first facial feature without more than a second threshold amount of movement of the second facial feature and a third facial feature:
moving the second and third avatar features based on the detected movement of the first facial feature; and
continuing to display the respective avatar feature moving independently of changes in pose of the face in the field of view of the one or more cameras.

4 . The electronic device of claim 1 , the one or more programs further including instructions for:
in response to detecting the movement of the one or more facial features and in accordance with a determination that the detected movement of the one or more facial features includes movement of both the first and second facial features:
moving the first avatar feature based on the detected movement of the first facial feature;
moving the second avatar feature based on the detected movement of the first and second facial features; and
continuing to display the respective avatar feature moving independently of changes in pose of the face in the field of view of the one or more cameras.

5 . The electronic device of claim 1 , wherein moving the first avatar feature in the first direction based on the detected movement of the first facial feature in the first direction includes:
in accordance with a determination that the detected movement of the first facial feature is less than or equal to a second threshold amount of movement of the first facial feature, moving the first avatar feature in the first direction by a first amount determined based on the detected movement of the first facial feature; and in accordance with a determination that the detected movement of the first facial feature is greater than the second threshold amount of movement of the first facial feature:
moving the first avatar feature in the first direction by a second amount of movement greater than the first amount of movement, and
displaying a first subfeature of the first avatar feature that was not displayed when the movement of the first facial feature was less than the second threshold amount of movement.

6 . The electronic device of claim 1 , wherein:
the first facial feature is a jaw of the face; the first avatar feature is an avatar jaw feature; the second avatar feature is one or more avatar eyes; and the third avatar feature is an avatar nose.

7 . The electronic device of claim 1 , the one or more programs further including instructions for:
in response to detecting the movement of the one or more facial features and in accordance with a determination that the detected movement of the one or more facial features includes movement of the first facial feature in a third direction:
moving the second avatar feature and the third avatar feature in a fourth direction different from the third direction and based on the detected movement of the first facial feature in the third direction; and
continuing to display the respective avatar feature moving independently of changes in pose of the face in the field of view of the one or more cameras.

8 . The electronic device of claim 7 , the one or more programs further including instructions for:
further in accordance with a determination that the detected movement of the one or more facial features includes movement of the first facial feature in the third direction:
moving the first avatar feature in the third direction based on the detected movement of the first facial feature in the third direction.

9 . The electronic device of claim 1 , wherein moving the second avatar feature and the third avatar feature in the second direction based on the detected movement of the first facial feature includes:
in accordance with a determination that the detected movement of the one or more facial features includes movement of the first facial feature by a first magnitude, moving the second avatar feature and the third avatar feature by a first amount of movement; and in accordance with a determination that the detected movement of the one or more facial features includes movement of the first facial feature by a second magnitude different from the first magnitude, moving the second avatar feature and

Assignees

• Apple Inc

Inventors

• Scapel Nicolas
• Barlier Guillaume Pierre André
• Guzman Aurelio
• Rickwald Jason

Classifications

• G06V40/169

  Holistic features and representations, i.e. based on the facial image taken as a whole · CPC title

• G06V40/166

  using acquisition arrangements · CPC title

• G06V40/176

  Dynamic expression · CPC title

• G06V40/167

  using comparisons between temporally consecutive images · CPC title

• A63F13/55

  Controlling game characters or game objects based on the game progress · CPC title