Systems and methods for creating and distributing modifiable animated video messages

Patent metadata

Field	Value
Publication number	US-9747495-B2
Application number	US-201313787541-A
Country	US
Kind code	B2
Filing date	Mar 6, 2013
Priority date	Mar 6, 2012
Publication date	Aug 29, 2017
Grant date	Aug 29, 2017

Abstract

Systems and methods in accordance with embodiments of the invention enable collaborative creation, transmission, sharing, non-linear exploration, and modification of animated video messages. One embodiment includes a video camera, a processor, a network interface, and storage containing an animated message application, and a 3D character model. In addition, the animated message application configures the processor to: capture a video sequence using the video camera; detect a human face within a sequence of video frames; track changes in human facial expression of a human face detected within a sequence of video frames; map tracked changes in human facial expression to motion data, where the motion data is generated to animate the 3D character model; apply motion data to animate the 3D character model; render an animation of the 3D character model into a file as encoded video; and transmit the encoded video to a remote device via the network interface.

First claim

What is claimed is:

1. A method for generating animated video messages comprising:
generating groupings of animation sequences, wherein:
each grouping of animation sequences corresponds to an action in an animated sequence; and
each animation within a grouping of animation sequences corresponds to an alternative way to perform the action corresponding to that grouping; receiving a user selection of an animation from each of the groupings of animation sequences; and
creating an interactive animated video message using the user selected animations.

2. The method as recited in claim 1 , wherein creating the interactive animated video message using the user selected animations comprises:
generating motion data based on the user selected animations; and
applying the motion data to a 3 D character model to cause the 3 D character model to perform the user selected animations.

3. The method as recited in claim 2 , further comprising arranging the user selected animations based on an order of the groupings of animation sequences.

4. The method as recited in claim 2 , further comprising randomizing an order of the user selected animations before generating the motion data.

5. The method as recited in claim 2 , further comprising extracting an animation sequence from the interactive animated video message for reuse in another animated video message.

6. The method as recited in claim 2 , further comprising transmitting the motion data to a remote device via a network interface.

7. The method as recited in claim 1 , further comprising obtaining an animation within a grouping of animation sequences corresponding to an action by:
receiving a sequence of video frames;
detecting a human face within the sequence of the video frames;
tracking, within the sequence of video frames, a facial expression change in the detected human face;
mapping the tracked facial expression changes to 3 D motion data; and
applying the mapped 3 D motion data to a 3 D character model to cause the 3 D character model to perform the animation.

8. The method as recited in claim 7 , further comprising matching the detected facial expressions to a stored facial expression model from a database of stored facial expressions.

9. The method as recited in claim 8 , further comprising obtaining, based on matching the detected facial expressions to the stored facial expression model, the mapped 3 D motion data for the stored facial expression model.

10. The method as recited in claim 9 , wherein the 3 D character model comprises data that describes an articulated skeleton, at least one mesh, and skinning weights that indicate how the articulated skeleton deforms the at least one mesh for a given motion.

11. The method as recited in claim 10 , wherein the articulated skeleton comprises a plurality of bones configured to rotate in one or more degrees of freedom around joints, and wherein the at least one mesh comprises a plurality of vertices defining a surface having a plurality of polygons.

12. The method as recited in claim 1 , further comprising obtaining a description of a space of human faces and principal component analysis facial expressions using a training data set containing multiple images of human faces registered to a template image of a human face and multiple images of human facial expressions registered to the same template image of the human face.

13. A system for generating animated video messages, the system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
generate groupings of animation sequences, wherein:
each grouping of animation sequences corresponds to an action in an animated sequence; and
each animation within a grouping of animation sequences corresponds to an alternative way to perform action corresponding to that grouping; receiving a user selection of an animation from each of the groupings of animation sequences;

create an interactive animated video message using the user selected animations; and
transmit the interactive animated video message to a remote device via a network interface.

14. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to extract an animation sequence from the interactive animated video message for reuse in another animated video message.

15. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to:
generate motion data based on the interactive animated video message; and
apply the motion data to a 3 D character model to cause the 3 D character model to perform the actions of the interactive animated video message.

16. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to playback the interactive animated video message via a display device.

17. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to obtain an animation within a grouping of animation sequences corresponding to an action is by:
receiving a sequence of video frames;
detecting a human face within the sequence of the video frames;
tracking, within the sequence of video frames, a facial expression change in the detected human face;
mapping the tracked facial expression changes to 3 D motion data; and
applying the mapped 3 D motion data to a 3 D character model to cause the 3 D character model to the animation.

18. The system as recited in claim 17 , further comprising instructions that, when executed by the at least one processor, cause the system to match the detected facial expressions to a stored facial expression model from a database of stored facial expressions.

19. The system as recited in claim 18 , further comprising instructions that, when executed by the at least one processor, cause the system to obtain, based on matching the detected facial expressions to the stored facial expression model, the mapped 3 D motion data for the stored facial expression model.

20. The system as recited in claim 19 , wherein the 3 D character model comprises data that describes an articulated skeleton, at least one mesh, and skinning weights that indicate how the articulated skeleton deforms the at least one mesh for a given motion.

21. The system as recited in claim 20 , wherein the articulated skeleton comprises a plurality of bones configured to rotate in one or more degrees of freedom around joints, and wherein the at least one mesh comprises a plurality of vertices defining a surface having a plurality of polygons.

22. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to obtain a description of a space of human faces and principal component analysis facial expressions using a training data set containing multiple images of human faces registered to a template image of a human face and multiple images of human facial expressions registered to the same template image of the human face.

23. A method for generating animated videos comprising:
generating groupings of animation sequences, wherein:
each grouping of animation sequences corresponds to an action in a sequence of animations; and
each anim

Assignees

• Adobe Systems Inc

Inventors

• Corazza Stefano
• Babcock Daniel
• Pina Charles
• Drouin Sylvio

Classifications

• G06V40/176

  Dynamic expression · CPC title

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06K9/00228

  Physics · mapped topic

• G06K9/00315 Primary

  Physics · mapped topic