Generating three-dimensional looping animations from still images

Patent metadata

Field	Value
Publication number	US-12524944-B2
Application number	US-202318340445-A
Country	US
Kind code	B2
Filing date	Jun 23, 2023
Priority date	Jun 23, 2023
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

A system that utilizes neural networks to generate looping animations from still images. The system fits a 3D model to a pose of a person in a digital image. The system receives a 3D animation sequence that transitions between a starting pose and an ending pose. The system generates, utilizing an animation transition neural network, first and second 3D animation transition sequences that respectively transition between the pose of the person and the starting pose and between the ending pose and the pose of the person. The system modifies each of the 3D animation sequence, the first 3D animation transition sequence, and the second 3D animation transition sequence by applying a texture map. The system generates a looping 3D animation by combining the modified 3D animation sequence, the modified first 3D animation transition sequence, and the modified second 3D animation transition sequence.

First claim

What is claimed is:

1 . A method comprising:
fitting a three-dimensional (“3D”) model to a pose of a person in a digital image; receiving, from a graphical user interface of a client device, a selection of a 3D animation sequence that transitions between a starting pose and an ending pose; generating, utilizing an animation transition neural network, a first 3D animation transition sequence of the 3D model that transitions between the pose of the person in the digital image and the starting pose; generating, utilizing the animation transition neural network, a second 3D animation transition sequence of the 3D model that transitions between the ending pose and the pose of the person in the digital image; modifying, utilizing an animation rendering neural network, each of the 3D animation sequence, the first 3D animation transition sequence, and the second 3D animation transition sequence by applying a texture map of the person to each animation sequence; and generating, for display on the graphical user interface, a looping 3D animation by combining the modified 3D animation sequence, the modified first 3D animation transition sequence, and the modified second 3D animation transition sequence.

2 . The method of claim 1 , wherein receiving the selection of the 3D animation sequence that transitions between the starting pose and the ending pose comprises receiving a selection of a 3D animations sequence from an animation repository.

3 . The method of claim 1 , wherein receiving the selection of the 3D animation sequence that transitions between the starting pose and the ending pose comprises:
receiving a selection of a video of a person performing a movement sequence; and fitting a 3D model to the person in various frames of the video to generate the 3D animation sequence.

4 . The method of claim 1 , further comprising:
generating a second looping 3D animation; and displaying the looping 3D animation and the second looping 3D animation together on a background image.

5 . The method of claim 1 , wherein receiving the selection of the 3D animation sequence comprises receiving a selected target motion type from a plurality of different motion types displayed in the graphical user interface.

6 . The method of claim 5 , wherein generating the first 3D animation transition sequence comprises generating a first 3D animation transition sequence to have the selected target motion type.

7 . The method of claim 5 , wherein the selected target motion type is dancing.

8 . The method of claim 1 , wherein generating, utilizing the animation transition neural network, the first 3D animation transition sequence of the 3D model that transitions between the pose of the person in the digital image and the starting pose comprises performing neural inbetweening.

9 . A system comprising:
a memory component; and one or more processing devices coupled to the memory component, the one or more processing devices to perform operations comprising:
fitting a three-dimensional (“3D”) model to a pose of a person in a digital image;
receiving a selection of a 3D animation sequence that transitions between a starting pose and an ending pose;
generating a first 3D animation transition sequence that transitions between the pose of the person in the digital image and the starting pose;
generating a second 3D animation transition sequence that transitions between the pose of the person in the digital image and the ending pose; and
generating a looping 3D animation of the person by:
combining the 3D animation sequence, the first 3D animation transition sequence, and the second 3D animation transition sequence into a combined 3D animation sequence;
synthesizing texture maps providing an appearance of the person on the combined 3D animation sequence; and
rendering the texture maps on the combined 3D animation sequence.

10 . The system of claim 9 , wherein generating the first 3D animation transition sequence that transitions between the pose of the person in the digital image and the starting pose comprises generating a plurality of sequential poses of a 3D model that transition from the pose of the person in the digital image to the starting pose.

11 . The system of claim 9 , wherein generating the second 3D animation transition sequence that transitions between the pose of the person in the digital image and the ending pose comprises generating a plurality of sequential poses of a 3D model that transition from the ending pose to the pose of the person in the digital image.

12 . The system of claim 9 , further comprising:
receiving a selection of a background; and compositing the looping 3D animation on the selected background.

13 . The system of claim 9 , wherein generating the looping 3D animation of the person comprises generating views of portions of the person not visible in the digital image.

14 . The system of claim 9 , wherein synthesizing the texture maps providing the appearance of the person on the combined 3D animation sequence comprises synthesizing motion-dependent texture that models wrinkles and shade in a motion dependent manner.

15 . The system of claim 9 , wherein generating the looping 3D animation is performed in response to user selection of the 3D animation sequence without further user input.

16 . A non-transitory computer-readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations comprising:
fitting a three-dimensional (“3D”) model to a pose of a person in a digital image; receiving a selection of a 3D animation sequence that transitions between a starting pose and an ending pose; generating, utilizing an animation transition neural network, a first 3D animation transition sequence that transitions between the pose of the person in the digital image and the starting pose; generating, utilizing the animation transition neural network, a second 3D animation transition sequence that transitions between the pose of the person in the digital image and the ending pose; and generating, utilizing an animation rendering neural network, a looping 3D animation by rendering a texture map specific to the person in the digital image to the 3D animation sequence, the first 3D animation transition sequence, and the second 3D animation transition sequence.

17 . The non-transitory computer-readable medium of claim 16 , wherein the operations further comprise:
receiving a selection of a background image; sizing the looping 3D animation based on the selected background image; and displaying the sized looping 3D animation on the selected background image.

18 . The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise:
generating a second looping 3D animation including a second 3D animation sequence that differs from the 3D animation sequence; and displaying the looping 3D animation and the second looping 3D animation together on the selected background image.

19 . The non-transitory computer-readable medium of claim 16 , wherein receiving the selection of the 3D animation sequence comprises receiving a selection of a target motion type from a plurality of different motion types.

20 . The non-transitory computer-readable medium of claim 16 , wherein the operations further comprise rotating the looping 3D animation to present the looping 3D animation from different viewpoints in response to user input.

Assignees

• Adobe Inc

Inventors

• Yoon Jae Shin
• Aksit Duygu Ceylan
• Wang Yangtuanfeng
• Lu Jingwan
• Yang Jimei
• Shu Zhixin
• He Chengan
• Zhou Yi
• Saito Jun
• Zachary James

Classifications

• G06T2210/44

  Morphing · CPC title

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06T13/80 Primary

  Two-dimensional [2D] animation, e.g. using sprites · CPC title