Method and device for generating a blended animation

Patent metadata

Field	Value
Publication number	US-11776192-B2
Application number	US-202318102177-A
Country	US
Kind code	B2
Filing date	Jan 27, 2023
Priority date	Jan 6, 2021
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

In one implementation, a method for generating a blended animation. The method includes: obtaining a motion input vector for a current time period; generating a motion output vector and pose information for the current time period based on the motion input vector; selecting an animated motion from a bank of animated motions for the current time period that matches the pose information within a threshold tolerance value; obtaining a blending coefficients vector for the current time period; generating a blended animation for the current time period by blending the motion output vector with the animated motion based on the blending coefficients vector; and generating a reward signal for the blended animation for the current time period.

First claim

What is claimed is:

1. A method comprising:
at a computing system including non-transitory memory and one or more processors, wherein the computing system is communicatively coupled to a display device and one or more input devices:
obtaining a motion input vector for a current time period;
generating a motion output vector and pose information for the current time period based on the motion input vector;
selecting an animated motion from a bank of animated motions for the current time period that matches the pose information within a threshold tolerance value;
generating a blended animation for the current time period by blending the motion output vector with the animated motion; and
presenting, via the display device, a virtual agent performing the blended animation.

2. The method of claim 1 , further comprising:
obtaining a blending coefficients vector for the current time period, wherein the blended animation is generated for the current time period by blending the motion output vector with the animated motion based on the blending coefficients vector.

3. The method of claim 2 , wherein the blending coefficients vector includes weights for the motion output vector and the animated motion.

4. The method of claim 2 , wherein the blending coefficients vector includes weights for the motion output vector and the animated motion for each actuatable component.

5. The method of claim 1 , wherein the motion output vector and the pose information for the current time period are generated via a machine learning system.

6. The method of claim 1 , wherein the motion input vector includes at least one of a task, an objective, a goal, a position information, or rotation information for particular extended reality (XR) content.

7. The method of claim 6 , wherein the particular XR content corresponds to one of the virtual agent or an animatable XR object.

8. The method of claim 1 wherein the display device corresponds to a transparent lens assembly, and wherein the particular XR content is projected onto the transparent lens assembly.

9. The method of claim 1 , wherein the display device corresponds to a near-eye system, and wherein presenting particular XR content includes compositing particular XR content with one or more images of a physical environment captured by an exterior-facing image sensor.

10. The method of claim 1 , wherein the motion output vector includes at least one of a change of position, a change of rotation, a change of velocity, or a change of acceleration for particular extended reality (XR) content.

11. The method of claim 1 , wherein the motion output vector includes at least one of a change of position, a change of rotation, a change of velocity, or a change of acceleration for actuatable components of particular extended reality (XR) content.

12. The method of claim 11 , wherein the actuatable components corresponds to one of a joint, a limb, or a body segment of the particular XR content.

13. The method of claim 1 , wherein the animated motion includes at least one of a fine-grained facial expression or a fine-grained body pose.

14. The method of claim 1 , wherein the blended animation is generated based on one of a linear blending technique, a spatial blending technique, a temporal blending technique, or a spatial-temporal blending technique.

15. The method of claim 1 , further comprising:
identifying at least one portion of the blended animation that exceeds a motion limit;
providing a feedback signal that identifies the at least one identified portion of the blended animation; and
adjusting one or more tunable parameters for a subsequent time period based on the feedback signal associated with the blended animation.

16. The method of claim 1 , further comprising:
identifying at least one portion of the blended animation that exceeds a motion limit;
providing a feedback signal that identifies the at least one identified portion of the blended animation; and
adjusting the at least one identified portion of the blended animation based on the feedback signal.

17. A device comprising:
one or more processors;
a non-transitory memory;
an interface for communicating with a display device and one or more input devices; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
obtain a motion input vector for a current time period;
generate a motion output vector and pose information for the current time period based on the motion input vector;
select an animated motion from a bank of animated motions for the current time period that matches the pose information within a threshold tolerance value;
generate a blended animation for the current time period by blending the motion output vector with the animated motion; and
present, via the display device, a virtual agent performing the blended animation.

18. The device of claim 17 , wherein the one or more programs further cause the device to:
obtain a blending coefficients vector for the current time period, wherein the blended animation is generated for the current time period by blending the motion output vector with the animated motion based on the blending coefficients vector.

19. The device of claim 17 , wherein the motion input vector includes at least one of a task, an objective, a goal, a position information, or rotation information for particular extended reality (XR) content.

20. The device of claim 19 , wherein the particular XR content corresponds to one of the virtual agent or an animatable XR object.

21. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device with an interface for communicating with a display device and one or more input devices, cause the device to:
obtain a motion input vector for a current time period;
generate a motion output vector and pose information for the current time period based on the motion input vector;
select an animated motion from a bank of animated motions for the current time period that matches the pose information within a threshold tolerance value;
generate a blended animation for the current time period by blending the motion output vector with the animated motion; and
present, via the display device, a virtual agent performing the blended animation.

22. The non-transitory memory of claim 21 , wherein the one or more programs further cause the device to:
obtain a blending coefficients vector for the current time period, wherein the blended animation is generated for the current time period by blending the motion output vector with the animated motion based on the blending coefficients vector.

23. The non-transitory memory of claim 21 , wherein the motion input vector includes at least one of a task, an objective, a goal, a position information, or rotation information for particular extended reality (XR) content.

24. The non-transitory memory of claim 23 , wherein the particular XR content corresponds to one of the virtual agent or an animatable XR object.

Assignees

• Apple Inc

Inventors

• Mahasseni Behrooz
• Manglik Aashi
• Drummond Mark
• Ahn Edward S
• Budhram Shaun
• Sivapurapu Siva Chandra Mouli

Classifications

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title