Communication using interactive avatars

Patent metadata

Field	Value
Publication number	US-9386268-B2
Application number	US-201213996009-A
Country	US
Kind code	B2
Filing date	Apr 9, 2012
Priority date	Apr 9, 2012
Publication date	Jul 5, 2016
Grant date	Jul 5, 2016

Abstract

Generally this disclosure describes a video communication system that replaces actual live images of the participating users with animated avatars. A method may include selecting an avatar; initiating communication; detecting a user input; identifying the user input; identifying an animation command based on the user input; generating avatar parameters; and transmitting at least one of the animation command and the avatar parameters.

First claim

What is claimed:

1. A system, comprising:
a user input device configured to capture a user input;
communication circuitry configured to transmit and receive information;
a microphone configured to capture sound and convert the captured sound into a corresponding audio signal; and
one or more non-transitory storage memories having stored thereon, individually or in combination, instructions that when executed by one or more processors result in the following operations comprising:
selecting an avatar;
receiving at least one image of a user;
passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;
detecting a user input with said user input device, said user input comprising at least one of a touch and a gesture;
determining one or more animation commands associated with a user input identifier corresponding to a detected user input;
determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;
modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar;
transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device;
capturing user speech and converting the user speech into a corresponding user speech signal;
transforming the user speech signal into an avatar speech signal; and
transmitting the avatar speech signal to the remote device.

2. The system of claim 1 , further comprising a camera configured to capture images, wherein the instructions that when executed by one or more processors result in the following additional operations:
capturing the at least one image of the user;
performing facial detection on said at least one image to detect a face in the image;
extracting features from the face; and
passively animating the avatar based at least in part on extracted features from said face, such that said passively animated avatar mimics motion of at least a portion of said face.

3. The system of claim 1 , further comprising a display, wherein the instructions that when executed by one or more processors result in the following additional operations:
displaying said avatar;
receiving at least one of a remote animation command and remote avatar parameters; and
passively animating said avatar at least in part based on at least one of the remote animation command and the remote avatar parameters.

4. The system of claim 1 , further comprising a speaker configured to convert an audio signal into sound, wherein the instructions that when executed by one or more processors result in the following additional operations:
receiving a remote avatar speech signal; and
converting the remote avatar speech signal into avatar speech.

5. The system of claim 1 , wherein:
the user input device is a depth camera; and
the user input is a gesture detected by said depth camera.

6. The system of claim 1 , wherein:
the user input device is a touch-sensitive display;
the user input is a touch event; and
said touch event comprises at least one of a touch type and a touch location.

7. The system of claim 1 , wherein the transforming comprises at least one of pitch shifting and time stretching.

8. A method, comprising:
selecting an avatar;
receiving at least one image of a user;
passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;
detecting a user input with a user input device, said user input comprising at least one of a touch and a gesture;
determining one or more animation commands associated with a user input identifier corresponding to a detected user input;
determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;
modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar;
transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device;
capturing user speech and converting the user speech into a corresponding user speech signal;
transforming the user speech signal into an avatar speech signal; and
transmitting the avatar speech signal to the remote device.

9. The method of claim 8 , further comprising:
capturing the at least one image of the user;
performing facial detection on said at least one image to detect a face in the image;
extracting features from the face; and
passively animating the avatar based at least in part on extracted features from said face, such that said passively animated avatar mimics motion of at least a portion of said face.

10. The method of claim 8 , further comprising:
displaying said avatar;
receiving at least one of a remote animation command and remote avatar parameters; and
passively animating said avatar at least in part based on at least one of the remote animation command and the remote avatar parameters.

11. The method of claim 8 , further comprising:
receiving a remote avatar speech signal; and
converting the remote avatar speech signal into avatar speech.

12. The method of claim 8 , wherein:
the user input device is a depth camera; and
the user input is a gesture detected by said depth camera.

13. The method of claim 8 , wherein:
The user input device is a touch-sensitive display;
the user input is a touch event; and
the touch event comprises at least one of a touch type and a touch location.

14. The method of claim 8 , wherein the transforming comprises at least one of pitch shifting and time stretching.

15. A system comprising one or more non-transitory storage memories having stored thereon, individually or in combination, instructions that when executed by one or more processors result in the following operations comprising:
selecting an avatar;
receiving at least one image of a user;
passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;
detecting a user input, said user input comprising at least one of a touch and a gesture;
determining one or more animation commands associated with a user input identifier corresponding to a detected user input;
determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;
modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar;
transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device;
capturing user speech and converting the user speech into a corresponding user speech signal;
transforming the user speech signal into an avatar speech signal; and
transmitting the avatar speech signal to the remote devic

Assignees

• Tong Xiaofeng
• Li Wenlong
• Du Yangzhou
• Hu Wei
• Zhang Yimin
• Intel Corp

Inventors

• Tong Xiaofeng
• Li Wenlong
• Du Yangzhou
• Hu Wei
• Zhang Yimin

Classifications

• H04N7/157 Primary

  defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827) · CPC title

• G10L21/013

  Adapting to target pitch · CPC title

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title

• H04N7/147 Primary

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• G10L21/003

  Changing voice quality, e.g. pitch or formants · CPC title