Ideograms for captured expressions

Patent metadata

Field	Value
Publication number	US-10013601-B2
Application number	US-201414173447-A
Country	US
Kind code	B2
Filing date	Feb 5, 2014
Priority date	Feb 5, 2014
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

Particular embodiments of a method comprise analyzing an image to classify an expression displayed on a face shown in a captured image. Image analysis may include detecting the face in the image, generating a characterization of features of the face, and classifying the expression based on the characterization. The characterization of facial features may be based on benchmark metrics for a particular expression. One or more ideograms (e.g., written characters, symbols or images that represent an idea or thing) may be selected based on the expression. The selection may be based on one or more match scores for the expression. The match scores may be determined based on an ideogram dictionary or an ideogram usage history. The selected ideograms may then be presented, in association with the face, on a screen for the computing device.

First claim

What is claimed is:

1. A method comprising:
by a computing device, accessing a plurality of pre-generated ideograms, wherein each pre-generated ideogram corresponds to one or more expressions;
by the computing device, analyzing an image to identify, from a plurality of users, a particular user corresponding to a face shown in the image;
by the computing device, analyzing the image to classify at least one expression present on the face shown in the image, the analysis being based on a comparison between a set of facial-feature attributes detected on the face shown in the image and a set of benchmark metrics uniquely associated with the particular user identified as corresponding to the face shown in the image, wherein the benchmark metrics correspond to metrics of facial features for a particular expression;
by the computing device, selecting one or more of the plurality of pre-generated ideograms based on one or more match scores, wherein the match scores are calculated based on the classified at least one expression present on the face shown in the image and a usage history associated with the particular user; and
by the computing device, presenting, on a screen associated with the computing device, the selected one or more pre-generated ideograms,
wherein the usage history associated with the particular user comprises previous user-inputted selections of ideograms for one or more of the facial-feature attributes detected for one or more previously-classified images associated with the particular user.

2. The method of claim 1 , wherein the analyzing the image to classify the at least one expression present on the face comprises:
detecting the face in the image;
generating a characterization of features of the face; and
classifying the at least one expression based on the characterization.

3. The method of claim 2 , wherein the face is associated with a user, and wherein the generating the characterization of the features comprises:
characterizing attributes of the features with respect to benchmark metrics for a particular expression.

4. The method of claim 3 , wherein the benchmark metrics for the features of the face associated with the user comprise a characterization of attributes of the features when the particular expression is present on the face.

5. The method of claim 2 , wherein the face is associated with a user, and wherein the generating the characterization of the features comprises:
characterizing attributes of the features with respect to benchmark metrics for the features, wherein the benchmark metrics are stored in association with a user profile for the user.

6. The method of claim 1 , wherein the match scores are determined based on an ideogram dictionary or an ideogram usage history.

7. The method of claim 6 , wherein the ideogram dictionary or the ideogram usage history is associated with a user, associated with one or more social-networking connections of the user, associated with one or more users sharing common attributes with the user, or based on general statistics regarding trending pre-generated ideograms.

8. The method of claim 1 , wherein presenting the selected pre-generated ideograms in association with the face comprises providing a user interface including the selected pre-generated ideograms for insertion into a message.

9. The method of claim 1 , wherein presenting the selected pre-generated ideograms in association with the face comprises providing a user interface to post the selected pre-generated ideograms for annotation of a content item.

10. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
access a plurality of pre-generated ideograms, wherein each pre-generated ideogram corresponds to one or more expressions;
analyze an image to identify, from a plurality of users, a particular user corresponding to a face shown in the image;
analyze the image to classify at least one expression present on the face shown in the image, the analysis being based on a comparison between a set of facial-feature attributes detected on the face shown in the image and a set of benchmark metrics uniquely associated with the particular user identified as corresponding to the face shown in the image, wherein the benchmark metrics correspond to metrics of facial features for a particular expression;
select one or more of the plurality of pre-generated ideograms based on one or more match scores, wherein the match scores are calculated based on the classified at least one expression present on the face shown in the image and a usage history associated with the particular user; and
present, on a screen associated with the computing device, the selected one or more pre-generated ideograms,
wherein the usage history associated with the particular user comprises previous user-inputted selections of ideograms for one or more of the facial-feature attributes detected for one or more previously-classified images associated with the particular user.

11. The media of claim 10 , wherein the software operable when executed to analyze the image to classify the at least one expression present on the face comprises software operable when executed to:
detect the face in the image;
generate a characterization of features of the face; and
classify the at least one expression based on the characterization.

12. The media of claim 11 , wherein the face is associated with a user, and wherein the software operable when executed to generate the characterization of the features comprises software operable when executed to:
characterize attributes of the features with respect to benchmark metrics for a particular expression.

13. The media of claim 12 , wherein the benchmark metrics for the features of the face associated with the user comprise a characterization of attributes of the features when the particular expression is present on the face.

14. The media of claim 11 , wherein the face is associated with a user, and wherein the software operable when executed to generate the characterization of the features comprises software operable when executed to:
characterize attributes of the features with respect to benchmark metrics for the features, wherein the benchmark metrics are stored in association with a user profile for the user.

15. A system comprising:
one or more processors; and
a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:
access a plurality of pre-generated ideograms, wherein each pre-generated ideogram corresponds to one or more expressions;
analyze an image to identify, from a plurality of users, a particular user corresponding to a face shown in the image;
analyze the image to classify at least one expression present on the face shown in the image, the analysis being based on a comparison between a set of facial-feature attributes detected on the face shown in the image and a set of benchmark metrics uniquely associated with the particular user identified as corresponding to the face shown in the image, wherein the benchmark metrics correspond to metrics of facial features for a particular expression;
select one or more of the plurality of pre-generated ideograms based on one or more match scores, wherein the match scores are calculated based on the classified at least one expression present on the face shown in the image and a usage history associated with the particular user; and
present, on a screen associated with the computing device, the selected one or more pre-generated ideograms,

wherein the usage

Assignees

• Facebook Inc

Inventors

• Ebersman David
• Lessin Samuel
• Stocky Thomas
• Vernal Michael

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/04817

  using icons (graphical or visual programming using iconic symbols G06F8/34) · CPC title

• G06F2203/011

  Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns · CPC title