Determining graphical element(s) for inclusion in an electronic communication

Patent metadata

Field	Value
Publication number	US-10416846-B2
Application number	US-201615350040-A
Country	US
Kind code	B2
Filing date	Nov 12, 2016
Priority date	Nov 12, 2016
Publication date	Sep 17, 2019
Grant date	Sep 17, 2019

Abstract

Methods, apparatus, and computer readable media related to determining graphical element(s) (e.g., emojis, GIFs, stickers) for inclusion in an electronic communication being formulated by a user via a computing device of the user, and providing the graphical element(s) for inclusion in the electronic communication. For example, the graphical element(s) may be provided for presentation to the user via a display of the computing device of the user and, in response to user interface input directed to one of the graphical element(s), that graphical element may be incorporated in the electronic communication. In various implementations, the electronic communication is a communication to be submitted as part of a dialog that involves the user and one or more additional users.

First claim

What is claimed is:

1. A method implemented by one or more processors, comprising:
receiving an electronic communication submitted as part of a dialog that includes a user and one or more additional users, the electronic communication being based on user interface input generated by an additional user of the one or more additional users, via one or more user interface input devices of an additional user computing device of the additional user;
determining at least one communication feature of the electronic communication, wherein the at least one communication feature is based on one or more terms of a plurality of terms of the electronic communication;
determining a first current state feature that indicates a first current state of the user and a second current state feature that indicates a second current state of the additional user, wherein the first current state includes a first sentiment of the user and the second current state includes a second sentiment of the additional user, wherein determining the first current state feature and the second current state feature includes determining a magnitude of the first sentiment and a magnitude of the second sentiment;
determining a combined current state feature based on the first current state feature and the second current state feature, wherein the combined current state feature is based on the magnitude of the first sentiment and the magnitude of the second sentiment; and
prior to receipt of textual input provided by the user for a subsequent electronic communication of the user to be submitted as part of the dialog:
determining a group of one or more graphical elements for inclusion in the subsequent electronic communication, wherein determining the group of one or more graphical elements comprises:
selecting, from candidate graphical elements, at least one graphical element to include in the group, wherein selecting the at least one graphical element is based on the at least one communication feature of the electronic communication and on the combined current state feature; and
providing the group of one or more graphical elements for potential inclusion in the subsequent electronic communication, wherein the group of one or more graphical elements is provided for presentation via a display of a user computing device of the user.

2. The method of claim 1 , further comprising:
identifying user interface interaction with a user interface element, the user interface element graphically presented to the user via the display in combination with a presentation of the electronic communication;
wherein providing the group of one or more graphical elements is in response to identifying the user interface interaction with the user interface element.

3. The method of claim 1 , wherein the at least one communication feature includes at least one of: text occurring in a particular portion of the electronic communication, a time the electronic communication was sent or received, a number of user participants in the dialog, or labels applied to text of the electronic communication.

4. The method of claim 1 , wherein the at least one communication feature further comprises at least one non-textual feature that is based at least in part on one or more signals that are in addition to the terms of the electronic communication.

5. The method of claim 1 , further comprising:
determining at least one additional communication feature of an additional electronic communication of the dialog;
wherein selecting the at least one graphical element is further based on the at least one additional communication feature.

6. The method of claim 1 , further comprising:
identifying a historical usage feature for the at least one graphical element, wherein the historical usage feature includes a quantity or frequency of past usage by the user of the at least one graphical element and of other graphical elements sharing one or more graphical element features with the at least one graphical element, wherein the one or more graphical element features include at least one of: a descriptor of content in the at least one graphical element, an identifier of an entity associated with the at least one graphical element, a term of a prior dialogue in which the at least one graphical element was included, or a user state associated with the at least one graphical element;
wherein selecting the at least one graphical element is further based on the historical usage feature.

7. The method of claim 1 , wherein selecting the at least one graphical element based on the at least one communication feature comprises:
determining a graphical element feature based on the at least one communication feature, the graphical element feature being assigned to the at least one graphical element and being assigned to a plurality of other graphical elements, wherein the graphical element feature includes at least one descriptor of content included in the at least one graphical element; and
selecting the at least one graphical element to include in the group based on the at least one graphical element being associated with a greater degree of historical usage, by the user, than other graphical elements.

8. The method of claim 7 , wherein the at least one graphical element is included in a cohesive pack of graphical elements and wherein the historical usage of the at least one graphical element is based on the historical usage, by the user, of the graphical elements of the cohesive pack.

9. The method of claim 1 , wherein selecting the at least one graphical element based on the at least one communication feature comprises:
applying input to a trained machine learning model stored in one or more computer readable media, the input comprising the at least one communication feature;
generating, over the model and based on the input, output that indicates one or more graphical element features; and
selecting the at least one graphical element based on the one or more graphical element features being assigned to the at least one graphical element.

10. The method of claim 9 , wherein the output that indicates one or more graphical element features includes one or more terms and wherein selecting the at least one graphical element based on the one or more graphical element features being assigned to the graphical element comprises selecting the at least one graphical element based on it being indexed by at least one of the one or more terms.

11. The method of claim 9 , further comprising:
generating the trained machine learning model based on a plurality of training examples derived from past user dialogs, each of the training examples including:
training example input based on communication features of a corresponding original electronic communication, and
training example output based on text of a corresponding reply electronic communication that is a reply to the corresponding original electronic communication;

wherein generating the trained machine learning model comprises training the trained machine learning model based on application of the training example input of the training examples and backpropagation based on the training example output of the training examples.

12. The method of claim 9 , further comprising determining at least one third current state feature that indicates a third current state of the user or the additional user, wherein the third current state includes a physical location of a user computing device of the user or of the additional user.

13. The method of claim 1 , wherein the at least one communication feature includes the one or more terms and wherein selecting the at least one graphical element based on the at leas

Assignees

• Google Llc

Inventors

• Cassidy Ryan
• Srivastava Utkarsh
• Volkov Anton
• Mcgregor Jr John Patrick

Classifications

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G06F3/04895

  Guidance during keyboard input operation, e.g. prompting · CPC title

• G06F3/0237

  using prediction or retrieval techniques · CPC title

• G06F40/274

  Converting codes to words; Guess-ahead of partial word inputs · CPC title

• G06F3/04886

  by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus · CPC title