Method and system for classifying man vs. machine generated e-mail

Patent metadata

Field	Value
Publication number	US-10778618-B2
Application number	US-201414150788-A
Country	US
Kind code	B2
Filing date	Jan 9, 2014
Priority date	Jan 9, 2014
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

A computer system, computer program product, and computer-implemented method for communicating electronic messages over a communication network coupled thereto are provided. The computer system comprises a network interface for receiving messages sent over the network and addressed to a user of the computer system; and computer executable electronic message processing software. The software comprises instructions for directing the computer system to receive a message over the network, and to identify whether a sender of the received electronic message is a human or a machine. The identifying includes first and second phases of operation. The first phase includes an offline phase employing information and activities resident on the computer system. The second phase includes an online phase employing resources remotely accessible over the network. The software further includes instructions for providing the user with the results of the identification as human or machine; and for performing automatic data extraction from the message if the message was identified to be from a machine.

First claim

What is claimed is:

1. A computer system for communicating electronic messages over a communication network coupled thereto, the computer system comprising:
a network interface for receiving new messages sent over the network and addressed to a user of the computer system; and
computer executable electronic message processing software comprising instructions for directing the computer system to:
execute an offline phase to categorize a plurality of uncategorized historical messages stored on the computer system, the offline phase comprising:
retrieving the historical messages from a local memory of the computer system;
retrieving resource information from the local memory, the resource information including a library of network addresses associated with known senders, and
categorizing the historical messages as generated by an automated process based on determining whether a network address of a respective historical messages is included within the library of known network addresses;

train a classifier using the categorized historical messages, the classifier configured to generate a quantitative value representing a probability that a new message is generated by the automated process;
receive a message over the network;
identify whether a received electronic message is generated by the automated process, the identifying comprising:
transmitting an identity of the sender to a remote network entity and receiving a remote classification of the sender from the remote network entity,
classifying the received electronic message using the classifier, the classifying generating a quantitative prediction that the received electronic message is generated by the automated process, and
identifying whether the sender is the automated process based on the remote classification and the classifying using the classifier;

provide the user with the results of the identification and corresponding information of the message, the providing comprising:
determining one or more types of the message based on one or more characteristics in the message, the one or more characteristics including a keyword count within a leading sentence of the message;
automatically extracting information from the message based on the determined one or more types of the message; and
sending the user the extracted information.

2. A computer system as recited in claim 1 , wherein the offline phase further includes identifying, within the body of the message, at least one of spelling errors, grammatical errors, slang expressions, and references to topical and popular cultural elements.

3. A computer system as recited in claim 1 , wherein the offline phase further includes identifying, within the body of the message, personal and formal occurrences of the user's name.

4. A computer system as recited in claim 1 , wherein:
the software further comprises results of machine-learning training performed on a set of past messages known to be generated by the automated process; and
the offline phase includes applying machine learning, in accordance with the results of the training, to determine whether the messages originated with the automated process.

5. A computer system as recited in claim 1 , wherein the online phase includes weight classification as to a number of messages sent by the sender, wherein, if the number of messages exceeds a threshold, then the message is deemed to be from the automated process.

6. A computer system as recited in claim 1 , wherein the online phase further includes referring to a map of identified senders, correlated with whether each respective sender is the automated process.

7. A computer program product for directing a computer system which communicates electronic messages over a communication network coupled thereto, the computer program product comprising:
a non-transitory computer-readable medium; and
software program code provided on the computer-readable medium, for directing the computer system to perform the actions of:
executing an offline phase to categorize a plurality of historical messages stored on the computer system, the offline phase comprising:
retrieving the historical messages from a local memory of the computer system;
retrieving resource information from the local memory, the resource information including a library of network addresses associated with known senders, and
categorizing the historical messages as generated by an automated process based on determining whether a network address of a respective historical messages is included within the library of known network addresses;

training a classifier using the categorized historical messages, the classifier configured to generate a quantitative value representing a probability that a new message is generated by the automated process;
receiving a message that was sent over the network through a network interface of the computer system, the message being addressed to a user of the computer system;
identifying whether a received electronic message is generated by the automated process, the identifying comprising:
transmitting an identity of the sender to a remote network entity and receiving a remote classification of the sender from the remote network entity,
classifying the received electronic message using the classifier, the classifying generating a quantitative prediction that the received electronic message is generated by the automated process, and
identifying whether the sender is the automated process based on the remote classification and the classifying using the classifier;

providing the user with the results of the identification and corresponding information of the message, the providing comprising:
determining one or more types of the message based on one or more characteristics in the message, the one or more characteristics including a keyword count within a leading sentence of the message;
automatically extracting information from the message based on the determined one or more types of the message; and
sending the user the extracted information.

8. A computer program product as recited in claim 7 , wherein the offline phase further includes identifying, within the body of the message, at least one of spelling errors, grammatical errors, slang expressions, and references to topical and popular cultural elements.

9. A computer program product as recited in claim 7 , wherein the offline phase further includes identifying, within the body of the message, personal and formal occurrences of the user's name.

10. A computer program product as recited in claim 7 , wherein:
the software program code further comprises results of machine-learning training performed on a set of past messages known to be generated by the automated process; and
the offline phase includes applying machine learning, in accordance with the results of the training, to determine whether the messages originated with the automated process.

11. A computer program product as recited in claim 7 , wherein the online phase includes weight classification as to a number of messages sent by the sender, wherein, if the number of messages exceeds a threshold, then the message is deemed to be from the automated process.

12. A computer program product as recited in claim 7 , wherein the online phase further includes referring to a map of identified senders, correlated with whether each respective sender is the automated process.

13. A computer-implemented method for directing a computer system which communicates electronic messages over a communication network coupled thereto, the method comprising:
executing an offline phase to categorize a plurality of historical messages stored on

Assignees

• Oath Inc

Inventors

• Karnin Zohar
• Halawi Guy
• Wajc David
• Liberty Edo

Classifications

• G06N20/00

  Machine learning · CPC title

• H04L51/02

  using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages · CPC title

• H04L51/046 Primary

  Interoperability with other network applications or services · CPC title