Multi-Modal Models for Detecting Malicious Emails

Patent metadata

Field	Value
Publication number	US-2025317461-A1
Application number	US-202519206800-A
Country	US
Kind code	A1
Filing date	May 13, 2025
Priority date	Mar 28, 2023
Publication date	Oct 9, 2025
Grant date	—

Abstract

In some aspects, the techniques described herein relate to a method for detecting malicious emails, the method including: receiving an email, wherein the email is associated with a markup payload; determining, based on the markup payload, text data associated with the email; determining, using the text data and a first machine learning model, a first representation of the email representing text associated with the email; rendering the email to generate image data that represents a rendering of the email; determining, using the image data and a second machine learning model, a second representation of the email that represents at least the rendering of the email; and determining a prediction for the email based on the first representation and the second representation, wherein the prediction represents whether the email is predicted to be malicious based on the first representation and the second representation.

First claim

What is claimed is:

1 . A method for detecting malicious emails, the method comprising:
receiving an email; rendering the email to generate image data that represents a rendering of the email; determining, using the image data and a machine learning model, a representation of the email that represents at least the rendering of the email; and determining a prediction for the email based on the representation, wherein the prediction represents whether the email is predicted to be malicious based on the representation.

2 . The method of claim 1 , wherein the machine learning model comprises a convolutional neural network layer.

3 . The method of claim 1 , wherein:
the method further comprises:
determining second image data based on an attached image file associated with the email,
providing the second image data to the machine learning model, and
receiving, from the machine learning model, a second representation of the email; and
the prediction is determined based on the representation and the second representation.

4 . The method of claim 1 , wherein:
the method further comprises:
determining second image data based on an image associated with a uniform resource locator (URL) included in the email,
providing the second image data to the machine learning model, and
receiving, from the machine learning model, a second representation of the email; and
the prediction is determined based on the representation and the second representation.

5 . The method of claim 1 , wherein:
the method further comprises:
determining, based on a markup payload associated with the email, text data associated with the email; and
determining, using the text data and a second machine learning model, a second representation; and
determining the prediction is based on the representation and the second representation.

6 . The method of claim 5 , wherein determining the prediction comprises:
providing the representation and the second representation to a third machine learning model, wherein the third machine learning model is configured to determine the prediction based on the representation and the second representation; and receiving the prediction from the third machine learning model.

7 . The method of claim 5 , wherein the second machine learning model comprises an attention-based text encoder layer.

8 . The method of claim 5 , wherein:
the method further comprises:
determining second text data based on content data associated with a uniform resource locator (URL) included in the email,
providing the second text data to the second machine learning model, and
receiving, from the second machine learning model, a third representation of the email; and
the prediction is determined based on the representation, the second representation, and the third representation.

9 . A system comprising:
one or more processors; and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving an email; rendering the email to generate image data that represents a rendering of the email; determining, using the image data and a machine learning model, a representation of the email that represents at least the rendering of the email; and determining a prediction for the email based on the representation, wherein the prediction represents whether the email is predicted to be malicious based on the representation.

10 . The system of claim 9 , wherein the machine learning model comprises a convolutional neural network layer.

11 . The system of claim 9 , wherein:
the operations further comprise:
determining second image data based on an attached image file associated with the email,
providing the second image data to the machine learning model, and
receiving, from the machine learning model, a second representation of the email; and
the prediction is determined based on the representation and the second representation.

12 . The system of claim 9 , wherein:
the operations further comprise:
determining second image data based on an image associated with a uniform resource locator (URL) included in the email,
providing the second image data to the machine learning model, and
receiving, from the machine learning model, a second representation of the email; and
the prediction is determined based on the representation and the second representation.

13 . The system of claim 9 , wherein:
the operations further comprise:
determining, based on a markup payload associated with the email, text data associated with the email; and
determining, using the text data and a second machine learning model, a second representation; and
determining the prediction is based on the representation and the second representation.

14 . The system of claim 13 , wherein determining the prediction comprises:
providing the representation and the second representation to a third machine learning model, wherein the third machine learning model is configured to determine the prediction based on the representation and the second representation; and receiving the prediction from the third machine learning model.

15 . The system of claim 13 , wherein the second machine learning model comprises an attention-based text encoder layer.

16 . The system of claim 13 , wherein:
the operations further comprise:
determining second text data based on content data associated with a uniform resource locator (URL) included in the email,
providing the second text data to the second machine learning model, and
receiving, from the second machine learning model, a third representation of the email; and
the prediction is determined based on the representation, the second representation, and the third representation.

17 . One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving an email; rendering the email to generate image data that represents a rendering of the email; determining, using the image data and a machine learning model, a representation of the email that represents at least the rendering of the email; and determining a prediction for the email based on the representation, wherein the prediction represents whether the email is predicted to be malicious based on the representation.

18 . The one or more non-transitory computer-readable media of claim 17 , wherein the machine learning model comprises a convolutional neural network layer.

19 . The one or more non-transitory computer-readable media of claim 17 , wherein:
the operations further comprise:
determining second image data based on an image associated with a uniform resource locator (URL) included in the email,
providing the second image data to the machine learning model, and
receiving, from the machine learning model, a second representation of the email; and
the prediction is determined based on the representation and the second representation.

20 . The one or more non-transitory computer-readable media of claim 17 , wherein:
the operations further comprise:
determining, based on a markup payload associated with the email, text data associated with the email; and
determining, using the text data and a second machine learning model, a second representation; and
determining the prediction

Assignees

• Cisco Tech Inc

Inventors

• Brabec Jan
• Starosta Radek

Classifications

• H04L63/1416

  Event detection, e.g. attack signature detection · CPC title

• H04L63/1441

  Countermeasures against malicious traffic (countermeasures against attacks on cryptographic mechanisms H04L9/002) · CPC title

• G06V10/82

  using neural networks · CPC title

• H04L63/1483

  service impersonation, e.g. phishing, pharming or web spoofing (detection of rogue wireless access points H04W12/12) · CPC title

• H04L63/1425 Primary

  Traffic logging, e.g. anomaly detection · CPC title