Methods and apparatus to visualize machine learning based malware classification

Patent metadata

Field	Value
Publication number	US-12483590-B2
Application number	US-202217714023-A
Country	US
Kind code	B2
Filing date	Apr 5, 2022
Priority date	Apr 5, 2021
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed. An example apparatus includes at least one memory, instructions, and processor circuitry to execute the instructions. The processor circuitry executes the instructions to identify a test data distribution, generate a first visualization of the identified test data distribution, select a visualization type for a machine learning model, generate a second visualization including an indication of features extracted from the test data by the machine learning model, and generate a third visualization of results of inference performed by the machine learning model, the inference performed on the test data.

First claim

What is claimed is:

1 . An apparatus comprising:
at least one memory; machine-readable instructions; and processor circuitry to execute the machine-readable instructions to:
identify a distribution associated with test data;
generate a first visualization of the identified distribution;
generate a recommendation to adjust a machine learning model, the recommendation based on a classification error, the recommendation to be displayed with the first visualization, the recommendation to include an instruction to reduce a malware classification error of the machine learning model;
select a visualization type for the machine learning model;
generate a second visualization including an indication of features extracted from the test data by the machine learning model, the second visualization generated using gradient weighted class activation mapping t-distributed stochastic neighbor embedding feature projections; and
generate a third visualization of results of an inference performed by the machine learning model, the inference performed on the test data, wherein the inference is at least one of a classification of a sample as malware or a classification of the sample as benign.

2 . The apparatus of claim 1 , wherein the processor circuitry is to generate a recommendation to improve inference accuracy based on a comparison of at least two of: the first visualization, the second visualization, or the third visualization.

3 . The apparatus of claim 1 , wherein the processor circuitry is to generate a fourth visualization of a machine learning pipeline, the fourth visualization including the first visualization, the second visualization, and the third visualization.

4 . The apparatus of claim 1 , wherein at least one of the first visualization, the second visualization, and the third visualization includes an indication that a data is an outlier data.

5 . The apparatus of claim 1 , wherein the third visualization includes a receiver operating characteristic curve and an indication of a partial area under the receiver operating characteristic curve.

6 . A non-transitory computer readable medium comprising instructions which, when executed, cause processor circuitry to:
identify a distribution associated with test data; generate a first visualization of the identified distribution; generate a recommendation to adjust a machine learning model, the recommendation based on a classification error, the recommendation to be displayed with the first visualization, the recommendation to include an instruction to reduce a malware classification error of the machine learning model; select a visualization type for the machine learning model; generate a second visualization including an indication of features extracted from the test data by the machine learning model, the second visualization generated using t-distributed stochastic neighbor embedding feature projections; and generate a third visualization of results of an inference performed by the machine learning model, the inference performed on the test data, wherein the inference is at least one of a classification of a sample as malware or a classification of the sample as benign.

7 . The non-transitory computer readable medium of claim 6 , wherein the instructions, when executed, cause the processor circuitry to generate a recommendation to improve inference accuracy based on a comparison of at least two of: the first visualization, the second visualization, or the third visualization.

8 . The non-transitory computer readable medium of claim 6 , wherein the instructions, when executed, cause the processor circuitry to generate a fourth visualization of a machine learning pipeline, the fourth visualization including the first visualization, the second visualization, and the third visualization.

9 . The non-transitory computer readable medium of claim 6 , wherein at least one of the first visualization, the second visualization, and the third visualization includes an indication that a data is an outlier data.

10 . The non-transitory computer readable medium of claim 6 , wherein the third visualization includes a receiver operating characteristic curve and an indication of a partial area under the receiver operating characteristic curve.

11 . A method for error analysis in a machine learning pipeline, the method comprising:
identifying, by executing an instruction with processor circuitry, a distribution associated with test data; generating, by executing an instruction with the processor circuitry, a first visualization of the identified distribution; generating, by executing an instruction with the processor circuitry, a recommendation to adjust a machine learning model, the recommendation based on a classification error, the recommendation to be displayed with the first visualization, the recommendation to include an instruction to reduce a malware classification error of the machine learning model; selecting, by executing an instruction with processor circuitry, a visualization type for the machine learning model; generating, by executing an instruction with processor circuitry, a second visualization including an indication of features extracted from the test data by the machine learning model, the second visualization generated using t-distributed stochastic neighbor embedding feature projections; and generating, by executing an instruction with processor circuitry, a third visualization of results of an inference performed by the machine learning model, the inference performed on the test data, wherein the inference is at least one of a classification of a sample as malware or a classification of the sample as benign.

12 . The method of claim 11 , further including generating a recommendation to improve inference accuracy based on a comparison of at least two of: the first visualization, the second visualization, and the third visualization.

13 . The method of claim 11 , further including generating a fourth visualization of the machine learning pipeline, the fourth visualization including the first visualization, the second visualization, and the third visualization.

14 . The method of claim 11 , wherein at least one of the first visualization, the second visualization, and the third visualization includes an indication that a data is an outlier data.

15 . The method of claim 11 , wherein the third visualization includes a receiver operating characteristic curve and an indication of a partial area under the receiver operating characteristic curve.

Assignees

• Mcafee Llc

Inventors

• Huang Yonghong
• Grobman Steven
• King Jonathan

Classifications

• H04L63/1408

  by monitoring network traffic (monitoring network traffic per se H04L43/00) · CPC title

• H04L63/0227

  Filtering policies (mail message filtering H04L51/212) · CPC title

• H04L41/16 Primary

  using machine learning or artificial intelligence · CPC title

• H04L63/20

  for managing network security; network security policies in general (filtering policies H04L63/0227) · CPC title

• H04L63/145 Primary

  the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms · CPC title