Verifying accuracy of ML pipelines using third party co-ordination

Patent metadata

Field	Value
Publication number	US-11070575-B2
Application number	US-201916293937-A
Country	US
Kind code	B2
Filing date	Mar 6, 2019
Priority date	Mar 6, 2019
Publication date	Jul 20, 2021
Grant date	Jul 20, 2021

Abstract

Systems, methods, computer-readable media, and devices are disclosed for verifying traffic classification. At a first node, a classification to a received packet is designated according to a local model. The classification of the packet by the first node is verified by sending packet information describing the packet to a distributed network comprising multiple nodes, where the packet information includes attributes of the packet. The classification of the packet is verified from receiving results from a second node that, based on the attributes, independently classifies the packet. Based on the verified classification, decentralized information for classifying packets is updated.

First claim

What is claimed is:

1. A method for verifying traffic classification comprising:
designating, at a first node, a classification to a received packet according to a local model; and
verifying the classification of the packet by the first node by:
sending packet information describing the packet to a distributed network comprising multiple nodes, wherein the packet information comprises attributes of the packet;
verifying the classification of the packet from receiving results from a second node that, based on the attributes, independently classifies the packet; and
updating decentralized information for classifying packets based on the verified traffic classification.

2. The method of claim 1 , wherein verifying the classification of the packet further comprises:
receiving results from a subset of nodes of the distributed network classifying the packet, the classification according to models specific to each node;
cross-verifying the classification among the subset of nodes by matching the results; and
based on a same result from a threshold number of nodes of the subset of the nodes, confirming the classification of the packet.

3. The method of claim 1 , further comprising appending the updated decentralized information to a ledger on the distributed network, wherein the updated decentralized information comprises the verified traffic classification and the attributes of the packet.

4. The method of claim 1 , wherein the attributes of the packet are associated with a threat pattern extracted from a portion of traffic, the threat pattern extracted based on statistically based patterns describing malware.

5. The method of claim 1 , wherein the classification is verified by at least one validating node, the at least one validating node having a reputation score above a threshold value.

6. The method of claim 1 , wherein a node within the distributed network having a reputation score below a threshold value cannot verify the classification, and wherein the node having the reputation score below the threshold is restricted from modifying the decentralized information.

7. The method of claim 1 , wherein receiving results from a second node comprises receiving an independent classification of the packet, using the attributes, based on a second model providing a unique solution from the local model.

8. A system for verifying traffic classification comprising:
a server to verify traffic classification within a network, the server to:
designate, at a first node, a classification to a received packet according to a local model; and
verify the classification of the packet by the first node by:
sending packet information describing the packet to a distributed network comprising multiple nodes, wherein the packet information comprises attributes of the packet;
verifying the classification of the packet from receiving results from a second node that, based on the attributes, independently classifies the packet; and
updating decentralized information for classifying packets based on the verified traffic classification.

9. The system of claim 8 , wherein verifying the classification of the packet further comprises the server to:
receive results from a subset of nodes of the distributed network classifying the packet, the classification according to models specific to each node;
cross-verify the classification among the subset of nodes by matching the results; and
based on a same result from a threshold number of nodes of the subset of the nodes, confirm the classification of the packet.

10. The system of claim 8 , the server further to:
append the updated decentralized information to a ledger on the distributed network, wherein the updated decentralized information comprises the verified traffic classification and the attributes of the packet.

11. The system of claim 8 , wherein the attributes of the packet are associated with a threat pattern extracted from a portion of traffic, the threat pattern extracted based on statistically based patterns describing malware.

12. The system of claim 8 , wherein the classification is verified by at least one validating node, the at least one validating node having a reputation score above a threshold value.

13. The system of claim 8 , wherein a node within the distributed network having a reputation score below a threshold value cannot verify the classification, and wherein the node having the reputation score below the threshold is restricted from modifying the decentralized information.

14. The system of claim 8 , wherein receiving results from a second node comprises receiving an independent classification of the packet, using the attributes, based on a second model providing a unique solution from the local model.

15. A non-transitory computer-readable medium comprising instructions stored thereon, the instructions for verifying traffic classification and executable by one or more processors of a computing system to cause the computing system to:
designate, at a first node, a classification to a received packet according to a local model; and
verify the classification of the packet by the first node by:
sending packet information describing the packet to a distributed network comprising multiple nodes, wherein the packet information comprises attributes of the packet;
verifying the classification of the packet from receiving results from a second node that, based on the attributes, independently classifies the packet; and
updating decentralized information for classifying packets based on the verified traffic classification.

16. The non-transitory computer-readable medium of claim 15 , wherein verifying the classification of the packet further comprises instructions further causing the computing system to:
receive results from a subset of nodes of the distributed network classifying the packet, the classification according to models specific to each node;
cross-verify the classification among the subset of nodes by matching the results; and
based on a same result from a threshold number of nodes of the subset of the nodes, confirm the classification of the packet.

17. The non-transitory computer-readable medium of claim 15 , the instructions further causing the computing system to:
append the updated decentralized information to a ledger on the distributed network, wherein the updated decentralized information comprises the verified traffic classification and the attributes of the packet.

18. The non-transitory computer-readable medium of claim 15 , wherein the attributes of the packet are associated with a threat pattern extracted from a portion of traffic, the threat pattern extracted based on statistically based patterns describing malware.

19. The non-transitory computer-readable medium of claim 15 , wherein the classification is verified by at least one validating node, the at least one validating node having a reputation score above a threshold value.

20. The non-transitory computer-readable medium of claim 15 , wherein a node within the distributed network having a reputation score below a threshold value cannot verify the classification, and wherein the node having the reputation score below the threshold is restricted from modifying the decentralized information.

Assignees

• Cisco Tech Inc

Inventors

• Ravindranath Ram Mohan
• Patil Prashanth
• Viswambharan Rajesh Indira

Classifications

• H04L63/1425 Primary

  Traffic logging, e.g. anomaly detection · CPC title

• H04L63/145

  the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms · CPC title

• H04L69/22

  Parsing or analysis of headers · CPC title

• H04L41/16

  using machine learning or artificial intelligence · CPC title

• H04L63/123

  received data contents, e.g. message integrity · CPC title