Detecting encrypted tunneling traffic

Patent metadata

Field	Value
Publication number	US-9843593-B2
Application number	US-201414476672-A
Country	US
Kind code	B2
Filing date	Sep 3, 2014
Priority date	Aug 31, 2011
Publication date	Dec 12, 2017
Grant date	Dec 12, 2017

Abstract

Techniques for detecting encrypted tunneling traffic are disclosed. In some embodiments, detecting encrypted tunneling traffic includes monitoring encrypted network communications between a client and a remote server, in which the encrypted network communications are encrypted using a first protocol (e.g., Secure Shell (SSH) protocol or another protocol for encrypted network communications); and determining if the client sends a request to create a tunnel using the first protocol with the remote server. In some embodiments, detecting encrypted tunneling traffic further includes performing an action in response to determining that the client sent a request to create a tunnel using the first protocol with the remote server.

First claim

What is claimed is:

1. A network device for monitoring network communications, comprising:
a processor; and
a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
monitor encrypted network communications between a client and a remote server;
decrypt encrypted session traffic between the client and remote server;
monitor, within the decrypted encrypted session traffic, for a request from the client to create a tunnel using a first protocol with the remote server by applying a trusted man-in-the-middle technique using a self-signed certificate to inspect monitored traffic between the client and the remote server to facilitate deep packet inspection of the encrypted session traffic between the client and remote server, comprising to:
determine, within the decrypted encrypted session traffic, if the client sends the request to create the tunnel using the first protocol with the remote server; and
in response to a determination that the client sent the request to create the tunnel using the first protocol with the remote server:
intercept the request to establish the tunnel using the first protocol with the remote server, wherein the request is found within the decrypted encrypted session traffic;
block, after the intercepting of the request, the request to create the tunnel; and
send a response to the client, the response informing the client that tunneling using the first protocol is not supported by the remote server;

perform a traffic analysis of the decrypted encrypted session traffic between the client and remote server, comprising to:
identify an application generating the monitored traffic, comprising to determine what type of traffic a session involves, the type of traffic the session involves includes Hypertext Transfer Protocol (HTTP) traffic, File Transfer Protocol (FTP) traffic, Secure Sockets Layer (SSL) traffic, Secure Shell (SSH) traffic, a Domain Name System (DNS) request, unclassified application traffic, or any combination thereof;
identify a user generating the monitored traffic, comprising to determine a source IP of the monitored traffic; and
identify content relating to the monitored traffic, comprising to determine peer-to-peer activities, social networking activities, web browsing on certain prohibited web sites, streaming music, streaming video, use of unauthorized protocols, use of unauthorized applications, or any combination thereof;

determine whether a firewall policy is violated based on the traffic analysis of the encrypted session traffic between the client and remote server; and
send a message to a cloud security service if the deep packet inspection determined that the client is using the encrypted tunnel to evade a firewall policy, wherein the message includes identifying information associated with the remote server.

2. The network device recited in claim 1 , wherein the first protocol is a Secure Sockets Layer (SSL) protocol.

3. The network device recited in claim 1 , wherein the first protocol is a Secure Shell (SSH) protocol.

4. The network device recited in claim 1 , wherein the network device includes a data appliance.

5. The network device recited in claim 1 , wherein the network device includes a security appliance.

6. The network device recited in claim 1 , wherein the network device includes a gateway.

7. The network device recited in claim 1 , wherein the network device includes a security gateway.

8. The network device recited in claim 1 , wherein the processor is further configured to: determine a protocol of the monitored network communications, wherein the identifying of the application is capable of identifying two or more of the following protocols:
HyperText Transfer Protocol (HTTP), File Transfer Protocol (FTP), Secure Sockets Layer (SSL), Secure Shell (SSH), Domain Name System (DNS) requests, and an unknown protocol.

9. The network device recited in claim 1 , wherein the processor is further configured to: determine whether a firewall policy is violated based on the traffic analysis of the encrypted session traffic between the client and remote server.

10. The network device recited in claim 1 , wherein the processor is further configured to: report the remote server as suspicious to a security cloud service if the traffic analysis of the encrypted session traffic between the client and remote server violates a firewall policy.

11. The network device recited in claim 1 , wherein the processor is further configured to: block the encrypted session traffic between the client and remote server if a firewall policy violation is determined.

12. The network device recited in claim 1 , wherein the processor is further configured to: block the encrypted session traffic between the client and remote server if the deep packet inspection determined that the client is using the encrypted tunnel to evade a firewall policy.

13. The network device recited in claim 1 , wherein the firewall policy includes one or more rules associated with peer-to-peer activities, social networking activities, web browsing on one or more prohibited web sites, streaming music, streaming video, one or more unauthorized protocols, and/or one or more unauthorized applications.

14. The network device recited in claim 1 ,
wherein the processor is further configured to: identify content associated with the monitored network communications, wherein the network device blocks unauthorized downloading of content from the remote server to the client using the first protocol based on a firewall policy.

15. A method of a network device for monitoring network communications, comprising:
monitoring encrypted network communications between a client and a remote server;
decrypting encrypted session traffic between the client and remote server;
monitoring, within the decrypted encrypted session traffic, for a request from the client to create a tunnel using a first protocol with the remote server by applying a trusted man-in-the-middle technique using a self-signed certificate to inspect monitored traffic between the client and the remote server to facilitate deep packet inspection of the encrypted session traffic between the client and remote server, comprising:
determining, within the decrypted encrypted session traffic, if the client sends the request to create the tunnel using the first protocol with the remote server; and
in response to a determination that the client sent the request to create the tunnel using the first protocol with the remote server:
intercepting the request to establish the tunnel using the first protocol with the remote server, wherein the request is found within the decrypted encrypted session traffic;
blocking, after the intercepting of the request, the request to create the tunnel; and
sending a response to the client, the response informing the client that tunneling using the first protocol is not supported by the remote server;

performing a traffic analysis of the decrypted encrypted session traffic between the client and remote server, comprising:
identifying an application generating the monitored traffic, comprising determining what type of traffic a session involves, the type of traffic the session involves includes Hypertext Transfer Protocol (HTTP) traffic, File Transfer Protocol (FTP) traffic, Secure Sockets Layer (SSL) traffic, Secure Shell (SSH) traffic, a Domain Name System (DNS) request, unclassified application traffic, or any combination thereof;
identifying a user generating the monitored traffic, comprising determining a source IP of the mo

Assignees

• Palo Alto Networks Inc

Inventors

• Rostami-Hesarsorkh Shadi
• Jacobsen Michael Soren

Classifications

• H04L63/1408 Primary

  by monitoring network traffic (monitoring network traffic per se H04L43/00) · CPC title

• H04L63/02

  for separating internal from external traffic, e.g. firewalls · CPC title

• H04L63/0464

  using hop-by-hop encryption, i.e. wherein an intermediate entity decrypts the information and re-encrypts it before forwarding it · CPC title

• H04L63/168 Primary

  above the transport layer · CPC title

• H04L63/20

  for managing network security; network security policies in general (filtering policies H04L63/0227) · CPC title