Intelligence behind IPS

There has been a lot of debate during the past two years about the future of intrusion prevention. Some have argued that the future of network security lies in intrusion prevention and that intrusion detection is dead. Others have made the case that detection is simply evolving into prevention. But now that prevention technology has had time to evolve, it is clear that enterprises require security that not only combines high-quality detection and prevention, but is more than just a sum of these two parts.

1. Introduction

Intrusion detection is a key component of any serious security strategy in today IT infrastructures. Like other measures such as data encryption, access control and authentication, vulnerability scanning, anti-virus, firewall, .. It adds a necessary layer in a defense in depth strategy.

Firewalls and Intrusion Detection Systems (IDS) are no longer sufficient to cope with these types of attack.

IDS have the capability to detect potential attacks and fire alarms, but they are by no way able to prevent them, or stop them at their early stage: the attack is rightly detected, but may be to late when the network is already infected. In 2001, the Code Red II worm, and more recently, in 2003, the SQL Slammer or the Blaster worms spread so fast that they had infected a lot of systems before an alert could be processed. Security professionals are overwhelmed by the quantity of alarms raised by IDS and either don’t analyze them in details or in the contrary, waste a precious time to study them, delaying the moment to intervene.

An Intrusion prevention system (IPS) has the capability of blocking offending operations. It shows a pro-active behaviour when the IDS one is only reactive: it

prevents attacks by fighting them before they may cause damages to the etwork or hosts, rather than simply reacting to them. Attacks are answered in real time. (Zero day answer).

According to Gartner Group, an IPS must meet three key criteria:

  • While it analyzes network traffic or data flow inside host, it must not block

normal operations. But it has to perform blocking actions against suspicious activities. It must have a high level of performance and must perform accurate

actions because bad attack identification will lead to a Denial Of Service (DOS).

  • It must block malicious actions using signature based blocking of known

attacks, as well as behavior and anomaly-based detection algorithms. These

algorithms must operate at the application level in addition to standard, networklevel firewall processing.

  • It must detect and block higher percentage of attacks than firewalls.

The very first IPS notions appeared in the 1990s. At the beginning IPS products were no more than IDS products with additional IPS functional packages. The first “real” IPS appeared in 1999. It was called Stormwatch and was created by Okena systems. This IPS was based on their INCORE (Intercept Correlate Rules Engine) architecture. It was analyzing files and network activities and was performing real-time decisions based on application behaviour. CISCO systems acquired Okena in 2003.

Attacks are more and more difficult to mitigate due to the complexity of today’s network environment. So IPS uses more and more sophisticated analysis methods to detect a potential attack.

2. Intrusion Detection Methods

The basis for intrusion detection is analysis of data. Depending on the type of IPS

these data correspond to:

  • Network Intrusion Detection System (NIPS): traffic flowing through the network under scrutiny.
  • Host Intrusion Detection System (HIPS): data exchange between well chosen processes or content of system events or log files.

Different methods of analysis may be used.

2.1 Misuse – Signature – Rule-based Detection

This method is based on the search of known attacks represented by signatures saved in a database. A signature consist in many information, such as the source and/or the destination of the attack, the source and/or the destination service, the communication protocol used, the payload content…

IPS scans the incoming data in order to detect either a pattern (pattern matching) or a traffic stream (stateful pattern matching) already known in the signature database.

In a pattern matching methodology, a single packet is analyzed in order to find a fixed sequence of bytes. The pattern is often associated with a service or port, a source or destination IP address, and/or the analysis of a specific portion of the payload. For instance, IPS will identify an attack if a received packet is an UDP (User Datagram Protocol) with a destination port 564, and if the payload contains the string “attacksucceeded”. But there are multiple protocols and attacks that do not use welldefined ports and the pattern matching solution has difficulty detecting such attacks. In addition, this method is not suitable to streamed based traffic such as HTTP.

In a stateful pattern matching methodology, IPS searches unique sequences spread across multiple packets within a stream of data. For this purpose, the session context has to be kept. From the preceding example, the improvement comes from the fact that the researched string can be detected even if “attack” is located in one packet and “succeeded” in another one. Even if more sophisticated than pattern matching, this method is also vulnerable to false positives due to attack types variants. Moreover this method requires more resources as more data has to be saved before it can be decided if an attack has occurred or not.

Figure 1 shows an example of a signature in the Snort product. The signature is

specified by the keyword “alert”. Then the type of protocol is specified (TCP) along with:

  • The source: any TCP port from $EXTERNAL_NET IP address
  • And the destination: “$HTTP_PORTS” port to “$HTTP_SERVERS” IP address.

The “msg” field indicates which type of message to look for: in this case a connection through the Common Gateway Interface (CGI) of a web server to get the “passwd.txt” file. The “content field” contains the searched data string.

alert :


(msg:”WEB-CGI /wwwboard/passwd.txt access”;


Figure : Intrusion signature example (Snort Signature SID 807)

2.2 Anomaly – Profile-based detection

This method is based on the comparison of networks or host data against a set of

profiles representing the “normal” state of the network’s traffic activities, user’s

activities or application activities. The profiles may be comprised of statistical

behaviour, such as “the system TCP traffic doesn’t exceed 60 % of the capacity” and of qualitative behaviour such as “the user TOTO never FTP files outside of the company”. First, there is a learning phase during which the IPS builds the profiles. During this stage, the system must not be submitted to intrusive attacks in order that they should not be recorded as “normal” conditions.

Once profiles are built, IPS will monitor network or host data and compare their state to the already defined profile. If there is a deviation from this profile, it considers that there is an anomaly.

There are three main categories of anomaly detection:

– Behavioural analysis: the IPS looks for deviations against the profiles.

– Protocol Decode-Based Checking: the IPS looks for network protocol violations or

misuse as defined by Requests for Comment (RFC). If this method is effective for welldefined protocols because it reduces false positive, this is not the case if the protocol is loosely defined.

– Traffic Pattern analysis: the IPS searches for suspect patterns linked to specific

protocols to guarantee that they are not used by attackers. This comes from the fact that some vendors do not implement protocols according to RFCs.