Modelling Behavioural Dynamics for Application Layer Distributed Denial of Service Attack Detection

Abstract

Distributed Denial of Service (DDoS) attacks are one of the oldest and most dangerous attacks against network infrastructure and web applications alike. Traditionally, DDoS attacks were executed using network layer protocols for generating a large volume of requests, thereby exhausting the network bandwidth and leading to a degradation in the quality of Internet services. These attacks, called Network layer DDoS attacks, no longer pose a significant threat due to the availability of cheap bandwidth and the development of sophisticated detection mechanisms against these attacks. With the success of network layer DDoS attacks no longer guaranteed, attackers have slowly started using application layer protocols to launch DDoS attacks. Application Layer DDoS (AL-DDoS) attacks attempt to take down web applications by exhausting server resources such as CPU, database, memory or socket connections by using the features of application layer protocols. Majority of these attacks use the HTTP/1.1 protocol for launching attacks, and in particular there has been a growing trend of using computationally expensive requests to launch DDoS attacks. These attacks are called Asymmetric AL-DDoS attacks and are generally imperceptible owing to the use of legitimate requests and a comparatively low attack volume. Due to these features, simple firewall rules and request inspection techniques are ineffective against these attacks and hence, an analysis of user behaviour is required for detecting these attacks. Most of the existing detection mechanisms focus on building a model of legitimate user behaviour as under the HTTP/1.1 protocol, and then identifying attacks by observing the deviation from the learned model. Existing detection approaches for asymmetric AL-DDoS attacks use indirect representations of actual user behaviour and use complex modelling techniques, which leads to a higher false positive rate (FPR) and longer detection time, which makes them unsuitable for real time use. In addition, most of these models are unable to adapt to changing user behaviour, which leads to the model becoming ineffective in the long run. A review of existing literature suggests that there is a need for alightweight, fast and adaptable detection mechanism for asymmetric AL-DDoS attacks that has a very low false positive rate. The recent standardization of HTTP/2 adds another layer of complexity over asymmetric AL-DDoS detection. HTTP/2 was designed to improve the performance of web servers, and has been greatly successful in reducing the average response time of web servers due to the introduction of features like multiplexing and server push. This has led to more and more web applications migrating to HTTP/2. However, while reducing page load time for clients, HTTP/2 puts additional load on web servers, leading to concerns about these servers being more vulnerable to asymmetric AL-DDoS attacks. In addition, there is no evidence found in existing literature regarding the possibility of multiplexing and server push being misused to launch potentially lethal asymmetric AL-DDoS attacks. This lack of understanding has led to existing mechanisms being unable to handle the HTTP/2 protocol effectively. In this work, an attempt is made to model the actual behavioural dynamics of legitimate users using an annotated Probabilistic Timed Automata (PTA) along with a suspicion scoring mechanism for differentiating between legitimate and malicious users. This allows the detection mechanism to be extremely fast and have a low FPR. In addition, the model can adapt to changing user behaviour in an incremental manner, which further reduces the FPR. Experiments on public datasets reveal that our proposed approach has a high detection rate and low FPR and adds negligible overhead to the web server, which makes it ideal for real time use. This work also explores the impact of asymmetric AL-DDoS attacks on HTTP/2 servers. Our experiments demonstrate that an HTTP/2 server is actually more resilient to asymmetric AL-DDoS attacks as compared to an HTTP/1.1 server. However, despite the improved resilience, HTTP/2 servers are vulnerable to a more sophisticated class of attacks. We demonstrate that multiplexing and server push features in HTTP/2 can be misused to launch a sophisticated attack called Multiplexed Asymmetric Attack, that can exhaust server resources much faster and with minimal number of attacking clients. In order to detect these attacks, the PTA-based behavioural model is extended to accommodate HTTP/2-specific features. Our experiments demonstrate that the iniiclusion of these features allows the system to detect Multiplexed Asymmetric Attacks effectively. There is a considerable degree of similarity between attacking connections in a DDoS attacks due to the use of common attack generation tools and botnets. In the case of an AL-DDoS attack, this similarity manifests itself in the form of repeating sequences of HTTP requests across attacking connections. Knowledge of this similarity allows for the early detection of AL-DDoS attacks, thereby reducing the average detection time of the system and allowing it to operate in real time. A dynamic signature based approach using HTTP request sequences is used in order to facilitate the early detection of AL-DDoS attack as part of the proposed approach. Experimental results indicate that the use of the early detection mechanism leads to a considerable decrease in detection time, and leads to efficient real time use.