An on-line learning statistical model to detect malicious web requests

Detecting malicious connection attempts and attacks against web-based applications is one of many approaches to protect the World Wide Web and its users. In this paper, we present a generic method for detecting anomalous and potentially malicious web requests from the network's point of view without prior knowledge or training data of the web-based application. The algorithm assumes that a legitimate request is an ordered sequence of semantic entities. Malicious requests are in different order or include entities which deviate from the structure of the majority of requests. Our method learns a variable-order Markov model from legitimate sequences of semantic entities. If a sequence's probability deviates from previously seen ones, it is reported as anomalous. Experiments were conducted on logs from a social networking web site. The results indicate that that the proposed method achieves good detection rates at acceptable false-alarm rates.