Moving dispersion method for statistical anomaly detection in intrusion detection systems

Document Type: REVIEW PAPER



A unified method for statistical anomaly detection in intrusion detection systems is theoretically introduced. It is based on estimating a dispersion measure of numerical or symbolic data on successive moving windows in time and finding the times when a relative change of the dispersion measure is significant. Appropriate dispersion measures, relative differences, moving windows, as well as techniques for their efficient estimation are proposed. In particular, the method can be used for detecting network traffic anomalies due to network failures and network attacks such as (distributed) denial of service attacks, scanning attacks, SPAM and SPIT attacks, and massive malicious software attacks.


[1] L. Portnoy, E. Eskin, and S. Stolfo. Intrusion Detection with Unlabeled Data Using Clustering. In Proceedings of the ACM CSS Workshop on Data Mining Applied to Security (DMSA'01), Philadelphia, PA, USA, 2001.

[2] S.C. Lee and D.V. Heinbuch. Training a Neural- Network Based Intrusion Detector to Recognize Novel Attacks. IEEE Transactions on Systems, Man, and Cybernetics, Part A, 31(4):294-299, 2001.

[3] A. Lakhina, M. Crovella, and C. Diot. Mining Anomalies Using Tra_c Feature Distributions. In Proceedings of the ACM 2005 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM'05), pages 217-228, Philadelphia, Pennsylvania, USA, 2005.

[4] G.L. MacIsaac. Network Bandwidth Anomaly Detector Apparatus and Method for Detecting Network Attacks Using Correlation Function. Patent Application WO 2004/056063 A1, 2004.

[5] L. Li and G. Lee. DDoS Detection andWavelets. Telecommunication Systems - Modeling, Analysis, Design and Management, 28(3/4):435-451, 2005.

[6] P. Barford and D. Plonka. Characteristics of Network Tra_c Flow Anomalies. In Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, pages 69-73, San Francisco, CA, USA, 2001.

[7] N. Ye, C. Borror, and Y. Zhang. EWMA Techniques for Computer Intrusion Detection Through Anomalous Changes in Event Intensity. Quality and Reliability Engineering International, 18:443-451, 2002.

[8] D.M. Dempsey. Dynamic Deviation. Patent US 6,601,014 B1, 2003.

[9] A.E. Dudfield and M.A. Poletto. Connection Based Denial of Service Detection. Patent Application US 2004/0220984 A1, 2004.

[10] J. Viinikka and H. Debar. Monitoring IDS Background Noise Using EWMA Control Charts and Alert Information. In Proceedings of the 7th International Symposium on Recent Advances in Intrusion Detection (RAID'04), volume 3224 of Lecture Notes in Computer Science (LNCS), pages 166-187, Sophia Antipolis, France, 2004. Springer.

[11] M. Mandjes, I. Saniee, and A.L. Stolyar. Load Characterization and Anomaly Detection for Voice Over IP Traffic. IEEE Transactions on Neural Networks, 16(5):1019-1026, 2005.

[12] L. Feinstein, D. Schnackenberg, R. Balupari, and D. Kindred. Statistical Approaches to DDoS Attack Detection and Response. In Proceedings of the DARPA Information Survivability Conference and Exposition (DISCEX'03), volume 1, pages 303{314, Washington, DC, USA, 2003.

[13] K. Kumar, R.C. Joshi, and K. Singh. A Distributed Approach Using Entropy to Detect DDoS Attacks in ISP Domain. In Proceedings of the IEEE International Conference on Signal Processing, Communications and Networking (ICSCN'07), pages 331-337, Chennai, India, 2007.

[14] A. Wagner and B. Plattner. Entropy Based Worm and Anomaly Detection in Fast IP Networks. In Proceedings of the 14th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE'05), pages 172-177, Linköping University, Sweden, 2005.

[15] T. Okabe, T. Kitamura, and T. Shizuno. Statistical Traffic Identification Method Based on Flow-Level Behaviour for Fair VoIP Service. In Proceedings of the 1st IEEE Workshop on VoIP Management and Security, pages 33-38, Vancouver, BC, Canada, 2006.

[16] C.D. Jeffries, W.J. Jong, G.W. Randall, and K.V. Vu. Detecting Randomness in Computer Network Traffic. Patent Application US 2003/0200441 A1, 2003.

[17] T. Peng, C. Leckie, and K. Ramamohanarao. Proactively Detecting Distributed Denial of Service Attacks Using Source IP Address Monitoring. In Proceedings of the 3rd International IFIP-TC6 Conference on Networking Technologies, Services, and Protocols; Performance of Computer and Communication Networks; Mobile and Wireless Communications (Networking 2004), volume 3042 of Lecture Notes in Computer Science (LNCS), pages 771-782, Athens, Greece, 2004. Springer.

[18] E. Haraldsson. DDoS Attack Detection Based on Netow Logs. Student thesis, Swiss Federal Institute of Technology, Zurich, 2003.

[19] A. Weisskopf. Plug-ins for DDoS Attack Detection in Realtime. Semester thesis, Swiss Federal Institute of Technology, Zurich, 2004.

[20] M.D. Esteban and D. Morales. A Summary on Entropy Statistics. Kybernetika, 31(4):337-346, 1995.

[21] J.Dj. Goliḉ. On the Relationship Between the Information Measures and the Bayes Probability of Error. IEEE Transactions on Information Theory, 33(5):681-693, 1987.

[22] D.E. Knuth. The Art of Computer Programming, Volume 2: Seminumerical Algorithms. Addison- Wesley, 1998.