Document Type : Research Article
Authors
1 Data and Communication Security Laboratory, Ferdowsi University of Mashhad, Mashhad, Iran
2 Software Quality Laboratory, Ferdowsi University of Mashhad, Mashhad, Iran
Abstract
Nowadays, targeted attacks like Advanced Persistent Threats (APTs) has become one of the major concern of many enterprise networks. As a common approach to counter these attacks, security staff deploy a variety of heterogeneous security and non-security sensors at different lines of defense (Network, Host, and Application) to track the attacker's behaviors during their kill chain. However, one of the drawbacks of this approach is the huge amount of events raised by heterogeneous sensors which makes it difficult to analyze logged events for later processing i.e. event correlation for timely detection of APT attacks. The main focus of the existing works is only on the degree to which the event volume is reduced, while the amount of security information lost during the event aggregation process is also very important. In this paper, we propose a three-phase event aggregation method to reduce the volume of heterogeneous events during APT attacks considering the lowest rate of loss of security information. To this aim, at first, low-level events of the sensors are clustered into some similar event groups and then, after filtering noisy event clusters, the remained clusters are summarized based on an Attribute-Oriented Induction (AOI) method in a controllable manner to reduce the unimportant or duplicated events. The method has been evaluated on the three publicly available datasets: SotM34, Bryant, and LANL. The experimental results show that the method is efficient enough in event aggregation and can reduce events volume up to 99.7 with an acceptable level of information loss ratio (ILR).
Keywords
[1] S. Quintero-Bonilla and A. Mart´ın del Rey. A new proposal on the advanced persistent threat: A survey. Applied Sciences, 10(11):3874, 2020.
[2] M. Hus´ak, J. Kom´arkov´a, E. Bou-Harb, and P. ˇCeleda. Survey of attack projection, prediction, and forecasting in cyber security. IEEE Communications Surveys & Tutorials, 21(1):640–660, 2018.
[3] S. Singh, P. K. Sharma, S. Y. Moon, D. Moon, and J. H. Park. A comprehensive study on apt attacks and countermeasures for future networks and communications: challenges and solutions. The Journal of Supercomputing, 75:4543–4574, 2019.
[4] G. Kim, C. Lee, J. Jo, and H. Lim. Automatic extraction of named entities of cyber threats using a deep bi-lstm-crf network. International journal of machine learning and cybernetics, 11:2341–2355, 2020.
[5] M. Khosravi-Farmad, A. A. Ramaki, and A. G.Bafghi. Moving target defense against advanced persistent threats for cybersecurity enhancement. In 2018 8th International Conference on Computer and Knowledge Engineering (ICCKE),
pages 280–285. IEEE, 2018.
[6] P. K. Bahrami, A. Dehghantanha, T. Dargahi, R. M. Parizi, K. R. Choo, H. Javadi, Lizhe Wang, Fatos Xhafa, and Wei Ren. A layered security architecture based on cyber kill chain against advanced persistent threats, 2019.
[7] B. D. Bryant and H. Saiedian. A novel kill-chain framework for remote security log analysis with siem software. Computers & Security, 67:198–210, 2017.
[8] F. Wilkens, F. Ortmann, S. Haas, M. Vallentin, and M. Fischer. Multi-stage attack detection via kill chain state machines. In Proceedings of the 3rd Workshop on Cyber-Security Arms Race, pages 13–24, 2021.
[9] E. M. Hutchins, M. J. Cloppert, and R. M. Amin. Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains. Leading Issues in Information Warfare & Security Research, 1(1):80, 2011.
[10] R. Luh, S. Marschalek, M. Kaiser, H. Janicke, and S. Schrittwieser. Semantics-aware detection of targeted attacks: a survey. Journal of Computer Virology and Hacking Techniques, 13(1):47–85, 2017.
[11] A. Spadaro. Event correlation for detecting advanced multi-stage cyber-attacks (Doctoral dissertation). Delft University of Technology, 2013.
[12] S. Salah, G. Maci´a-Fern´andez, and J. E. DiAz-Verdejo. A model-based survey of alert correlation techniques. Computer Networks, 57(5):1289–1317, 2013.
[13] A. A. Ramaki, A. Rasoolzadegan, and A. G.Bafghi. A systematic mapping study on intrusion alert analysis in intrusion detection systems. ACM Computing Surveys (CSUR), 51(3):55, 2018.
[14] F. Valeur, G. Vigna, C. Kruegel, and R. A. Kemmerer. Comprehensive approach to intrusion detection alert correlation. IEEE Transactions on dependable and secure computing, 1(3):146–169, 2004.
[15] R. H. Syed, J. Pazardzievska, and J. Bourgeois. Fast attack detection using correlation and summarizing of security alerts in grid computing networks. The Journal of Supercomputing, 62(2):804–827, 2012.
[16] A. A. Ramaki, M. Amini, and R. E. Atani. Rteca: Real time episode correlation algorithm for multistep attack scenarios detection. Computers & Security, 49:206–219, 2015.
[17] M. Husak, M. Cermak, M. Lastovicka, and J. Vykopal. Exchanging security events: Which and how many alerts can we aggregate? In IFIP/IEEE Symposium on Integrated Network and Service Management (IM), pages 604–607. IEEE, 2017.
[18] G. P. Spathoulas and S. K. Katsikas. Enhancing ids performance through comprehensive alert post-processing. Computers & Security, 37:176–196, 2013.
[19] Y. Sun and X. Chen. An improved frequent pattern growth based approach to intrusion detection system alert aggregation. In Journal of Physics: Conference Series, 1437:1, 2020.
[20] F. M. Alserhani. Alert correlation and aggregation techniques for reduction of security alerts and detection of multistage attack. International Journal of Advanced Studies in Computers, Science and Engineering, 5(2):1, 2016.
[21] R. Shittu, A. Healing, R. Ghanea-Hercock, R. Bloomfield, and M. Rajarajan. Intrusion alert prioritisation and attack detection using postcorrelation analysis. Computers & Security, 50:1–15, 2015.
[22] M. Soleimani and A. A. Ghorbani. Multi-layer episode filtering for the multi-step attack detection. Computer Communications, 35(11):1368–1379, 2012.
[23] S. C. de Alvarenga, Barbon Jr, Miani S., R. S.,M. Cukier, and B. B. Zarpel¯ao. Process mining and hierarchical clustering to help intrusion alert visualization. Computers & Security, 73:474–491, 2018.
[24] R. Zhang, T. Guo, and J. Liu. An ids alerts aggregation algorithm based on rough set theory. In IOP Conference Series: Materials Science and Engineering, 322:6, 2018.
[25] J. Kim, I. Moon, K. Lee, S. C. Suh, and I. Kim.Scalable security event aggregation for situation analysis. In First International Conference on Big Data Computing Service and Applications, pages 14–23. IEEE, 2015.
[26] S. Saad and I. Traore. Heterogeneous multi-sensor ids alerts aggregation using semantic analysis. Journal of Information Assurance & Security, 7:2, 2012.
[27] A. Nadeem, S. Verwer, and S. J. Yang. Sage: Intrusion alert-driven attack graph extractor. In 2021 IEEE Symposium on Visualization for Cyber Security (VizSec), pages 36–41. IEEE, 2021.
[28] O. B. Fredj. A realistic graph-based alert correlation system. Security and Communication Networks, 8(15):2477–2493, 2015.
[29] A. A. Ramaki and A. Rasoolzadegan. Causal knowledge analysis for detecting and modeling multi-step attacks. Security and Communication Networks, 9(18):6042–6065, 2016.
[30] H. Kim, H. Kwon, and K. K. Kim. Modified cyber kill chain model for multimedia service environments. Multimedia Tools and Applications, 78(3):3153–3170, 2019.
[31] J. R. Rutherford and G. B. White. Using an improved cybersecurity kill chain to develop an improved honey community. In 49th Hawaii International Conference on System Sciences (HICSS), pages 2624–2632. IEEE, 2016.
[32] S. K. Pandey and B. M. Mehtre. A lifecycle based approach for malware analysis. In Fourth International Conference on Communication Systems and Network Technologies (CSNT), pages 767–771. IEEE, 2014.
[33] M. I. Center. APT1: Exposing one of China’s cyber espionage units. Mandiant.com, 2013.
[34] J. Flynn. Intrusion along the kill chain. In Proceedings of BlackHat USA. BlackHat, 2012.
[35] Command Five Pty Ltd. Advanced persistent threats: A decade in review. http://www.commandfive.com/papers/C5_APT_ADecadeInReview.pdf.
[36] Dell SecureWorks. Breaking the kill chain- know-ing, detecting, disrupting and eradicating the advanced threat. https://webobjects.cdw.com/webobjects/media/pdf/paloalto/Breaking-the-Attack-Kill-Chain.pdf, 2015.
[37] P. Pols and J. van den Berg. The unified kill chain. CSA Thesis, Hague, pages 1–104, 2017.
[38] A. J. C. Lima. PhD thesis, Advanced persistent threats, 2015.
[39] B. Hudson. Advanced Persistent Threats: Detection, Protection and Prevention. Sophos Ltd, 2014.
[40] E. Tonelli. WatchGuard APT Blocker. Watch-Guard Technologies, 2014.
[41] Cox A. Stalking the kill chain: The attacker’s chain. 2012.
[42] P. Chen, L. Desmet, and C. Huygens. A study on advanced persistent threats. In IFIP International Conference on Communications and Multimedia Security, pages 63–72. Springer, 2014.
[43] Advanced Persistent Threats and other Advanced Attacks, editors. Websense, 2013.
[2] M. Hus´ak, J. Kom´arkov´a, E. Bou-Harb, and P. ˇCeleda. Survey of attack projection, prediction, and forecasting in cyber security. IEEE Communications Surveys & Tutorials, 21(1):640–660, 2018.
[3] S. Singh, P. K. Sharma, S. Y. Moon, D. Moon, and J. H. Park. A comprehensive study on apt attacks and countermeasures for future networks and communications: challenges and solutions. The Journal of Supercomputing, 75:4543–4574, 2019.
[4] G. Kim, C. Lee, J. Jo, and H. Lim. Automatic extraction of named entities of cyber threats using a deep bi-lstm-crf network. International journal of machine learning and cybernetics, 11:2341–2355, 2020.
[5] M. Khosravi-Farmad, A. A. Ramaki, and A. G.Bafghi. Moving target defense against advanced persistent threats for cybersecurity enhancement. In 2018 8th International Conference on Computer and Knowledge Engineering (ICCKE),
pages 280–285. IEEE, 2018.
[6] P. K. Bahrami, A. Dehghantanha, T. Dargahi, R. M. Parizi, K. R. Choo, H. Javadi, Lizhe Wang, Fatos Xhafa, and Wei Ren. A layered security architecture based on cyber kill chain against advanced persistent threats, 2019.
[7] B. D. Bryant and H. Saiedian. A novel kill-chain framework for remote security log analysis with siem software. Computers & Security, 67:198–210, 2017.
[8] F. Wilkens, F. Ortmann, S. Haas, M. Vallentin, and M. Fischer. Multi-stage attack detection via kill chain state machines. In Proceedings of the 3rd Workshop on Cyber-Security Arms Race, pages 13–24, 2021.
[9] E. M. Hutchins, M. J. Cloppert, and R. M. Amin. Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains. Leading Issues in Information Warfare & Security Research, 1(1):80, 2011.
[10] R. Luh, S. Marschalek, M. Kaiser, H. Janicke, and S. Schrittwieser. Semantics-aware detection of targeted attacks: a survey. Journal of Computer Virology and Hacking Techniques, 13(1):47–85, 2017.
[11] A. Spadaro. Event correlation for detecting advanced multi-stage cyber-attacks (Doctoral dissertation). Delft University of Technology, 2013.
[12] S. Salah, G. Maci´a-Fern´andez, and J. E. DiAz-Verdejo. A model-based survey of alert correlation techniques. Computer Networks, 57(5):1289–1317, 2013.
[13] A. A. Ramaki, A. Rasoolzadegan, and A. G.Bafghi. A systematic mapping study on intrusion alert analysis in intrusion detection systems. ACM Computing Surveys (CSUR), 51(3):55, 2018.
[14] F. Valeur, G. Vigna, C. Kruegel, and R. A. Kemmerer. Comprehensive approach to intrusion detection alert correlation. IEEE Transactions on dependable and secure computing, 1(3):146–169, 2004.
[15] R. H. Syed, J. Pazardzievska, and J. Bourgeois. Fast attack detection using correlation and summarizing of security alerts in grid computing networks. The Journal of Supercomputing, 62(2):804–827, 2012.
[16] A. A. Ramaki, M. Amini, and R. E. Atani. Rteca: Real time episode correlation algorithm for multistep attack scenarios detection. Computers & Security, 49:206–219, 2015.
[17] M. Husak, M. Cermak, M. Lastovicka, and J. Vykopal. Exchanging security events: Which and how many alerts can we aggregate? In IFIP/IEEE Symposium on Integrated Network and Service Management (IM), pages 604–607. IEEE, 2017.
[18] G. P. Spathoulas and S. K. Katsikas. Enhancing ids performance through comprehensive alert post-processing. Computers & Security, 37:176–196, 2013.
[19] Y. Sun and X. Chen. An improved frequent pattern growth based approach to intrusion detection system alert aggregation. In Journal of Physics: Conference Series, 1437:1, 2020.
[20] F. M. Alserhani. Alert correlation and aggregation techniques for reduction of security alerts and detection of multistage attack. International Journal of Advanced Studies in Computers, Science and Engineering, 5(2):1, 2016.
[21] R. Shittu, A. Healing, R. Ghanea-Hercock, R. Bloomfield, and M. Rajarajan. Intrusion alert prioritisation and attack detection using postcorrelation analysis. Computers & Security, 50:1–15, 2015.
[22] M. Soleimani and A. A. Ghorbani. Multi-layer episode filtering for the multi-step attack detection. Computer Communications, 35(11):1368–1379, 2012.
[23] S. C. de Alvarenga, Barbon Jr, Miani S., R. S.,M. Cukier, and B. B. Zarpel¯ao. Process mining and hierarchical clustering to help intrusion alert visualization. Computers & Security, 73:474–491, 2018.
[24] R. Zhang, T. Guo, and J. Liu. An ids alerts aggregation algorithm based on rough set theory. In IOP Conference Series: Materials Science and Engineering, 322:6, 2018.
[25] J. Kim, I. Moon, K. Lee, S. C. Suh, and I. Kim.Scalable security event aggregation for situation analysis. In First International Conference on Big Data Computing Service and Applications, pages 14–23. IEEE, 2015.
[26] S. Saad and I. Traore. Heterogeneous multi-sensor ids alerts aggregation using semantic analysis. Journal of Information Assurance & Security, 7:2, 2012.
[27] A. Nadeem, S. Verwer, and S. J. Yang. Sage: Intrusion alert-driven attack graph extractor. In 2021 IEEE Symposium on Visualization for Cyber Security (VizSec), pages 36–41. IEEE, 2021.
[28] O. B. Fredj. A realistic graph-based alert correlation system. Security and Communication Networks, 8(15):2477–2493, 2015.
[29] A. A. Ramaki and A. Rasoolzadegan. Causal knowledge analysis for detecting and modeling multi-step attacks. Security and Communication Networks, 9(18):6042–6065, 2016.
[30] H. Kim, H. Kwon, and K. K. Kim. Modified cyber kill chain model for multimedia service environments. Multimedia Tools and Applications, 78(3):3153–3170, 2019.
[31] J. R. Rutherford and G. B. White. Using an improved cybersecurity kill chain to develop an improved honey community. In 49th Hawaii International Conference on System Sciences (HICSS), pages 2624–2632. IEEE, 2016.
[32] S. K. Pandey and B. M. Mehtre. A lifecycle based approach for malware analysis. In Fourth International Conference on Communication Systems and Network Technologies (CSNT), pages 767–771. IEEE, 2014.
[33] M. I. Center. APT1: Exposing one of China’s cyber espionage units. Mandiant.com, 2013.
[34] J. Flynn. Intrusion along the kill chain. In Proceedings of BlackHat USA. BlackHat, 2012.
[35] Command Five Pty Ltd. Advanced persistent threats: A decade in review. http://www.commandfive.com/papers/C5_APT_ADecadeInReview.pdf.
[36] Dell SecureWorks. Breaking the kill chain- know-ing, detecting, disrupting and eradicating the advanced threat. https://webobjects.cdw.com/webobjects/media/pdf/paloalto/Breaking-the-Attack-Kill-Chain.pdf, 2015.
[37] P. Pols and J. van den Berg. The unified kill chain. CSA Thesis, Hague, pages 1–104, 2017.
[38] A. J. C. Lima. PhD thesis, Advanced persistent threats, 2015.
[39] B. Hudson. Advanced Persistent Threats: Detection, Protection and Prevention. Sophos Ltd, 2014.
[40] E. Tonelli. WatchGuard APT Blocker. Watch-Guard Technologies, 2014.
[41] Cox A. Stalking the kill chain: The attacker’s chain. 2012.
[42] P. Chen, L. Desmet, and C. Huygens. A study on advanced persistent threats. In IFIP International Conference on Communications and Multimedia Security, pages 63–72. Springer, 2014.
[43] Advanced Persistent Threats and other Advanced Attacks, editors. Websense, 2013.
[44] Paul Giura and Wei Wang. A context-based detection framework for advanced persistent threats. In 2012 International Conference on Cyber Security, pages 69–74. IEEE, 2012.
[45] K. Pei, Z. Gu, B. Saltaformaggio, S. Ma, F. Wang, Z. Zhang, L. Si, X. Zhang, and D. Xu. Hercule: Attack story reconstruction via community discovery on correlated log graph. In Proceedings of the 32nd Annual Conference on Computer Security Applications, pages 583–595, 2016.
[46] P. Bhatt, E. T. Yano, and P. Gustavsson. Towards a framework to detect multi-stage advanced persistent threats attacks. In 8th International Symposium on Oriented System Engineering (SOSE), pages 390–395. IEEE, 2014.
[47] P. Giura and W. Wang. Using large scale distributed computing to unveil advanced persistent threats. Science, 1(3):93–102, 2013.
[48] S. Bhatt, P. K. Manadhata, and L. Zomlot. The operational role of security information and event management systems. IEEE Security & Privacy, 12(5):35–41, 2014.
[49] C. Vega, P. Roquero, R. Leira, I. Gonzalez, and J. Aracil. Loginson: a transform and load system for very large-scale log analysis in large it infrastructures. The Journal of Supercomputing, 73(9):3879–3900, 2017.
[50] S. Chhajed. Learning ELK stack. Packt Publishing Ltd, 2015.
[51] O. Negoita and M. Carabas. Enhanced security using elasticsearch and machine learning. In Science and Information Conference, pages 244–254, 2020.
[52] A. Sapegin, D. Jaeger, A. Azodi, M. Gawron, F. Cheng, and C. Meinel. Hierarchical object log format for normalization of security events. In 9th International Conference on Information Assurance and Security (IAS), pages 25–30. IEEE, 2013.
[53] X. Zhao, J. Liang, and F. Cao. A simple and effective outlier detection algorithm for categorical data. International Journal of Machine Learning and Cybernetics, 5(3):469–477, 2014.
[54] M. Amer and M. Goldstein. Nearest-neighbor and clustering based anomaly detection algorithms for rapidminer. In In Proc of 3rd RapidMiner Community Meeting and Conference (RCOMM), pages 1–12, 2012.
[55] Z. He, X. Xu, and S. Deng. Discovering clusterbased local outliers. Pattern Recognition Letters, 24(9-10):1641–1650, 2003.
[56] M. Ahmed. Data summarization: a survey. Knowledge and Information Systems, pages 1–25, 2018.
[57] J. Jiang, J. Chen, K. K. R. Choo, C. Liu, K. Liu, and M. Yu. A visualization scheme for network forensics based on attribute oriented induction based frequent item mining and hyper graph. In International Conference on Digital Forensics and Cyber Crime, pages 130–143, 2017.
[58] A. Chuvakin. Scan 34 - the honeypot project.
[59] A. D. Kent. Comprehensive, multi-source cybersecurity events data set. Technical report, Los Alamos National Lab.(LANL), Los Alamos, NM (United States), 2015.
[60] S. Panichprecha. Abstracting and correlating heterogeneous events to detect complex scenarios (Doctoral dissertation). Queensland University of Technology, 2009.
[61] M. Halkidi, Y. Batistakis, and M. Vazirgiannis. Clustering validity checking methods: part ii. ACM Sigmod Record, 31(3):19–27, 2002.
[62] T. Dziopa. Clustering validity indices evaluation with regard to semantic homogeneity. In FedCSIS Position Papers, pages 3–9, January 2016.
[63] A. Hassanzadeh and R. Burkett. Samiit: Spiral attack model in iiot mapping security alerts to attack life cycle phases. In 5th International Symposium for ICS & SCADA Cyber Security Research 2018, pages 11–20, 2018.
[64] J. M. Butler. Benchmarking security information event management (SIEM). A SANS Whitepaper, 2009.
[45] K. Pei, Z. Gu, B. Saltaformaggio, S. Ma, F. Wang, Z. Zhang, L. Si, X. Zhang, and D. Xu. Hercule: Attack story reconstruction via community discovery on correlated log graph. In Proceedings of the 32nd Annual Conference on Computer Security Applications, pages 583–595, 2016.
[46] P. Bhatt, E. T. Yano, and P. Gustavsson. Towards a framework to detect multi-stage advanced persistent threats attacks. In 8th International Symposium on Oriented System Engineering (SOSE), pages 390–395. IEEE, 2014.
[47] P. Giura and W. Wang. Using large scale distributed computing to unveil advanced persistent threats. Science, 1(3):93–102, 2013.
[48] S. Bhatt, P. K. Manadhata, and L. Zomlot. The operational role of security information and event management systems. IEEE Security & Privacy, 12(5):35–41, 2014.
[49] C. Vega, P. Roquero, R. Leira, I. Gonzalez, and J. Aracil. Loginson: a transform and load system for very large-scale log analysis in large it infrastructures. The Journal of Supercomputing, 73(9):3879–3900, 2017.
[50] S. Chhajed. Learning ELK stack. Packt Publishing Ltd, 2015.
[51] O. Negoita and M. Carabas. Enhanced security using elasticsearch and machine learning. In Science and Information Conference, pages 244–254, 2020.
[52] A. Sapegin, D. Jaeger, A. Azodi, M. Gawron, F. Cheng, and C. Meinel. Hierarchical object log format for normalization of security events. In 9th International Conference on Information Assurance and Security (IAS), pages 25–30. IEEE, 2013.
[53] X. Zhao, J. Liang, and F. Cao. A simple and effective outlier detection algorithm for categorical data. International Journal of Machine Learning and Cybernetics, 5(3):469–477, 2014.
[54] M. Amer and M. Goldstein. Nearest-neighbor and clustering based anomaly detection algorithms for rapidminer. In In Proc of 3rd RapidMiner Community Meeting and Conference (RCOMM), pages 1–12, 2012.
[55] Z. He, X. Xu, and S. Deng. Discovering clusterbased local outliers. Pattern Recognition Letters, 24(9-10):1641–1650, 2003.
[56] M. Ahmed. Data summarization: a survey. Knowledge and Information Systems, pages 1–25, 2018.
[57] J. Jiang, J. Chen, K. K. R. Choo, C. Liu, K. Liu, and M. Yu. A visualization scheme for network forensics based on attribute oriented induction based frequent item mining and hyper graph. In International Conference on Digital Forensics and Cyber Crime, pages 130–143, 2017.
[58] A. Chuvakin. Scan 34 - the honeypot project.
[59] A. D. Kent. Comprehensive, multi-source cybersecurity events data set. Technical report, Los Alamos National Lab.(LANL), Los Alamos, NM (United States), 2015.
[60] S. Panichprecha. Abstracting and correlating heterogeneous events to detect complex scenarios (Doctoral dissertation). Queensland University of Technology, 2009.
[61] M. Halkidi, Y. Batistakis, and M. Vazirgiannis. Clustering validity checking methods: part ii. ACM Sigmod Record, 31(3):19–27, 2002.
[62] T. Dziopa. Clustering validity indices evaluation with regard to semantic homogeneity. In FedCSIS Position Papers, pages 3–9, January 2016.
[63] A. Hassanzadeh and R. Burkett. Samiit: Spiral attack model in iiot mapping security alerts to attack life cycle phases. In 5th International Symposium for ICS & SCADA Cyber Security Research 2018, pages 11–20, 2018.
[64] J. M. Butler. Benchmarking security information event management (SIEM). A SANS Whitepaper, 2009.
)