Quantitative evaluation of software security: an approach based on UML/SecAM and evidence theory



School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran


Quantitative and model-based prediction of security in the architecture design stage facilitates early detection of design faults hence reducing modification costs in subsequent stages of software life cycle. However, an important question arises with respect to the accuracy of input parameters. In practice, security parameters can rarely be estimated accurately due to the lack of sufficient knowledge. This inaccuracy is ignored in most of the existing evaluation methods. The aim of this paper is to explicitly consider parameter uncertainty in the software security evaluation process. In particular, we use the Dempster-Shafer theory of evidence to formulate the uncertainties in input parameters and determine their effects on output measures. In the proposed method, security attacks are expressed using UML diagrams (i.e., misuse case and mal-activity diagrams) and security parameters are specified using the SecAM profile. UML/SecAM models are then transformed into attack trees, which allow quantifying the probability of security breaches. The applicability of the method is validated by a case study on an online marketing system.


[1] Amita Devaraj et al. Uncertainty propagation in analytic availability models. In Reliable Distributed Systems, 2010 29th IEEE Symposium on, pages 121-130. IEEE, 2010.

[2] Yuan-Shun Dai et al. Uncertainty analysis in software reliability modeling by bayesian analysis with maximum-entropy principle. IEEE Transactions on Software Engineering, 33(11):781-795, 2007.

[3] Indika Meedeniya et al. Evaluating probabilistic models with uncertain model parameters. Software & Systems Modeling, 13(4):1395-1415, 2014.

[4] Lev V Utkin et al. Imprecise reliability: an introductory overview. In Computational intelligence in reliability engineering, pages 261-306. Springer, 2007.

[5] Spyros T Halkidis et al. Architectural risk analysis of software systems based on security patterns. IEEE Transactions on Dependable and Secure Computing, 5(3):129-142, 2008.

[6] Baoding Liu. Why is there a need for uncertainty theory. Journal of Uncertain Systems, 6(1):3-10, 2012.

[7] Simona Bernardi et al. A dependability profile within marte. Software & Systems Modeling, 10 (3):313-336, 2011.

[8] Barbara Kordy et al. Dag-based attack and defense modeling: Dont miss the forest for the attack trees. Computer science review, 13:1-38, 2014.

[9] Abdulaziz Alkussayer et al. Security risk analysis of software architecture based on ahp. In Networked Computing (INC), 2011 The 7th International Conference on, pages 60-67. IEEE, 2011.

[10] Mohamed Almorsy et al. Automated software architecture security risk analysis using formalized signatures. In Proceedings of the 2013 International Conference on Software Engineering, pages 662-671. IEEE Press, 2013.

[11] Zahra Aghajani et al. Security evaluation of an intrusion tolerant web service architecture using stochastic activity networks. In International Conference on Information Security and Assurance, pages 260-269. Springer, 2009.

[12] Ricardo J Rodríguez et al. Modeling and analyzing resilience as a security issue within uml. In Proceedings of the 2nd International Workshop on Software Engineering for Resilient Systems, pages 42-51. ACM, 2010.

[13] Nianhua Yang et al. Modeling and quantitatively predicting software security based on stochastic petri nets. Mathematical and Computer Modeling, 55(1):102-112, 2012.

[14] Vibhu Saujanya Sharma et al. Quantifying software performance, reliability and security: An architecture-based approach. Journal of Systems and Software, 80(4):493-509, 2007.

[15] Gaogao Yan et al. Formal throughput and response time analysis of marte models. In International Conference on Formal Engineering Methods, pages 430-445. Springer, 2014.

[16] Murray Woodside et al. Transformation challenges: from software models to performance models. Software & Systems Modeling, 13(4):1529-1552, 2014.

[17] Thanh-Trung Pham et al. Reliability prediction for component-based software systems with architectural-level fault tolerance mechanisms. In Availability, Reliability and Security (ARES), 2013 Eighth International Conference on, pages 11-20. IEEE, 2013.

[18] Marco Autili et al. Eagle: Engineering software in the ubiquitous globe by leveraging uncertainty. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, pages 488-491. ACM, 2011.

[19] Indika Meedeniya et al. Architecture-based reliability evaluation under uncertainty. In Proceedings of the joint ACM SIGSOFT conference QoSA and ACM SIGSOFT symposium-ISARCS on Quality of software architectures-QoSA and architecting critical systems-ISARCS, pages 85-94. ACM, 2011.

[20] Faramarz Khosravi et al. Uncertainty-aware reliability analysis and optimization. In 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 97-102. IEEE, 2015.

[21] Catia Trubiani et al. Model-based performance analysis of software architectures under uncertainty. In Proceedings of the 9th international ACM Sigsoft conference on Quality of software architectures, pages 69-78. ACM, 2013.

[22] William S Jewell. Bayesian extensions to a basic model of software reliability. IEEE Transactions on Software engineering, (12):1465-1471, 1985.

[23] PV Suresh et al. Uncertainty in fault tree analysis: a fuzzy approach. Fuzzy sets and Systems, 83(2):135-141, 1996.

[24] YA Mahmood et al. Fuzzy fault tree analysis: A review of concept and application. International Journal of System Assurance Engineering and Management, 4(1):19-32, 2013.

[25] DK Mohanta et al. Importance and uncertainty analysis in software reliability assessment of computer relay. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 225(1):50-61, 2011.

[26] Ricardo J Rodríguez et al. Modeling security of critical infrastructures: a survivability assessment. The Computer Journal, 58(10):2313-2327, 2015.

[27] Glenn Shafer et al. A mathematical theory of evidence, volume 1. Princeton university press Princeton, 1976.

[28] Kari Sentz et al. Combination of evidence in Dempster-Shafer theory, volume 4015. Citeseer,


[29] Scott Ferson et al. Dependence in probabilistic modeling, dempster-shafer theory, and probability bounds analysis. Sandia National Laboratories, Report No. SAND2004-3072, 2004.

[30] Guttorm Sindre et al. Eliciting security requirements with misuse cases. Requirements engineering, 10(1):34-44, 2005.

[31] Guttorm Sindre. Mal-activity diagrams for capturing attacks on business processes. In International Working Conference on Requirements Engineering: Foundation for Software Quality, pages 355-366. Springer, 2007.

[32] Mohammad Jabed Morshed Chowdhury. To wards security risk-oriented mal activity diagram. International Journal of Computer Applications, 56(10), 2012.

[33] Sébastien Gérard et al. 19 papyrus: A uml2 tool for domain-specific language modeling. In Model-Based Engineering of Embedded Real-Time Systems, pages 361-368. Springer, 2010.

[34] Abel Gmez et al. Dice profiles, 2015. http://dice-project.github.io/DICE-Profiles.

[35] Ali Sedaghatbaf. Sqme tool, 2015. http://twcl.iust.ac.ir/projects/sqme.html.

[36] George J Klir. Uncertainty and information: foundations of generalized information theory. John Wiley & Sons, 2005.