A novel way to identify effective test-case in software testing

Document Type : Original Article

Authors

1 Assistant Professor, Imam Hossein University (AS), Tehran, Iran

2 Master's student, Imam Hossein University (AS), Tehran, Iran

Abstract

Test data generation is one of the costly parts of the software testing, which is performed according to the designed test cases. The problem of designing test cases and then generating optimized test data is one of the challenges of the software testing, including the mutation testing technique. mutation testing has the ability to measure the test cases quality and determine the adequate test cases. However, to perform mutation testing, you need a test set that provides the maximize Coverage of source code and thus have the ability to identify the program errors. In this work, we use code coverage techniques to design test cases and automatically generate optimized test data using the meta-heuristic FA-MABC algorithm. The results are a test suite that cover and test the maximum number of source code lines. Such test suite is more likely to identify errors and get a higher score in the mutation testing. In the proposed method to obtain effective test cases, first generated test cases are applied to mutation testing and then effective test cases are extracted using the Extinguished mutation table. The results of the evaluation show that the FA-MABC algorithm reduces the time of the test data generation, and “modified condition / decision coverage”, increases the mutation score.

Keywords

Smiley face

References

[1]     D. Galin, Software quality assurance: from theory to implementation. Pearson education, 2004.
[2]     J. L. Mitchell and R. Black, Advanced Software Testing: Guide to the ISTQB Advanced Certification As an Advanced Technical Test Analyst. Vol. 3, 2nd Edition. Rocky Nook, 2015.
[3]     Y. Jia and M. Harman, "An Analysis and Survey of the Development of Mutation Testing," IEEE Transactions on Software Engineering, vol. 37, no. 5, pp. 649-678, 2010, doi: 10.1109/TSE.2010.62.
[4]     B. Ivona, S. S. Predrag, L. Shuai, and C. Xinwei, "A Hybrid Firefly and Multi-Strategy Artificial Bee Colony Algorithm," International Journal of Computational Intelligence Systems, vol. 13, no. 1, pp. 810-821, 2020/06/23 2020, doi: 10.2991/ijcis.d.200612.001.
[5]     IEEE Std 730-2014, "IEEE Standard for Software Quality Assurance Processes", 2014. doi: 10.1109/IEEESTD.2014.6835311.
[6]     ISO/IEC/IEEE 24765:2017(E), "ISO/IEC/IEEE International Standard - Systems and software engineering--Vocabulary, 2017. doi: 10.1109/IEEESTD.2017.8016712.
[7]     Structured programming. Academic Press Ltd., 1972.
[8]     M. Hamblen. "Killer software: 4 lessons from the deadly 737 MAX crashes." fierceelectronics. https://www.fierceelectronics.com/electronics/killer-software-4-lessons-from-deadly-737-max-crashes (accessed 2022).
[9]     J. Herkert, J. Borenstein, and K. Miller, "The Boeing 737 MAX: Lessons for Engineering Ethics," Science and Engineering Ethics, vol. 26, no. 6, pp. 2957-2974, 2020/12/01 2020, doi: 10.1007/s11948-020-00252-y.
[10]   H. Zhu and P. A. V. Hall, "Test data adequacy measurement," Software Engineering Journal, vol. 8, no. 1, pp. 21-30, 1993. 
[11]   H. Zhu, P. A. V. Hall, and J. H. R. May, "Software unit test coverage and adequacy," ACM Comput. Surv., vol. 29, no. 4, pp. 366–427, 1997, doi: 10.1145/267580.267590.
[12]   B. Choi, "Test adequacy measurement using a combination of criteria," International Journal of Reliability, Quality and Safety Engineering, vol. 7, pp. 191-203, 2000.
[13]   A. P. Mathur, Foundations of software testing, 2E. Pearson Education India, 2013.
[14]   V. Sharma. "What is Software Quality Assurance And Types of Testing?" volumetree. https://www.volumetree.com/what-is-software-quality-assurance-and-types-of-testing/ (accessed 2022).
[15]   "Quality Assurance, Quality Control and Testing — the Basics of Software Quality Management." altexsoft. https://www.altexsoft.com/whitepapers/quality-assurance-quality-control-and-testing-the-basics-of-software-quality-management/ (accessed 2022).
[16]   A. Spillner and T. Linz, Software Testing Foundations, 5th Edition: A Study Guide for the Certified Tester Exam. Rocky Nook, 2021.
[17]   N. T. "Basis Path Testing." binaryterms. https://binaryterms.com/basis-path-testing.html (accessed 2022).
[18]   "Basis Path Testing in Software Testing." geeksforgeeks. https://www.geeksforgeeks.org/basis-path-testing-in-software-testing/ (accessed 2022).
[19]   T. J. McCabe, "A Complexity Measure," IEEE Transactions on Software Engineering, vol. SE-2, no. 4, pp. 308-320, 1976, doi: 10.1109/TSE.1976.233837.
[20]   P. C. Jorgensen and B. DeVries, Software Testing: A Craftsman's Approach, Fifth Edition. Taylor & Francis, 2021.
[21]   R. Lipton, "Fault diagnosis of computer programs," Student report, Carnegie Mellon University, vol. 2, p. 2, 1971.
[22]   A. J. Offutt and R. H. Untch, "Mutation 2000: Uniting the Orthogonal," in Mutation Testing for the New Century, W. E. Wong Ed. Boston, MA: Springer US, 2001, pp. 34-44.
[23]   R. A. DeMillo, R. J. Lipton, and F. G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," Computer, vol. 11, no. 4, pp. 34-41, 1978, doi: 10.1109/C-M.1978.218136.
[24]    R. A. DeMillo, D. S. Guindi, W. M. McCracken, A. J. Offutt, and K. N. King, "An extended overview of the Mothra software testing environment," in Proceedings. Second Workshop on Software Testing, Verification, and Analysis, 19-21 July 1988 1988, pp. 142-151, doi: 10.1109/WST.1988.5369. 
[25]   K. Naik and P. Tripathy, Software Testing and Quality Assurance: Theory and Practice. Wiley-Spektrum, 2008.
[26]   J. Gao, H.-S. Tsao, and Y. Wu, Testing and quality assurance for component-based software. Artech House, 2003.
[27]   B. B. Agarwal, S. P. Tayal, and M. Gupta, Software Engineering and Testing. Jones & Bartlett Learning, 2010.
[28]   F. C. M. Souza, M. Papadakis, V. H. S. Durelli, and M. E. Delamaro, "Test Data Generation Techniques for Mutation Testing: A Systematic Mapping," presented at the Workshop on Experimental Software Engineering (ESELAW), Pucón, Chile, 2014.
[29]   L. T. M. Hanh, N. T. Binh, and K. T. Tung, "Survey on Mutation-based Test Data Generation," International Journal of Electrical and Computer Engineering (IJECE), vol. 5, no. 5, pp. 1164-1173, 2015.
[30]   C. Blum and A. Roli, "Metaheuristics in combinatorial optimization: Overview and conceptual comparison," ACM Comput. Surv., vol. 35, no. 3, pp. 268–308, 2003, doi: 10.1145/937503.937505.
[31]   M. Črepinšek, S.-H. Liu, and M. Mernik, "Exploration and exploitation in evolutionary algorithms: A survey," ACM Comput. Surv., vol. 45, no. 3, p. Article 35, 2013, doi: 10.1145/2480741.2480752.
[32]    N. Tracey, J. Clark, K. Mander, and J. McDermid, "An automated framework for structural test-data generation," in Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239), 13-16 Oct. 1998 1998, pp. 285-288, doi: 10.1109/ASE.1998.732680. 
[33]   Z. Awedikian, K. Ayari, and G. Antoniol, "MC/DC automatic test input data generation," presented at the Proceedings of the 11th Annual conference on Genetic and evolutionary computation, Montreal, Québec, Canada, 2009.
[34]   M. Sharma and B. Pathik, "Crow Search Algorithm with Improved Objective Function for Test Case Generation and Optimization," Intelligent Automation & Soft Computing, vol. 32, no. 2, pp. 1125-1140, 2021.
[35]   J.-C. Lin and P.-L. Yeh, "Automatic test data generation for path testing using GAs," Information Sciences, vol. 131, no. 1, pp. 47-64, 2001/01/01/ 2001, doi: 10.1016/S0020-0255(00)00093-1.
[36]   A. J. Offutt, "Automatic Test Data Generation," PhD Thesis, Georgia Institute of Technology, Atlanta, Georgia, vol. Technical report GIT-ICS 88/28, 1988.
[37]   A. J. Offutt, Z. Jin, and J. Pan, "The dynamic domain reduction procedure for test data generation," Software: Practice and Experience, vol. 29, no. 2, pp. 167-193, 1999.
[38]   M. Papadakis, N. Malevris, and M. Kallia, "Towards automating the generation of mutation tests," presented at the Proceedings of the 5th Workshop on Automation of Software Test, Cape Town, South Africa, 2010.
[39]    D. B. Mishra, R. Mishra, A. A. Acharya, and K. N. Das, "Test Data Generation for Mutation Testing Using Genetic Algorithm," in Soft Computing for Problem Solving, Singapore, J. C. Bansal, K. N. Das, A. Nagar, K. Deep, and A. K. Ojha, Eds., 2019 2019: Springer Singapore, pp. 857-867. 
[40]    S. Rani, H. Dhawan, G. Nagpal, and B. Suri, "Implementing Time-Bounded Automatic Test Data Generation Approach Based on Search-Based Mutation Testing," in Progress in Advanced Computing and Intelligent Engineering, Singapore, C. R. Panigrahi, A. K. Pujari, S. Misra, B. Pati, and K.-C. Li, Eds., 2019 2019: Springer Singapore, pp. 113-122. 
[41]   N. Jatana and B. Suri, "An Improved Crow Search Algorithm for Test Data Generation Using Search-Based Mutation Testing," Neural Processing Letters, vol. 52, no. 1, pp. 767-784, 2020/08/01 2020, doi: 10.1007/s11063-020-10288-7.
[42]    X.-S. Yang, "Firefly Algorithms for Multimodal Optimization," in Stochastic Algorithms: Foundations and Applications, Berlin, Heidelberg, O. Watanabe and T. Zeugmann, Eds., 2009// 2009: Springer Berlin Heidelberg, pp. 169-178. 
[43]   D. Karaboga, "An idea based on honey bee swarm for numerical optimization," Technical report-tr06, Erciyes university, engineering faculty, computer …, 2005.
Electronic and Cyber Defense
Volume 11, Issue 2 - Serial Number 42
No. 42, Summer
July 2023
Pages 103-116

Files

History

  • Receive Date: 09 October 2022
  • Revise Date: 16 April 2023
  • Accept Date: 17 May 2023
  • Publish Date: 22 June 2023

Share

How to cite

Statistics