Automatic Test Data Generation in File Format Fuzzers

Document Type : Original Article

Authors

1 Software, Computer Engineering School, Iran University of Science and Technology, Tehran, Iran.

2 Iran University of Science and Technology

Abstract

Fuzzing is a dynamic software testing technique. In this technique with repeated generation and injection of malformed test data to the software under test (SUT), we are looking for the possible errors and vulnerabilities. Files are significant inputs to most real-world applications. Many of test data which are generated for fuzzing such programs are rejected by the parser because they are not in the acceptable format and this results in a low code coverage in the process of fuzz testing. Using the grammatical structure of input files to generate test data leads to increase code coverage. However, often, the grammar extraction is performed manually, which is a time consuming, costly and error-prone task. In this paper, a new method, based on deep neural language models (NLMs), is proposed for automatically learning the file structure and then generating and fuzzing test data. Our experiments demonstrate that the data produced by this method leads to an increase in the code coverage compared to previous test data generation methods. For MuPDF software, which accepts the PDF complex file format as an input, we have more than 1.30 to 12 percent improvement in code coverage than both the intelligence and random methods.

Keywords

References

[1]          B. P. Miller, L. Fredriksen, and B. So, “An empirical study of the reliability of unix utilities,” Commun. ACM, vol. 33, no. 12, pp. 32–44, 1990.##
[2]          B. P. Miller et al., “Fuzz revisited: a  re-examination of the reliability of unix utilities and services,” 1995.#3
[3]          J. E. Forrester and B. P. Miller, “An empirical study of the robustness of  Windows NT applications using random testing,” Proc. 4th USENIX Wind. Syst. Symp., no. August, pp.   59–68, 2000.##
[4]          B. P. Miller, G. Cooksey, and F. Moore, “An empirical study of the robustness of MacOS applications using random testing,” Proc. 1st Int. Work. Random Testing, RT’06, vol. 2006, no. March 2017, pp. 46–54, 2006.##
[5]          G. Evron and N. Rathaus, “Open source fuzzing tools,” 2007.##
[6]          S. Rawat, V. Jain, A. Kumar, L. Cojocar, C. Giuffrida, and H. Bos, “VUzzer: application-aware evolutionary fuzzing,” In Proceedings of the Network and Distributed System Security Symposium (NDSS), 2017.##
[7]          M. Zalewsky, “American fuzzy lop,” [Online]. Available: http://lcamtuf.coredump.cx/afl/. [Accessed: 11-Oct-2017]. 2013.##
[8]          P. Godefroid, H. Peleg, and R. Singh, “Learn&Fuzz: machine learning for input fuzzing,” In Proceedings of the 32Nd IEEE/ACM International Conference on Automated Software Engineering, pp. 50–59, 2017.##
[9]          I. Sutskever, O. Vinyals, and Q. V Le, “Sequence to sequence learning with neural networks,” In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., pp. 3104–3112, 2014.##
[10]        K. Cho et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734, 2014.##
[11]        D. Jurafsky and J. H. Martin, “Speech and language processing (second edition),” Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 2009.##
[12]        T. Mikolov, M. Karafiát, L. Burget, J. Cernocký, and S. Khudanpur, “Recurrent neural network based language model,” In Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010, vol. 2, pp. 1045–1048, 2010.##
[13]        M. Sutton, A. Greene, and P. Amini, “Fuzzing brute force vulnerability discovery,” 1st ed. Addison-Wesley, 2007.##
[14]        A. S. Incorporated, “PDF reference, version 1.7,” no. November, Adobe, 2006.##
[15]        “MuPDF,” [Online]. Available: https://mupdf.com/. [Accessed: 27-Jul-2018]. 2018.##
[16]        A. Takanen, J. D. Demott, and C. Miller, “Fuzzing for Software Security Testing and Quality Assurance,” 2nd ed. Norwood, MA, USA: Artech House, Inc., 2018.##
[17]        C. Chen, B. Cui, J. Ma, R. Wu, J. Guo, and W. Liu, “A systematic review of fuzzing techniques,” Comput. Secur., vol. 75, pp. 118–137, 2018.##
[18]        A. Kettunen, “Test harness for web browser fuzz testing,” University of Oulu, 2014.##
[19]        R. Mcnally, K. Yiu, and D. Grove, “Fuzzing : the state of the art,” DSTO Def. Sci. Technol. Organ., p. 55, 2012.##
[20]        P. Godefroid, A. Kiezun, and M. Y. Levin, “Grammar-based whitebox fuzzing,” ACM SIGPLAN Not., vol. 43, no. 6, p. 206, 2008.##
[21]        S. M. Yaghoubi, “Design and implementation fuzzer to determine web browser vulnerabilities,” Iran University of Science and Technology, School of Computer Engineering, 2013. (In Persian)##
[22]        S. Amini, “Design and implementation of test data generation method for software vulnerability detection,” Iran University of Science and Technology, School of Computer Engineering, 2016. (In Persian)##
[23]   

Files

History

  • Receive Date: 05 September 2020
  • Accept Date: 05 September 2020

Share

How to cite

Statistics