TY - CHAP
T1 - Automated Test Data Generation for Coverage: Haven't We Solved This Problem Yet?
AU - Lakhotia, Kiran
AU - McMinn, Phil
AU - Harman, Mark
PY - 2009
Y1 - 2009
N2 - Whilst there is much evidence that both concolic and search based testing can outperform random testing, there has been little work demonstrating the effectiveness of either technique with complete real world software applications. As a consequence, many researchers have doubts not only about the scalability of both approaches but also their applicability to production code. This paper performs an empirical study applying a concolic tool, CUTE, and a search based tool, AUSTIN, to the source code of four large open source applications. Each tool is applied 'out of the box'; that is without writing additional code for special handling of any of the individual subjects, or by tuning the tools' parameters. Perhaps surprisingly, the results show that both tools can only obtain at best a modest level of code coverage. Several challenges remain for improving automated test data generators in order to achieve higher levels of code coverage.
AB - Whilst there is much evidence that both concolic and search based testing can outperform random testing, there has been little work demonstrating the effectiveness of either technique with complete real world software applications. As a consequence, many researchers have doubts not only about the scalability of both approaches but also their applicability to production code. This paper performs an empirical study applying a concolic tool, CUTE, and a search based tool, AUSTIN, to the source code of four large open source applications. Each tool is applied 'out of the box'; that is without writing additional code for special handling of any of the individual subjects, or by tuning the tools' parameters. Perhaps surprisingly, the results show that both tools can only obtain at best a modest level of code coverage. Several challenges remain for improving automated test data generators in order to achieve higher levels of code coverage.
UR - http://www.scopus.com/inward/record.url?scp=77949808417&partnerID=8YFLogxK
U2 - 10.1109/TAICPART.2009.15
DO - 10.1109/TAICPART.2009.15
M3 - Conference paper
SN - 978-0-7695-3820-4
T3 - 2009 TESTING: ACADEMIC AND INDUSTRIAL CONFERENCE-PRACTICE AND RESEARCH TECHNIQUES, TAIC PART 2009
SP - 95
EP - 104
BT - Unknown
PB - IEEE COMPUTER SOC
CY - LOS ALAMITOS
T2 - Proceedings of 2009 Testing: Academic and Industrial Conference - Practice and Research Techniques
Y2 - 4 September 2009 through 6 September 2009
ER -