Exploring Output-Based Coverage for Testing PHP Web Applications
In software testing, different testers focus on different aspects of the software such as functionality, performance, design, and other attributes. While many tools and coverage metrics exist to support testers at the code level, not much support is targeted for testers who want to inspect the output of a program such as a dynamic web application. To support this category of testers, we propose a family of output-coverage metrics (similar to statement, branch, and path coverage metrics on code) that measure how much of the possible output has been produced by a test suite and what parts of the output are still uncovered. To do that, we first approximate the output universe using our existing symbolic execution technique. Then, given a set of test cases, we map the produced outputs onto the output universe to identify the covered and uncovered parts and compute output-coverage metrics. In our empirical evaluation on seven real-world PHP web applications, we show that selecting test cases by output coverage is more effective at identifying presentation faults such as HTML validation errors and spelling errors than selecting test cases by traditional code coverage. In addition, to help testers understand output coverage and augment test cases, we also develop a tool called WebTest that displays the output universe in one single web page and allows testers to visually explore covered and uncovered parts of the output.
Tue 12 NovDisplayed time zone: Tijuana, Baja California change
10:40 - 12:20 | Testing and CoverageResearch Papers / Demonstrations / Journal First Presentations / Industry Showcase at Cortez 1 Chair(s): Jonathan Bell George Mason University | ||
10:40 20mTalk | Automatic Self-Validation for Code Coverage Profilers Research Papers Yibiao Yang Huazhong University of Science and Technology, Yanyan Jiang Nanjing University, Zhiqiang Zuo Nanjing University, China, Yang Wang Nanjing University, Hao Sun Unaffiliated, Hongmin Lu Nanjing University, Yuming Zhou Nanjing University, Baowen Xu Nanjing University Pre-print | ||
11:00 20mTalk | Efficient Test Generation Guided by Field Coverage Criteria Research Papers Ariel Godio Dept. of Software Engineering Instituto Tecnológico de Buenos Aires, Valeria Bengolea Dept. of Computer Science FCEFQyN, University of Rio Cuarto, Pablo Ponzio Dept. of Computer Science FCEFQyN, University of Rio Cuarto, Nazareno Aguirre Dept. of Computer Science FCEFQyN, University of Rio Cuarto, Marcelo F. Frias Dept. of Software Engineering Instituto Tecnológico de Buenos Aires | ||
11:20 20mTalk | Exploring Output-Based Coverage for Testing PHP Web Applications Journal First Presentations Hung Viet Nguyen Google LLC, USA, Hung Dang Phan ECpE Department, Iowa State University, Christian Kästner Carnegie Mellon University, Tien N. Nguyen University of Texas at Dallas Link to publication | ||
11:40 20mTalk | PHANTA: Diversified Test Code Quality Measurement for Modern Software Development Industry Showcase Media Attached | ||
12:00 10mDemonstration | TestCov: Robust Test-Suite Execution and Coverage Measurement Demonstrations Pre-print Media Attached File Attached | ||
12:10 10mDemonstration | VisFuzz: Understanding and Intervening Fuzzing with Interactive Visualization Demonstrations Chijin Zhou Tsinghua University, Mingzhe Wang Tsinghua University, Jie Liang Tsinghua University, Zhe Liu Nanjing University of Aeronautics and Astronautics, Chengnian Sun Waterloo University, Yu Jiang Tsinghua University |