Code coverage as the primitive dynamic program behavior information, is widely adopted to facilitate a rich spectrum of software engineering tasks, such as testing, fuzzing, debugging, fault detection, reverse engineering, and program understanding. Thanks to the widespread applications, it is crucial to ensure the reliability of the code coverage profilers.
Unfortunately, due to the lack of research attention and the existence of testing oracle problem, the coverage profilers are far away from being tested sufficiently. Bugs are still regularly seen in the widely deployed profilers, like gcov and llvm-cov, along with gcc and llvm, respectively.
This paper proposes Cod, a fully automated self-validator for effectively uncovering bugs in the coverage profilers. Cod takes a single profiler and a program (either from a compiler’s test suite or generated randomly) as input and uncovers the bugs by identifying the inconsistency of coverage results from the input program and its equivalent mutated variants whose coverage statistics are expected to be identical.
We evaluated Cod over two of the most well-known code coverage profilers, namely gcov and llvm-cov. Within a fourmonth testing period, a total of 196 potential bugs (123 for gcov, 73 for llvm-cov) are found, among which 23 are confirmed by the developers.
Tue 12 NovDisplayed time zone: Tijuana, Baja California change
10:40 - 12:20 | Testing and CoverageResearch Papers / Demonstrations / Journal First Presentations / Industry Showcase at Cortez 1 Chair(s): Jonathan Bell George Mason University | ||
10:40 20mTalk | Automatic Self-Validation for Code Coverage Profilers Research Papers Yibiao Yang Huazhong University of Science and Technology, Yanyan Jiang Nanjing University, Zhiqiang Zuo Nanjing University, China, Yang Wang Nanjing University, Hao Sun Unaffiliated, Hongmin Lu Nanjing University, Yuming Zhou Nanjing University, Baowen Xu Nanjing University Pre-print | ||
11:00 20mTalk | Efficient Test Generation Guided by Field Coverage Criteria Research Papers Ariel Godio Dept. of Software Engineering Instituto Tecnológico de Buenos Aires, Valeria Bengolea Dept. of Computer Science FCEFQyN, University of Rio Cuarto, Pablo Ponzio Dept. of Computer Science FCEFQyN, University of Rio Cuarto, Nazareno Aguirre Dept. of Computer Science FCEFQyN, University of Rio Cuarto, Marcelo F. Frias Dept. of Software Engineering Instituto Tecnológico de Buenos Aires | ||
11:20 20mTalk | Exploring Output-Based Coverage for Testing PHP Web Applications Journal First Presentations Hung Viet Nguyen Google LLC, USA, Hung Dang Phan ECpE Department, Iowa State University, Christian Kästner Carnegie Mellon University, Tien N. Nguyen University of Texas at Dallas Link to publication | ||
11:40 20mTalk | PHANTA: Diversified Test Code Quality Measurement for Modern Software Development Industry Showcase Media Attached | ||
12:00 10mDemonstration | TestCov: Robust Test-Suite Execution and Coverage Measurement Demonstrations Pre-print Media Attached File Attached | ||
12:10 10mDemonstration | VisFuzz: Understanding and Intervening Fuzzing with Interactive Visualization Demonstrations Chijin Zhou Tsinghua University, Mingzhe Wang Tsinghua University, Jie Liang Tsinghua University, Zhe Liu Nanjing University of Aeronautics and Astronautics, Chengnian Sun Waterloo University, Yu Jiang Tsinghua University | ||