Mutation analysis, which introduces artificial defects into software systems, is the basis of mutation testing, a technique widely applied to evaluate and enhance the quality of test suites. However, despite the deep analogy between tests and formal proofs, mutation analysis has seldom been considered in the context of deductive verification. We propose mutation proving, a technique for analyzing verification projects that use proof assistants. We implemented our technique for the Coq proof assistant in a tool dubbed mCoq. mCoq applies a set of mutation operators to Coq definitions of functions and datatypes, inspired by operators previously proposed for functional programming languages. mCoq then checks proofs of lemmas affected by operator application. To make our technique feasible in practice, we implemented several optimizations in mCoq such as parallel proof checking. We applied mCoq to several medium and large scale Coq projects, and recorded whether proofs passed or failed when applying different mutation operators. We then qualitatively analyzed the mutants, and found several examples of weak and incomplete specifications. For our evaluation, we made many improvements to serialization of Coq code and even discovered a notable bug in Coq itself, all acknowledged by developers. We believe mCoq can be useful both to proof engineers for improving the quality of their verification projects and to researchers for evaluating proof engineering techniques.
Wed 13 NovDisplayed time zone: Tijuana, Baja California change
13:40 - 15:20 | Verification and Bug DetectionDemonstrations / Research Papers at Cortez 1 Chair(s): Raghavan Komondoor Indian Institute of Science, Bangalore | ||
13:40 20mTalk | Mutation Analysis for Coq Research Papers Ahmet Celik The University of Texas at Austin, Karl Palmskog University of Texas at Austin, Marinela Parovic The University of Texas at Austin, Emilio Jesús Gallego Arias MINES ParisTech, Milos Gligoric The University of Texas at Austin | ||
14:00 20mTalk | Verifying Arithmetic in Cryptographic C Programs Research Papers Jiaxiang Liu Shenzhen University, Xiaomu Shi Shenzhen University, Ming-Hsien Tsai Academia Sinica, Taiwan, Bow-Yaw Wang Academia Sinica, Bo-Yin Yang Academia Sinica Pre-print | ||
14:20 20mTalk | Model checking embedded control software using OS-in-the-loop CEGAR Research Papers Pre-print | ||
14:40 20mTalk | Get rid of inline assembly through verification-oriented lifting Research Papers Frédéric Recoules CEA LIST, Sébastien Bardin CEA LIST, Richard Bonichon CEA LIST, Laurent Mounier Université Grenoble Alpes, Marie-Laure Potet Université Grenoble Alpes DOI Pre-print | ||
15:00 10mDemonstration | VeriAbs : Verification by Abstraction and Test Generation Demonstrations Mohammad Afzal Tata Cosultancy Services, A Asia Tata Cosultancy Services, Avriti Chauhan Tata Cosultancy Services, Bharti Chimdyalwar Tata Consultancy Services, Priyanka Darke Tata Consultancy Services, Advaita Datar Tata Consultancy Services Ltd, Shrawan Kumar Tata Cosultancy Services, R Venkatesh Tata Research Development and Design Centre | ||
15:10 10mDemonstration | SGUARD: A Feature-based Clustering Tool for Effective Spreadsheet Defect Detection Demonstrations Da Li State Key Lab. for Novel Software Tech. and Dept. of Comp. Sci. and Tech., Nanjing University, Nanjing, China, Huiyan Wang State Key Lab. for Novel Software Tech. and Dept. of Comp. Sci. and Tech., Nanjing University, Nanjing, China, Chang Xu Nanjing University, Ruiqing Zhang Search Tech. Center Asia, Microsoft, Suzhou, China, Shing-Chi Cheung Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Xiaoxing Ma State Key Lab. for Novel Software Tech. and Dept. of Comp. Sci. and Tech., Nanjing University |