As Introductory Programming classes continue to grow, it becomes ever more important to ensure that automatic grading is both efficient yet still maintains high accuracy. Specifically, the use of automated testing for this type of grading has become standard, but it is not without fault. Instructor test suites can often misclassify correct student submissions as incorrect or vice versa. To address this discrepancy, we propose a novel approach which adapts the technique of Test Suite Augmentation to an educational setting in order to improve instructor test suite quality. Our approach first partitions student programs into clusters based on Behavioral Equivalence. We then use these clusters to better guide the Pex test-generator in generating a minimal set of tests which can augment an instructor’s suite and improve grading accuracy. We evaluated our approach on an extensive set of student submissions from a CS1 course and were able to find flaws in existing instructor test suites, and then augment these suites to substantially improve their quality.
Tue 12 NovDisplayed time zone: Tijuana, Baja California change
13:40 - 15:20 | Testing and VerificationResearch Papers / Demonstrations / Industry Showcase at Cortez 1 Chair(s): Weihang Wang University at Buffalo, SUNY | ||
13:40 20mTalk | Systematically Covering Input Structure Research Papers Nikolas Havrikov CISPA Helmholtz Center for Information Security, Andreas Zeller CISPA Helmholtz Center for Information Security Pre-print | ||
14:00 20mTalk | SEGATE: Unveiling Semantic Inconsistencies between Code and Specification of String Inputs Research Papers Pre-print | ||
14:20 20mTalk | Detecting Error-Handling Bugs without Error Specification Input Research Papers Zhouyang Jia National University of Defense Technology, Shanshan Li National University of Defense Technology, Tingting Yu University of Kentucky, Liao Xiangke National University of Defense Technology, China, Ji Wang National University of Defense Technology, Xiaodong Liu National University of Defense Technology, Yunhuai Liu Peking University | ||
14:40 20mTalk | Test Automation and its Limitations Industry Showcase Ahyoung Sung Samsung Electronics, Yangsu Kim Samsung Electronics, Sangjun Kim Samsung Electronics, Jongin Kim Samsung Electronics, Neo Jang Samsung Electronics | ||
15:00 10mTalk | Grading-Based Test Suite Augmentation Research Papers Jonathan Osei-Owusu University of Illinois at Urbana-Champaign, Angello Astorga University of Illinois at Urbana-Champaign, Liia Butler University of Illinois at Urbana-Champaign, Tao Xie Peking University, Geoffrey Challen University of Illinois at Urbana-Champaign | ||
15:10 10mDemonstration | MutAPK: Source-Codeless Mutant Generation for Android Apps Demonstrations Camilo Escobar-Velásquez Universidad de los Andes, Michael Osorio-Riaño Universidad de los Andes, Mario Linares-Vásquez Systems and Computing Engineering Department , Universidad de los Andes , Bogotá, Colombia |