Blogs (1) >>
ASE 2019
Sun 10 - Fri 15 November 2019 San Diego, California, United States
Wed 13 Nov 2019 16:20 - 16:40 at Cortez 2&3 - API and Renaming Chair(s): Massimiliano Di Penta

High quality method names are critical for the readability and maintainability of programs. However, constructing concise and consistent method names is often challenging, especially for inexperienced developers. To this end, advanced machine learning techniques have been recently leveraged to recommend method names automatically for given method bodies/implementation. Recent large-scale evaluations also suggest that such approaches are accurate. However, little is known about where and why such approaches work or don’t work. To figure out the state of the art as well as the rationale for the success/failure, in this paper we conduct an empirical study on the state-of-the-art approach code2vec. We assess code2vec on a new dataset with more realistic settings. Our evaluation results suggest that although switching to new dataset does not significantly influence the performance, more realistic settings do significantly reduce the performance of code2vec. Further analysis on the successfully recommended method names also reveals the following findings: 1) around half (48.3%) of the accepted recommendations are made on getter/setter methods; 2) a large portion (19.2%) of the successfully recommended method names could be copied from the given bodies. To further validate its usefulness, we ask developers to manually score the difficulty in naming methods they developed. Code2vec is then applied to such manually scored methods to evaluate how often it works in need. Our evaluation results suggest that code2vec rarely works when it is really needed. Finally, to intuitively reveal the state of the art and to investigate the possibility of designing simple and straightforward alternative approaches, we propose a heuristics based approach to recommending method names. Evaluation results on large-scale dataset suggest that this simple heuristics-based approach significantly outperforms the state-of-the-art machine learning based approach, improving precision and recall by 65.25% and 22.45%, respectively. The comparison suggests that machine learning based recommendation of method names still has a long way to go.

Wed 13 Nov
Times are displayed in time zone: (GMT-07:00) Tijuana, Baja California change

ase-2019-paper-presentations
16:00 - 17:40: Papers - API and Renaming at Cortez 2&3
Chair(s): Massimiliano Di PentaUniversity of Sannio
ase-2019-papers16:00 - 16:20
Talk
Xiaodong GuThe Hong Kong University of Science and Technology, Hongyu ZhangThe University of Newcastle, Sunghun KimHong Kong University of Science and Technology
Pre-print
ase-2019-papers16:20 - 16:40
Talk
Lin Jiangbeijing university of posts and telecommunication, Hui LiuBeijing Institute of Technology, He JiangSchool of Software, Dalian University of Technology
Link to publication Pre-print
ase-2019-papers16:40 - 17:00
Talk
Daye NamCarnegie Mellon University, Amber HorvathCarnegie Mellon University, Andrew MacveanGoogle, Inc., Brad MyersCarnegie Mellon University, Bogdan VasilescuCarnegie Mellon University
Pre-print
ase-2019-papers17:00 - 17:20
Talk
Jeremy LacomisCarnegie Mellon University, Pengcheng YinCarnegie Mellon University, Edward J. SchwartzCarnegie Mellon University Software Engineering Institute, Miltiadis AllamanisMicrosoft Research, Cambridge, Claire Le GouesCarnegie Mellon University, Graham NeubigCarnegie Mellon University, Bogdan VasilescuCarnegie Mellon University
Pre-print Media Attached
ase-2019-Journal-First-Presentations17:20 - 17:40
Talk
Seonah LeeGyeongsang National University, Rongxin WuDepartment of Computer Science and Engineering, The Hong Kong University of Science and Technology, Shing-Chi CheungDepartment of Computer Science and Engineering, The Hong Kong University of Science and Technology, Sungwon KangKorea Advanced Institute of Science and Technology
Link to publication