Blogs (1) >>
ASE 2019
Sun 10 - Fri 15 November 2019 San Diego, California, United States
Thu 14 Nov 2019 10:40 - 11:00 at Cortez 2&3 - Deep Models Chair(s): Nazareno Aguirre

Game testing has been long recognized as a notoriously challenging task, which mainly relies on manual playing and scripting based testing in game industry. Even until recently, automated game testing still remains to be largely untouched niche. A key challenge is that game testing often requires to play the game as a sequential decision process. A bug may only be triggered until completing certain difficult intermediate tasks, which requires a certain level of intelligence. The recent success of deep reinforcement learning (DRL) sheds light on advancing automated game testing, without human competitive intelligent support. However, the existing DRLs mostly focus on winning the game rather than game testing. To bridge the gap, in this paper, we first perform an in-depth analysis of 1349 real bugs from four real-world commercial game products. Based on this, we propose four oracles to support automated game testing, and further propose Wuji, an on-the-fly game testing framework, which leverages evolutionary algorithm, DRL and multi-objective optimization to perform automatic game testing. Wuji balances between winning the game and exploring the space of the game. Winning the game allows the agent to make progress in the game, while space exploration increases the possibility of discovering bugs. We conduct a large-scale evaluation on a simple game and two popular commercial games. The results demonstrate the effectiveness of Wuji in exploring space and detecting bugs. Moreover, Wuji found 3 previously unknown bugs, which have been confirmed by the developers, in the commercial games.

Thu 14 Nov

Displayed time zone: Tijuana, Baja California change

10:40 - 12:20
Deep ModelsResearch Papers / Demonstrations at Cortez 2&3
Chair(s): Nazareno Aguirre Dept. of Computer Science FCEFQyN, University of Rio Cuarto
10:40
20m
Talk
Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement LearningACM SIGSOFT Distinguished Paper Award
Research Papers
Yan Zheng Tianjin University, Xiaofei Xie Nanyang Technological University, Ting Su ETH Zurich, Lei Ma Kyushu University, Jianye Hao Tianjin University, Zhaopeng Meng Tianjin University, Yang Liu Nanyang Technological University, Singapore, Ruimin Shen Fuxi AI Lab in Netease, Yinfeng Chen Fuxi AI Lab in Netease, Changjie Fan Fuxi AI Lab in Netease
Link to publication Pre-print
11:00
20m
Talk
A Study of Oracle Approximations in Testing Deep Learning Libraries
Research Papers
Mahdi Nejadgholi Concordia University, Jinqiu Yang Concordia University, Montreal, Canada
11:20
20m
Talk
Property Inference for Deep Neural Networks
Research Papers
Divya Gopinath Carnegie Mellon University, Hayes Converse The University of Texas at Austin, Corina S. Pasareanu Carnegie Mellon University Silicon Valley, NASA Ames Research Center, Ankur Taly Google
11:40
20m
Talk
An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms
Research Papers
Qianyu Guo Tianjin University, Sen Chen Nanyang Technological University, Singapore, Xiaofei Xie Nanyang Technological University, Lei Ma Kyushu University, Qiang Hu Kyushu University, Japan, Hongtao Liu Tianjin University, Yang Liu Nanyang Technological University, Singapore, Jianjun Zhao Kyushu University, Li Xiaohong TianJin University
Pre-print
12:00
10m
Demonstration
DeepMutation++: a Mutation Testing Framework for Deep Learning Systems
Demonstrations
Qiang Hu Kyushu University, Japan, Lei Ma Kyushu University, Xiaofei Xie Nanyang Technological University, Bing Yu Kyushu University, Japan, Yang Liu Nanyang Technological University, Singapore, Jianjun Zhao Kyushu University
12:10
10m
Demonstration
DeepHunter: A Coverage-Guided Fuzzer for Deep Neural Networks
Demonstrations
Xiaofei Xie Nanyang Technological University, Hongxu Chen Nanyang Technological University, Yi Li Nanyang Technological University, Lei Ma Kyushu University, Yang Liu Nanyang Technological University, Singapore, Jianjun Zhao Kyushu University