Tue 5 Dec 2023 14:30 - 14:45 at Golden Gate C1 - Testing II Chair(s): Brittany Johnson

Mutation testing is a powerful technique for assessing and improv- ing test suite quality that artificially introduces bugs and checks whether the test suites catch them. However, it is also computation- ally expensive and thus does not scale to large systems and projects. One promising recent approach to tackling this scalability prob- lem uses machine learning to predict whether the tests will detect the synthetic bugs, without actually running those tests. However existing predictive mutation testing approaches still misclassify 48% of undetected bugs on a randomly sampled set of mutant-test suite pairs. We propose a novel machine learning approach for predictive mutation testing that simultaneously encodes the source method mutation and test method, capturing key context in the input representation. We use this input representation to leverage recent advances in transformers for machine learning for source code tasks. We show that our approach, MutationBERT, outper- forms the state-of-the-art in both same project and cross project settings, with meaningful improvements in precision, recall and F1 score. We empirically validate our novel input representation, and aggregation approaches for lifting predictions from the test matrix level to the test suite level. Finally, we show that our approach saves up to 10,758 test executions compared to the prior approach, depending on whether the model was trained on same project or cross project data and the size of projects being run.

Tue 5 Dec

Displayed time zone: Pacific Time (US & Canada) change

14:00 - 15:30
14:00
15m
Talk
Statfier: Automated Testing of Static Analyzers via Semantic-preserving Program Transformations
Research Papers
Huaien Zhang Southern University of Science and Technology, The Hong Kong Polytechnic University, Yu Pei Hong Kong Polytechnic University, Junjie Chen Tianjin University, Shin Hwei Tan Concordia University
Media Attached
14:15
15m
Talk
Towards Efficient Record and Replay: A Case Study in WeChat
Industry Papers
Sidong Feng Monash University, Haochuan Lu Tencent, Ting Xiong Tencent Inc., Yuetang Deng Tencent Inc., Chunyang Chen Monash University
DOI Media Attached
14:30
15m
Talk
Contextual Predictive Mutation Testing
Research Papers
Kush Jain Carnegie Mellon University, Uri Alon Carnegie Mellon University, Alex Groce Northern Arizona University, Claire Le Goues Carnegie Mellon University
Media Attached
14:45
15m
Talk
Towards Automated Software Security Testing: Augmenting Penetration Testing through LLMs
Ideas, Visions and Reflections
Media Attached
15:00
7m
Talk
LazyCow: A Lightweight Crowdsourced Testing Tool for Taming Android Fragmentation
Demonstrations
Xiaoyu Sun Australian National University, Australia, Xiao Chen Monash University, Yonghui Liu Monash University, John Grundy Monash University, Li Li Beihang University
Media Attached
15:08
7m
Talk
Rotten Green Tests in Google Test
Industry Papers
DOI Media Attached
15:15
15m
Talk
MuAkka: Mutation Testing for Actor Concurrency in Akka Using Real-World Bugs
Research Papers
Mohsen Moradi Moghadam Oakland University, Mehdi Bagherzadeh Oakland University, Raffi Khatchadourian City University of New York (CUNY) Hunter College, Hamid Bagheri University of Nebraska-Lincoln
Pre-print Media Attached