Tue 5 Dec 2023 11:30 - 11:45 at Golden Gate C1 - Testing I Chair(s): Marcelo d'Amorim

Mutation testing has been demonstrated to be one of the most powerful fault-revealing tools in the tester’s tool kit. Much previous work implicitly assumed it to be sufficient to re-compute mutant suites per release. Sadly, this makes mutation results inconsistent; mutant scores from each release cannot be directly compared, making it harder to measure test improvement. Furthermore, regular code change means that a mutant suite’s relevance will naturally degrade over time. We measure this degradation in relevance for 143,500 mutants in 4 non-trivial systems finding that, on overage, 52% degrade. We introduce a mutant brittleness measure and use it to audit software systems and their mutation suites. We also demonstrate how consistent-by-construction long-standing mutant suites can be identified with a 10x improvement in mutant relevance over an arbitrary test suite. Our results indicate that the research community should avoid the re-computation of mutant suites and focus, instead, on long-standing mutants, thereby improving the consistency and relevance of mutation testing.

Tue 5 Dec

Displayed time zone: Pacific Time (US & Canada) change

11:00 - 12:30
11:00
15m
Talk
[Remote] CAmpactor: A Novel and Effective Local Search Algorithm for Optimizing Pairwise Covering Arrays
Research Papers
Qiyuan Zhao Beihang University, Chuan Luo Beihang University, Shaowei Cai Institute of Software, Chinese Academy of Sciences, Wei Wu L3S Research Center, Leibniz University Hannover, Germany, Jinkun Lin Seed Math Technology Limited, Hongyu Zhang Chongqing University, Chunming Hu Beihang University
DOI Pre-print Media Attached
11:15
15m
Talk
Accelerating Continuous Integration with Parallel Batch Testing
Research Papers
Emad Fallahzadeh Concordia University, Amir Hossein Bavand Concordia University, Peter Rigby Concordia University; Meta
Pre-print Media Attached
11:30
15m
Talk
Keeping Mutation Test Suites Consistent and Relevant with Long-Standing Mutants
Ideas, Visions and Reflections
Milos Ojdanic University of Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Mark Harman Meta Platforms Inc. and UCL
Media Attached
11:45
15m
Talk
DistXplore: Distribution-guided Testing for Evaluating and Enhancing Deep Learning Systems
Research Papers
Longtian Wang Xi'an Jiaotong University, Xiaofei Xie Singapore Management University, Xiaoning Du Monash University, Australia, Meng Tian Singapore Management University, Qing Guo IHPC and CFAR at A*STAR, Singapore, Yang Zheng TTE Lab, Huawei, Chao Shen Xi’an Jiaotong University
Media Attached
12:00
15m
Talk
Input Distribution Coverage: Measuring Feature Interaction Adequacy in Neural Network Testing
Journal First
Swaroopa Dola University of Virginia, Matthew B Dwyer University of Virginia, Mary Lou Soffa University of Virginia
Media Attached
12:15
15m
Talk
A Unified Framework for Mini-game Testing: Experience on WeChat
Industry Papers
Chaozheng Wang The Chinese University of Hong Kong, Haochuan Lu Tencent, Cuiyun Gao The Chinese University of Hong Kong, Li Zongjie Hong Kong University of Science and Technology, Ting Xiong Tencent Inc., Yuetang Deng Tencent Inc.
DOI Media Attached