Tue 5 Dec 2023 17:45 - 18:00 at Golden Gate C3 - Fault Diagnosis and Root Cause Analysis I Chair(s): Akond Rahman

Benchmarking software testing tools against known bugs is a classic approach to evaluating the tools’ bug finding abilities. However, this approach is difficult to give some clues on the tool-missed bugs to aid diagnosing the testing tools. As a result, heavy and ad hoc manual analysis is needed. In this work, in the setting of GUI testing for Android apps, we introduce an \emph{automata-based trace analysis} approach to tackling the key challenge of manual analysis, i.e., how to analyze the lengthy event traces generated by a testing tool against a missed bug to find the clues. Our \emph{key} idea is that, we model a bug in the form of a finite automaton which captures its bug-triggering traces; and match the event traces generated by the testing tool (which misses this bug) against this automaton to obtain the clues. Specifically, the clues are presented in the form of three designated automata-based coverage values. We apply our approach to enhance Themis, a representative benchmark suite for Android, to aid diagnosing GUI testing tools. Our extensive evaluation on nine state-of-the-art GUI testing tools and the involvement with several tool developers shows that our approach is \emph{feasible} and \emph{useful}. Our approach enables Themis+ (the enhanced benchmark suite) to provide the clues on the tool-missed bugs, and \emph{all} the Themis+’s clues are identical or useful, compared to the manual analysis results of tool developers. Moreover, the clues have helped find several tool weaknesses, which were unknown or unclear before. Based on the clues, two actively-developing industrial testing tools in our study have quickly made several optimizations and demonstrated their improved bug finding abilities. \emph{All} the tool developers give positive feedback on the usefulness and usability of Themis+’s clues.

Tue 5 Dec

Displayed time zone: Pacific Time (US & Canada) change

16:00 - 18:00
Fault Diagnosis and Root Cause Analysis IResearch Papers / Journal First / Industry Papers at Golden Gate C3
Chair(s): Akond Rahman Auburn University
16:00
15m
Talk
[Remote] Nezha: Interpretable Fine-Grained Root Causes Analysis for Microservices on Multi-Modal Observability Data
Research Papers
Guangba  Yu Sun Yat-Sen University, Pengfei Chen Sun Yat-Sen University, Yufeng Li Sun Yat-sen University, Hongyang Chen School of Computer Science and Engineering, Sun Yat-sen University, Xiaoyun Li Sun Yat-sen University, Zibin Zheng Sun Yat-sen University
Pre-print
16:15
15m
Full-paper
[Remote] DiagConfig: Configuration Diagnosis of Performance Violations in Configurable Software Systems
Research Papers
Zhiming Chen Sun Yat-sen University, Pengfei Chen Sun Yat-Sen University, Guangba  Yu Sun Yat-Sen University, Zilong He Sun Yat-Sen University, Genting Mai Sun Yat-sen University, Peipei Wang ByteDance Infrastructure System Lab
Pre-print Media Attached
16:30
15m
Talk
[Remote] Pre-training Code Representation with Semantic Flow Graph for Effective Bug Localization
Research Papers
Yali Du Shandong University, Zhongxing Yu Shandong University
Media Attached
16:45
15m
Talk
[Remote] A Practical Human Labeling Method for Online Just-in-Time Software Defect Prediction
Research Papers
Liyan Song Southern University of Science and Technology, China, Leandro Minku University of Birmingham, Cong Teng Southern University of Science and Technology, Xin Yao Southern University of Science and Technology
Pre-print Media Attached
17:00
15m
Talk
Trace Diagnostics for Signal-Based Temporal Properties
Journal First
Chaima Boufaied University of Ottawa, Claudio Menghi University of Bergamo; McMaster University, Domenico Bianculli University of Luxembourg, Lionel Briand University of Ottawa, Canada / University of Luxembourg, Luxembourg
Media Attached
17:15
15m
Talk
TraceDiag: Adaptive, Interpretable, and Efficient Root Cause Analysis on Large-Scale Microservice Systems
Industry Papers
Ruomeng Ding Microsoft, Chaoyun Zhang Microsoft, Lu Wang Microsoft Research, Yong Xu Microsoft Research, Minghua Ma Microsoft Research, Xiaomin Wu Microsoft, Meng Zhang , Qingjun Chen Microsoft 365, Xin Gao Microsoft 365, Xuedong Gao Microsoft 365, Hao Fan , Saravan Rajmohan Microsoft 365, Qingwei Lin Microsoft, Dongmei Zhang Microsoft Research
DOI Media Attached
17:30
15m
Talk
Triggering Modes in Spectrum-Based Multi-location Fault Localization
Industry Papers
Tung Dao Cvent, Na Meng Virginia Tech, ThanhVu Nguyen George Mason University
DOI Media Attached
17:45
15m
Talk
Automata-based Trace Analysis for Aiding Diagnosing GUI Testing Tools for Android
Research Papers
Enze Ma East China Normal University, Shan Huang East China Normal University, weigang he East China Normal University, Ting Su East China Normal University, Jue Wang Nanjing University, Huiyu Liu East China Normal University, Geguang Pu East China Normal University, Zhendong Su ETH Zurich
Media Attached