Wed 6 Dec 2023 16:00 - 16:15 at Golden Gate C3 - Automated Repair II Chair(s): Luciano Baresi

Automated Program Repair (APR) techniques have drawn wide attention from both academia and industry. Meanwhile, one main limitation with the current state-of-the-art APR tools is that patches passing all the original tests are not necessarily the correct ones wanted by developers, i.e., the plausible patch problem. To date, various Patch-Correctness Checking (PCC) techniques have been proposed to address this important issue. However, they are only evaluated on very limited datasets as the APR tools used for generating such patches can only explore a small subset of the search space of possible patches, posing serious threats to external validity to existing PCC studies. In this paper, we construct an extensive PCC dataset (the largest manually labeled PCC dataset to our knowledge) to revisit all state-of-the-art PCC techniques. More specifically, our PCC dataset includes 1,988 patches generated from the recent PraPR APR tool, which leverages highly-optimized bytecode-level patch executions and can exhaustively explore all possible plausible patches within its large predefined search space (including well-known fixing patterns from various prior APR tools). Our extensive study of representative PCC techniques on the new dataset has revealed various surprising findings, including: 1) the assumption made by existing static PCC techniques that correct patches are more similar to buggy code than incorrect plausible patches no longer holds, 2) state-of-the-art learning-based techniques tend to suffer from the dataset overfitting problem, 3) while dynamic techniques overall retain their effectiveness on our new dataset, their performance drops substantially on patches with more complicated changes and 4) the very recent naturalness-based techniques can substantially outperform traditional static techniques and could be a promising direction for PCC. Based on our findings, we also provide various guidelines/suggestions for advancing PCC in the near future.

Wed 6 Dec

Displayed time zone: Pacific Time (US & Canada) change

16:00 - 18:00
Automated Repair IIJournal First / Research Papers at Golden Gate C3
Chair(s): Luciano Baresi Politecnico di Milano
16:00
15m
Talk
A Large-scale Empirical Review of Patch Correctness Checking Approaches
Research Papers
Jun Yang UIUC, Yuehan Wang University of Illinois at Urbana-Champaign, Yiling Lou Fudan University, Ming Wen Huazhong University of Science and Technology, Lingming Zhang University of Illinois at Urbana-Champaign
Media Attached
16:15
15m
Talk
Program Repair Guided by Datalog-Defined Static Analysis
Research Papers
Yu Liu Beijing University of Technology, Sergey Mechtaev University College London, Pavle Subotic Microsoft, Abhik Roychoudhury National University of Singapore
Media Attached
16:30
15m
Talk
SynShine: Improved Fixing of Syntax Errors
Journal First
Toufique Ahmed University of California at Davis, Noah Rose Ledesma UC Davis, Prem Devanbu University of California at Davis
Media Attached
16:45
15m
Talk
Baldur: Whole-Proof Generation and Repair with Large Language Models
Research Papers
Emily First University of California, San Diego, Markus Rabe Google, Talia Ringer University of Illinois at Urbana-Champaign, Yuriy Brun University of Massachusetts
Media Attached
17:00
15m
Talk
KG4CraSolver: Recommending Crash Solutions via Knowledge Graph
Research Papers
Xueying Du Fudan University, Yiling Lou Fudan University, Mingwei Liu Fudan University, Xin Peng Fudan University, Tianyong Yang Fudan University
Pre-print Media Attached
17:15
15m
Talk
[Remote] Automated and Context-Aware Repair of Color-Related Accessibility Issues for Android Apps
Research Papers
Yuxin Zhang Tianjin University, Sen Chen College of Intelligence and Computing, Tianjin University, Lingling Fan College of Cyber Science, Nankai University, Chunyang Chen Monash University, Xiaohong Li Tianjin University
Media Attached
17:30
15m
Talk
[Remote] Semantic Test Repair for Web applications
Research Papers
Xiaofang Qi School of Computer Science and Engineering, Southeast University, Xiang Qian School of Computer Science and Engineering, Southeast University, Yanhui Li Nanjing University
Media Attached