Tue 5 Dec 2023 17:08 - 17:23 at Golden Gate C1 - Log Analysis and Debugging Chair(s): Yiming Tang

Log parsing, which extracts log templates from semi-structured logs and produces structured logs, is the first and the most critical step in automated log analysis. While existing log parsers have achieved high accuracy on publicly available log datasets, they suffer from two major limitations by design. First, all existing parsers do not natively support hybrid logs that consist of both single-line logs and multi-line logs (\textit{e.g.,} Java Exception and Hadoop Counters). Second, most existing parsers fall short in integrating expert domain knowledge during parsing, which makes them hard to identify ambiguous log templates in complex real-world environments. To address these issues, this paper proposes \textit{Hue}, the first log parsing approach for hybrid logs. Hue can parse both hybrid and single-line logs in an online manner, which effectively leverages both patterns in the incoming log messages and domain knowledge from the experts. Specifically, Hue converts each log message to a sequence of special wildcards using a key casting table and then conducts line aggregation and pattern extraction. In addition, Hue can effectively utilize user feedback via a novel merge-reject strategy, which makes it possible to quickly adapt to complex and changing log templates. We evaluated Hue on three hybrid log datasets and sixteen widely-used single-line log datasets (\textit{i.e.,} Loghub). The results show that Hue achieves an average accuracy of 0.845 on hybrid logs, which largely outperforms the best results (0.563) obtained by existing parsers. Hue also exhibits SOTA accuracy and efficiency on single-line log datasets. Furthermore, Hue has been successfully deployed in a real production environment for daily hybrid log parsing.

Tue 5 Dec

Displayed time zone: Pacific Time (US & Canada) change

16:00 - 18:00
Log Analysis and DebuggingIndustry Papers / Research Papers at Golden Gate C1
Chair(s): Yiming Tang Rochester Institute of Technology
16:00
15m
Talk
[Remote] STEAM: Observability-Preserving Trace Sampling
Industry Papers
Shilin He Microsoft Research, Botao Feng Microsoft, Liqun Li Microsoft Research, Xu Zhang Microsoft Research, Yu Kang Microsoft Research, Qingwei Lin Microsoft, Saravan Rajmohan Microsoft 365, Dongmei Zhang Microsoft Research
DOI Media Attached
16:15
15m
Talk
[Remote] Demystifying Dependency Bugs in Deep Learning Stack
Research Papers
Kaifeng Huang Fudan University, Bihuan Chen Fudan University, Susheng Wu Fudan University, Junming Cao Fudan University, Lei Ma The University of Tokyo / University of Alberta, Xin Peng Fudan University
Media Attached
16:30
15m
Talk
From Point-wise to Group-wise: A Fast and Accurate Microservice Trace Anomaly Detection Approach
Industry Papers
Zhe Xie Tsinghua University, Changhua Pei Computer Network Information Center at Chinese Academy of Sciences, Wanxue Li eBay, USA, Huai Jiang eBay, USA, Liangfei Su eBay, USA, Jianhui Li Computer Network Information Center at Chinese Academy of Sciences, Gaogang Xie Computer Network Information Center at Chinese Academy of Sciences, Dan Pei Tsinghua University
DOI Media Attached
16:45
15m
Talk
Semantic Debugging
Research Papers
Martin Eberlein Humboldt University of Berlin, Marius Smytzek CISPA Helmholtz Center for Information Security, Dominic Steinhöfel CISPA Helmholtz Center for Information Security, Lars Grunske Humboldt-Universität zu Berlin, Andreas Zeller CISPA Helmholtz Center for Information Security
Media Attached
17:00
7m
Talk
Analyzing Microservice Connectivity with Kubesonde
Industry Papers
Jacopo Bufalino Aalto University, Mario Di Francesco Eficode; Aalto University, Tuomas Aura Aalto University
DOI Media Attached
17:08
15m
Talk
[Remote] Hue: A User-Adaptive Parser for Hybrid Logs
Research Papers
Junjielong Xu Chinese University of Hong Kong, Shenzhen, Qiuai Fu Huawei Cloud Computing Technologies CO., LTD., Zhouruixing Zhu Chinese University of Hong Kong, Shenzhen, Yutong Cheng Chinese University of Hong Kong, Shenzhen, zhijing li , Yuchi Ma Huawei Cloud Computing Technologies CO., LTD., Pinjia He The Chinese University of Hong Kong, Shenzhen
Media Attached
17:23
15m
Talk
[Remote] Log Parsing with Generalization Ability under New Log Types
Research Papers
Siyu Yu Guangxi University, Yifan Wu Peking University, Zhijing Li The Chinese University of Hong Kong, Shenzhen, Pinjia He The Chinese University of Hong Kong, Shenzhen, Ningjiang Chen Guangxi University, Changjian Liu Guangxi University
Media Attached