Tue 5 Dec 2023 17:45 - 18:00 at Golden Gate C2 - Machine Learning II Chair(s): Iftekhar Ahmed

Analyzing reliability of Machine Vision Components (MVC) against scene changes (such as rain or fog) in their operational environment is crucial for safety-critical applications. Safety analysis relies on the availability of precisely specified and, ideally, machine-verifiable requirements. A state-of-the-art reliability framework ICRAF developed machine-verifiable requirements obtained using human performance data. However, ICRAF is limited to analyzing reliability of MVCs solving simple vision tasks, such as image classification. Yet, many real-world safety-critical systems require solving more complex vision tasks, such as object detection and instance segmentation. Fortunately, many complex vision tasks (which we call ``c-tasks'') can be represented as a sequence of simple vision subtasks. For instance, object detection can be decomposed as object localization followed by classification. Based on this fact, in this paper, we show that the analysis of c-tasks can also be decomposed as a sequential analysis of their simple subtasks, which allows us to apply existing techniques for analyzing simple vision tasks. % for c-task analysis. Specificallly, we propose a modular reliability framework, DecompoVision, that decomposes: (1) the problem of solving a c-task, (2) the reliability requirements, and (3) the reliability analysis, and, as a result, provides deeper insights into MVC reliability. DecompoVision extends ICRAF to handle complex vision tasks and enables reuse of existing artifacts across different c-tasks. We capture new reliability gaps by checking our requirements on 13 widely used object detection MVCs, and, for the first time, benchmark segmentation MVCs.

Tue 5 Dec

Displayed time zone: Pacific Time (US & Canada) change

16:00 - 18:00
Machine Learning IIResearch Papers / Ideas, Visions and Reflections at Golden Gate C2
Chair(s): Iftekhar Ahmed University of California at Irvine
16:00
15m
Talk
[Remote] Compatibility Issues in Deep Learning Systems: Problems and Opportunities
Research Papers
Jun Wang Nanjing University of Aeronautics and Astronautics, Nanjing, China, Guanping Xiao Nanjing University of Aeronautics and Astronautics, China, Shuai Zhang Nanjing University of Aeronautics and Astronautics, China, Huashan Lei Nanjing University of Aeronautics and Astronautics, China, Yepang Liu Southern University of Science and Technology, Yulei Sui University of New South Wales, Australia
DOI Pre-print Media Attached
16:15
15m
Talk
[Remote] An Extensive Study on Adversarial Attack against Pre-trained Models of Code
Research Papers
Xiaohu Du Huazhong University of Science and Technology, Ming Wen Huazhong University of Science and Technology, Zichao Wei Huazhong University of Science and Technology, Shangwen Wang National University of Defense Technology, Hai Jin Huazhong University of Science and Technology
Media Attached
16:30
15m
Talk
Can Machine Learning Pipelines Be Better Configured?
Research Papers
Yibo Wang Northeastern University, Ying Wang Northeastern University, Tingwei Zhang Northeastern University, Yue Yu National University of Defense Technology, Shing-Chi Cheung Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hai Yu Software College, Northeastern University, Zhiliang Zhu Software College, Northeastern University
Media Attached
16:45
15m
Talk
Towards Feature-Based Analysis of the Machine Learning Development Lifecycle
Ideas, Visions and Reflections
Boyue Caroline Hu University of Toronto, Marsha Chechik University of Toronto
Media Attached
17:00
15m
Talk
Fix Fairness, Don’t Ruin Accuracy: Performance Aware Fairness Repair using AutoML
Research Papers
Giang Nguyen Dept. of Computer Science, Iowa State University, Sumon Biswas Carnegie Mellon University, Hridesh Rajan Dept. of Computer Science, Iowa State University
Pre-print Media Attached
17:15
15m
Talk
BiasAsker: Measuring the Bias in Conversational AI System
Research Papers
Yuxuan Wan The Chinese University of Hong Kong, Wenxuan Wang Chinese University of Hong Kong, Pinjia He The Chinese University of Hong Kong, Shenzhen, Jiazhen Gu Chinese University of Hong Kong, Haonan Bai The Chinese University of Hong Kong, Michael Lyu The Chinese University of Hong Kong
Media Attached
17:30
15m
Talk
Pitfalls in Experiments with DNN4SE: An Analysis of the State of the Practice
Research Papers
Sira Vegas Universidad Politecnica de Madrid, Sebastian Elbaum University of Virginia
Media Attached
17:45
15m
Talk
DecompoVision: Reliability Analysis of Machine Vision Components Through Decomposition and Reuse
Research Papers
Boyue Caroline Hu University of Toronto, Lina Marsso University of Toronto, Nikita Dvornik Waabi, Huakun Shen University of Toronto, Marsha Chechik University of Toronto
Media Attached