BiasAsker: Measuring the Bias in Conversational AI System
Powered by advanced Artificial Intelligence (AI) techniques, conversational AI systems, such as chatbots and digital assistants, have been widely deployed in daily life. However, such systems may still produce content containing biases and stereotypes, causing potential social problems. Due to the data-driven, black-box nature of modern AI techniques, comprehensively identifying and measuring biases in conversational systems remains a challenging task. Particularly, it is hard to generate inputs that can comprehensively trigger potential bias due to the lack of data containing both social groups as well as biased properties. In addition, modern conversational systems can produce diverse responses (\eg chatting and explanation), which makes existing bias detection methods simply based on the sentiment and the toxicity hardly being adopted. In this paper, we propose BiasAsker, an automated framework to identify and measure social bias in conversational AI systems. To obtain social groups and biased properties, we construct a comprehensive social bias dataset, containing a total of 841 groups and 8,110 biased properties. Given the dataset, BiasAsker automatically generates questions and adopts a novel method based on existence measurement to identify two types of biases (\ie absolute bias and related bias) in conversational systems. Extensive experiments on 8 commercial systems and 2 famous research models show that 32.83% of the questions generated by BiasAsker can trigger biased behaviors in these widely deployed conversational systems. All the code, data, and experimental results have been released to facilitate future research.
Tue 5 DecDisplayed time zone: Pacific Time (US & Canada) change
16:00 - 18:00 | Machine Learning IIResearch Papers / Ideas, Visions and Reflections at Golden Gate C2 Chair(s): Iftekhar Ahmed University of California at Irvine | ||
16:00 15mTalk | [Remote] Compatibility Issues in Deep Learning Systems: Problems and Opportunities Research Papers Jun Wang Nanjing University of Aeronautics and Astronautics, Nanjing, China, Guanping Xiao Nanjing University of Aeronautics and Astronautics, China, Shuai Zhang Nanjing University of Aeronautics and Astronautics, China, Huashan Lei Nanjing University of Aeronautics and Astronautics, China, Yepang Liu Southern University of Science and Technology, Yulei Sui University of New South Wales, Australia DOI Pre-print Media Attached | ||
16:15 15mTalk | [Remote] An Extensive Study on Adversarial Attack against Pre-trained Models of Code Research Papers Xiaohu Du Huazhong University of Science and Technology, Ming Wen Huazhong University of Science and Technology, Zichao Wei Huazhong University of Science and Technology, Shangwen Wang National University of Defense Technology, Hai Jin Huazhong University of Science and Technology Media Attached | ||
16:30 15mTalk | Can Machine Learning Pipelines Be Better Configured? Research Papers Yibo Wang Northeastern University, Ying Wang Northeastern University, Tingwei Zhang Northeastern University, Yue Yu National University of Defense Technology, Shing-Chi Cheung Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hai Yu Software College, Northeastern University, Zhiliang Zhu Software College, Northeastern University Media Attached | ||
16:45 15mTalk | Towards Feature-Based Analysis of the Machine Learning Development Lifecycle Ideas, Visions and Reflections Media Attached | ||
17:00 15mTalk | Fix Fairness, Don’t Ruin Accuracy: Performance Aware Fairness Repair using AutoML Research Papers Giang Nguyen Dept. of Computer Science, Iowa State University, Sumon Biswas Carnegie Mellon University, Hridesh Rajan Dept. of Computer Science, Iowa State University Pre-print Media Attached | ||
17:15 15mTalk | BiasAsker: Measuring the Bias in Conversational AI System Research Papers Yuxuan Wan The Chinese University of Hong Kong, Wenxuan Wang Chinese University of Hong Kong, Pinjia He The Chinese University of Hong Kong, Shenzhen, Jiazhen Gu Chinese University of Hong Kong, Haonan Bai The Chinese University of Hong Kong, Michael Lyu The Chinese University of Hong Kong Media Attached | ||
17:30 15mTalk | Pitfalls in Experiments with DNN4SE: An Analysis of the State of the Practice Research Papers Media Attached | ||
17:45 15mTalk | DecompoVision: Reliability Analysis of Machine Vision Components Through Decomposition and Reuse Research Papers Boyue Caroline Hu University of Toronto, Lina Marsso University of Toronto, Nikita Dvornik Waabi, Huakun Shen University of Toronto, Marsha Chechik University of Toronto Media Attached | ||