Overview
Collocated with ACM KDD 2024, the first workshop on AI Behavioral Sciences (AIBS 2024) calls for broad discussions on the concept of AI behavioral science, which represents an emerging field that seeks to understand, model, and direct how AI behaves. Critical questions include: Do AIs have personalities? How to describe the patterns of AI behaviors? How to quantify the similarity between AI and humans behaviorally? How to conceal the objectives of AI (rather than generating next words) and align them with the distribution of human objectives? How to model and optimize human-AI collaboration? What are the unique challenges in AI behavioral studies (e.g., sensitivity in prompting)? What is the key difference between AI behavioral science and human behavioral science? Do we need to design new experiment methodologies and measurements tailored for AI? What could be the potential applications (e.g., AI agents)?
The workshop aims to create a collaborative and interdisciplinary platform that brings together researchers from different fields, especially generative AI, data mining, and behavioral sciences to discuss these questions. By fostering an open and forward-looking environment, our ultimate goal is to facilitate discussions on the current landscape of AI behavioral science at large. This workshop provides an opportunity for participants to share insights, exchange ideas, and explore innovative approaches in the field.
For more information, please visit the Call for Papers page.
Schedule
We have a half-day program from 2PM to 6PM on Sunday (Aug. 25) in Room 118 (BiColor Foyer).
| Time (PDT) | Agenda |
|---|---|
| 02:00-02:10 PM | Opening Remarks |
| 02:10-03:00 PM | Keynote by Jessie Li Bridging the Gap: AI, Human Behavior, and the Future of Cities |
| 03:00-04:00 PM | In-person Paper Presentations How Different AI Chatbots Behave? Benchmarking Large Language Models in Behavioral Economics Games Assessing AI Utility: The Random Guesser Test for Sequential Decision-Making Systems Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory A Comparison of Large Language Model and Human Performance on Random Number Generation Tasks |
| 04:00-04:50 PM | Poster Session and Coffee Break |
| 04:50-05:20 PM | In-person Paper Presentations Evaluating and Predicting the Quality of Mental Health Support Responses using Large Language Models Adaptive Behavioral AI: Reinforcement Learning to Enhance Pharmacy Services |
| 05:20-05:50 PM | Remote/video Paper Presentations Can Large Language Model Agents Simulate Human Trust Behaviors? Behavioral Bias of Vision-Language Models: A Behavioral Finance View Investigating Context Effects in Similarity Judgements in Large Language Models |
| 05:50-06:00 PM | Closing Remarks |
Keynote Speaker
Jessie Li
Yunqi Academy of Engineering
https://jessielzh.com/
Bridging the Gap: AI, Human Behavior, and the Future of Cities
Abstract: AI is increasingly influencing the dynamics of our cities. From traffic cameras autonomously reading vehicle plates to self-driving cars navigating our streets and intelligent traffic signals optimizing intersections, AI is becoming integral to the fabric of our cities. But how do these AI technologies influence the behavior of city dwellers, and how can we enhance the coordination between AI systems and human activity in these dynamic urban spaces? In this talk, the presenter will draw on extensive research and hands-on experience in applying AI to real-world city challenges. Over recent years, she has worked closely with city practitioners, gaining valuable insights into how AI is actually deployed in cities, the significant bottlenecks that exist, and why there is still much work to be done in advancing AI technologies to solve complex urban problems. She will share her observations and discuss the critical innovations needed to make AI truly effective in our cities.
Bio
Bio: Dr. Zhenhui (Jessie) Li currently serves as the chief scientist at the Yunqi Academy of Engineering, a non-profit institution situated in Hangzhou, China. Prior to this role, she held a tenured associate professor position at Pennsylvania State University. She earned her doctoral degree in Computer Science from the University of Illinois at Urbana-Champaign. Her research has been primarily devoted to advancing computing technologies to unlock the potential of data for cross-disciplinary research, with a specific emphasis on city applications. For further information, you can visit her website at (https://jessielzh.com/).
Accepted Papers
| Paper ID | Title | Link |
|---|---|---|
| 1 | Assessing AI Utility: The Random Guesser Test for Sequential Decision-Making Systems Shun Ide, Allison Blunt, Djallel Bouneffouf |
arXiv |
| 2 | Adaptive Behavioral AI: Reinforcement Learning to Enhance Pharmacy Services Ana Fernández del Río, Michael Brennan Leong, Paulo Saraiva, Ivan Nazarov, Aditya Rastogi, Moiz Hassan, Dexian Tang, África Periáñez |
arXiv |
| 3 | Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory Gordon Dai, Weijia Zhang, Jinhan Li, Siqi Yang, Chidera Ibe, Srihas Rao, Yunze Xiao, Arthur Caetano, Misha Sra |
arXiv |
| 6 | Can Large Language Model Agents Simulate Human Trust Behaviors? Chengxing Xie, Canyu Chen, Feiran Jia, Ziyu Ye, Kai Shu, Adel Bibi, Ziniu Hu, Philip Torr, Bernard Ghanem, Guohao Li |
arXiv |
| 7 | A Comparison of Large Language Model and Human Performance on Random Number Generation Tasks Rachel Harrison |
arXiv |
| 8 | Behavioral Bias of Vision-Language Models: A Behavioral Finance View Yuhang Xiao, Yudi Lin, Ming-Chang Chiu |
link |
| 9 | How Different AI Chatbots Behave? Benchmarking Large Language Models in Behavioral Economics Games Yutong Xie, Yiyao Liu, Zhuang Ma, Lin Shi, Xiyuan Wang, Qiaozhu Mei |
link |
| 10 | Investigating Context Effects in Similarity Judgements in Large Language Models Sagar Uprety, Amit Kumar Jaiswal, Haiming Liu, Dawei Song |
arXiv |
| 12 | Leveraging LLMs to Analyze Attention Allocation in Large-Scale Peer-to-Peer Mental Health Support Networks Teng Ye, Hanson Yan, Jason Huang, Connor Grogan, Qiaozhu Mei, Walter Yuan, Matt Jackson |
link |
Organization
Please contact us through this email address if you have any questions.
Himabindu Lakkaraju
Harvard University
https://himalakkaraju.github.io/
Bio: Himabindu (Hima) Lakkaraju is an assistant professor at Harvard University focusing on explainability, fairness, and robustness of machine learning models. She has also been working with various domain experts in policy and healthcare to understand the real-world implications of explainable and fair ML. Hima has been named as one of the world’s top innovators under 35 by both MIT Tech Review and Vanity Fair. Her research has also received best paper awards at SIAM International Conference on Data Mining (SDM) and INFORMS, and grants from NSF, Google, Amazon, and Bayer. Hima has given keynote talks at various top ML conferences and workshops including CIKM, ICML, NeurIPS, AAAI, and CVPR, and her research has also been showcased by popular media outlets including the New York Times, MIT Tech Review, TIME magazine, and Forbes. More recently, she co-founded the Trustworthy ML Initiative to enable easy access to resources on trustworthy ML and to build a community of researchers/practitioners working on the topic.
Qiaozhu Mei
University of Michigan
https://websites.umich.edu/~qmei/
Bio: Qiaozhu is a professor in the School of Information and the Department of EECS at the University of Michigan. His research focuses on large-scale data mining, machine learning, information retrieval, and natural language processing, with broad applications to networks, Web, and healthcare. Qiaozhu is an ACM distinguished member (2017) and a recipient of the NSF Career Award (2011). His work has received multiple best paper awards at WWW, ICML, KDD, WSDM, and other major conferences. He is the founding director of the master degree of applied data science at the University of Michigan. He has rich experience organizing workshops and related events, including being the General Co-Chair of SIGIR 2018.
Chenhao Tan
University of Chicago
https://www.chenhaot.com/
Bio: Chenhao Tan is an assistant professor of computer science and data science at the University of Chicago, and is also affiliated with the Harris School of Public Policy. He obtained his PhD degree in the Department of Computer Science at Cornell University and bachelor’s degrees in computer science and in economics from Tsinghua University. Prior to joining the University of Chicago, he was an assistant professor at the University of Colorado Boulder and a postdoc at the University of Washington. His research interests include human-centered AI, natural language processing, and computational social science. His work has been covered by many news media outlets, such as the New York Times and the Washington Post. He also won a Sloan research fellowship, an NSF CAREER award, an NSF CRII award, a Google research scholar award, research awards from Amazon, IBM, JP Morgan, and Salesforce, a Facebook fellowship, and a Yahoo! Key Scientific Challenges award.
Jie Tang
Tsinghua University
https://keg.cs.tsinghua.edu.cn/jietang/
Bio: Jie is a Professor of the Department of Computer Science and Technology of Tsinghua University. He is a Fellow of the ACM, a Fellow of AAAI, and a Fellow of the IEEE. His research interests include artificial general intelligence (AGI), data mining, social networks, machine learning and knowledge graph, with an emphasis on designing new algorithms for information and social network mining and designing new paradigms for artificial general intelligence. Similar to Open AI’s GPT serials, Jie, leading a big research team, have designed GLM-130B, ChatGLM, CogView&CogVideo, CodeGeex, toward teaching machines to think like humans. Jie also invented AMiner.org, which has attracted over 30,000,000 users from 220 countries/regions in the world. He has been honored with the SIGKDD Test-of-Time Award for Applied Science (Ten-year Best Paper Award), the 2nd National Award for Science&Technology, NSFC for Distinguished Young Scholar, UK Royal Society-Newton Advanced Fellowship Award, and SIGKDD Service Award. He served as PC Co-Chair of CIKM’16, WSDM’15, Associate General Chair of KDD’18, and the General Co-Chair of WWW’23.
Yutong Xie
University of Michigan
https://yutxie.com/
Bio: Yutong is a Ph.D. candidate in the School of Information at the University of Michigan. She has a general research interest in AI for scientific innovation, AI for creativity, and AI behavioral science. Yutong has published research papers in major conferences and journals at PNAS, WWW, ICLR, AAAI, etc. She co-organized the workshop on Graph Neural Networks for Recommendation and Search at ACM RecSys ’21, and regularly served as a reviewer in AI-related conferences including WWW, KDD, NeurIPS, ICML, AAAI, etc.