Gonnermann-Müller, Jana | Sahling, Kristina, Haase, Jennifer
Let's Be Realistic: AI-Recommender Use in a Complex Management Setting
Abstract
Labels like "AI-powered" or "Human-Expert" activate mental models and shape user decisions. Yet, the transferability of these labels on performance in complex, realistic tasks needs investigation. This study examines how recommender labeling and human factors (mindset, expertise) impact performance in a complex business management scenario. We conducted an online experiment employing a management dashboard, where participants (N = 395) received recommendations labeled as either Artificial Intelligence (AI) or Human-Expert-generated. Unlike previous research, labeling did not significantly influence task performance. Instead, graph literacy and cognitive load were key predictors of performance. Participants with positive attitudes toward AI found recommendations helpful, but their performance did not improve with their use. Expertise seems to be dominant in AI labeling in this context. These findings highlight the interaction between expertise, mindset, and labeling, advocating for further research investigating in which contexts labeling and human factors critically influence performance when using AI recommendations.
Kategorie | Proceedings |
Autoren | Gonnermann-Müller, Jana; Sahling, Kristina, Haase, Jennifer |
Erscheinungsort | Yokohama, Japan |
Bandtitel | CHI EA '25: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems |
Datum | 04/2025 |
Konferenztitel | CHI Conference on Human Factors in Computing Systems |
DOI | 10.1145/3706599.3720131 |