Utilization of Artificial Intelligence (AI) Chatbots in Improving Public Services: A Meta-Analysis Study

Abstract

AI chatbots have emerged as a transformative tool in public service delivery. This study aims to conduct a systematic review and meta-analysis of existing literature to assess the effectiveness of AI chatbots in improving efficiency, response time and user satisfaction in various public service contexts. A comprehensive literature search was conducted on the Scopus database, limiting studies published between 2018 and 2024. Inclusion criteria included quantitative studies that evaluated the impact of AI chatbots on at least one of three outcome variables: efficiency, response time, or user satisfaction. Data were extracted and effect sizes (in this case Standardized Mean Difference - SMD) were calculated for each study. Moderator analysis was conducted to investigate the influence of the type of public service, the complexity of the chatbot's tasks, the type of AI, and the level of human interaction on the effectiveness of the chatbot. Meta-analysis of 30 studies (N = 9,380) shows that AI chatbots have a significant positive effect on the efficiency of public services (SMD = 0.35, 95% CI [0.25, 0.45]), reducing response time (SMD = -0.40, 95% CI [-0.50, -0.30]), and increased user satisfaction (SMD = 0.50, 95% CI [0.40, 0.60]). Moderator analysis revealed that AI chatbots were more effective in healthcare and for simple tasks. Machine learning-based chatbots also show higher effectiveness than rule-based chatbots. In conclusion, AI chatbots offer significant potential to improve various aspects of public services. However, their effectiveness varies depending on the implementation context. These findings provide valuable empirical evidence for policymakers and practitioners to effectively design and implement AI chatbots in public services.