Usability analysis of smart assistants in the scope of human-computer interaction: Microsoft Copilot example

Abstract

Microsoft Copilot is an AI-based assistant designed to help users perform various tasks. However, the adoption and effective use of such systems are related to usability and user experience. Human-Computer Interaction (HCI) developments aim to make AI systems more intuitive, accessible and user-friendly. In this study, the usability of Microsoft Copilot was examined in terms of effectiveness and efficiency, and the relationship between usability satisfaction and AI acceptance was analysed. The study was conducted with 20 voluntary university students, using quantitative and qualitative methods. Participants completed tasks of varying difficulty using Copilot and completed a satisfaction survey and an AI acceptance scale. The findings indicate that Copilot is generally user-friendly and that users exhibit high levels of satisfaction. Additionally, a strong and significant relationship was found between usability satisfaction and the AI acceptance dimensions of effort expectancy, performance expectancy and facilitating conditions. Users found Copilot's content creative, considered its helpful functionality and assessed its interface as accessible. However, some improvements were deemed necessary, including enhancing motivation, ensuring operational consistency and addressing data security concerns. In this context, it is recommended that Copilot focuses on performance-enhancing improvements and security measures to enhance the user experience further.