A real-time eye movement-based computer interface for people with disabilities

Abstract

It is costly to develop systems that enable individuals exposed to Amyotrophic Lateral Sclerosis and similar diseases that directly affect the neuromotor ability to communicate with the outside world. In this study, a budget friendly, high-accuracy, software-based, gaze-controlled, real-time virtual keyboard approach that can enable these people to communicate effectively is proposed. The proposed application requires only a computer and a webcam and has a user-friendly interface that meets the basic daily needs of individuals with disabilities. Since the proposed system does not require an extra action such as blinking, it makes it possible to use computers in advanced stage patients who cannot blink their eyes. The application which uses a deep learning-based facial landmark detector, determines the letters the user focuses on the screen and converts thoughts into text. The part of the screen that the user focuses on is determined with a new selection approach inspired by the K-Nearest Neighbors algorithm. This approach, which does not require blinking, offers high speed and accuracy. In the tests, a typing speed of 23.33 characters per minute is achieved with an accuracy rate of 95.12%. It is anticipated that the study will increase computer accessibility for disabled individuals with limited mobility and contribute to the development of real-time eye tracking systems. © 2024 Elsevier B.V., All rights reserved.