Full Length Article
DOI: https://doi.org/10.54216/JCHCI.090104
Interaction-Stability Gated Multimodal Learning for Cognitive Friction Detection in Human-Computer Interaction
Cognitive human-computer interaction (HCI) requires systems that do not only estimate whether a user is under high cognitive load, but also detect moments when interaction demands become unstable, disruptive, or cognitively misaligned with the user’s current state. Existing cognitive-load models commonly treat workload estimation as a static classification task, which limits their usefulness for adaptive interfaces. This paper introduces an interaction-stability gated multimodal learning framework for detecting cognitive friction during HCI. The proposed model combines subject-normalized physiological and gaze features with a temporal stability gate that adjusts the contribution of electrocardiography (ECG), electrodermal activity (EDA), electroencephalography (EEG), and gaze streams according to local signal reliability and cognitive-state fluctuation. A Cognitive Friction Index is further proposed to identify transition periods where cognitive load rises sharply or remains unstable across modalities. The study is designed for reproducibility using the public CLARE dataset, which contains multimodal physiological and gaze recordings from participants performing Multi-Attribute Task Battery II (MATB-II) computer-based workload tasks. Baseline evidence from the CLARE benchmark shows that multimodal learning improves cognitive-load estimation, but leave-one-subject-out performance remains lower than random 10-fold validation, indicating a strong personalization challenge. The proposed framework addresses this gap by modeling temporal instability and subject-level calibration rather than only point-level workload labels. The paper contributes a reproducible Cognitive HCI model, a friction oriented interpretation layer, and an adaptive-interface decision mechanism.
Aa Hubur,
Andino Maseleno
visibility
1928
download
2488