Volume 11 • Issue 1 • PP: 01–09 • 2026
Task-Conditioned Early Prediction of Navigation Failure in Information Architecture Evaluation
View PDF Abstract
The interaction logs which researchers collected during their information-architecture evaluation process contain detailed proof which shows how users select between successful and unsuccessful navigation routes. The predictive signal displays its initial appearance during task execution yet users exhibit different navigation patterns depending on their current task and interface they are using. The researchers of this study developed an early navigation failure prediction system which uses public interaction data to create task-specific prefix classification models. The study analyzes data from an open dataset which includes 180 participants completing 1800 tasks across six testing conditions that evaluate tree testing and highfidelity prototype navigation. A prefix-structural encoder works together with a regularized task-conditioned logistic model which predicts success based on the first k navigation actions. The researchers assessed model performance through participant-specific validation using three different machine learning techniques which included random forest, extra trees, and gradient boosting. The optimal configuration achieved 0.7833 accuracy, 0.7513 balanced accuracy, 0.8350 F1-score, and 0.7949 ROC–AUC performance at k = 3. The horizon analysis demonstration shows that predictive signals become accessible after users complete their first three actions. The ablation study proves that task conditioning functions as an essential component. The study results demonstrate that early trace analytics enable quick identification of navigation failures in information-architecture research while providing a useful method for customized assessment during usability testing.
Keywords
References
[1] L. Abb and J.-R. Rehse, “Process-related user interaction logs: State of the art, reference model, and objectcentric implementation,” Information Systems, vol. 124, p. 102386, 2024.
[2] A. Esposito, G. Desolda, and R. Lanzilotti, “A dataset of interactions and emotions for website user experience evaluation,” Scientific Data, vol. 12, p. 1794, 2025.
[3] D. H. Jeong, B. K. Jeong, and S. Y. Ji, “Leveraging machine learning to analyze semantic user interactions in visual analytics,” Information, vol. 15, no. 6, p. 351, 2024.
[4] E. Kuric, P. Demcak, and M. Krajcovic, “Validation of information architecture: Cross-methodological comparison of tree testing variants and prototype user testing,” Information and Software Technology, vol. 183, p. 107740, 2025.
[5] I. Atoum, “Measurement of key performance indicators of user experience based on software requirements,” Science of Computer Programming, vol. 226, p. 102929, 2023.
[6] A. Callejo and J. A. Macias, “Enhancing tree testing analysis to improve the usability evaluation of websites,” Behaviour & Information Technology, vol. 45, no. 6, pp. 1–19, 2025.
[7] J. A. Macias and C. R. Borges, “Monitoring and forecasting usability indicators: A business intelligence approach for leveraging user-centered evaluation data,” Science of Computer Programming, vol. 234, p. 103077, 2024.
[8] A. Rebmann and H. van der Aa, “Recognizing task-level events from user interaction data,” Information Systems, vol. 124, p. 102404, 2024.
[9] A. Martinez-Rojas, A. Jimenez-Ramirez, J. G. Enriquez, and H. A. Reijers, “A screenshot-based task mining framework for disclosing the drivers behind variable human actions,” Information Systems, vol. 121, p. 102340, 2024.
[10] R. Hamdani and I. Chihi, “Adaptive human-computer interaction for industry 5.0: A novel concept, with comprehensive review and empirical validation,” Computers in Industry, vol. 168, p. 104268, 2025.
Cite This Article
Choose your preferred format