Volume 8 • Issue 2 • PP: 11–18 • 2026
Algorithmic Hallucination in LLM-Generated Metaheuristics: A Reproducibility, Constraint-Violation, and Failure-Mode Audit
Open Access & Copyright
© 2026 The Author(s). Published by ASPG. This article is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
View PDF Abstract
Large language models now routinely produce metaheuristic optimisation code in response to natural-language prompts, yet the structural integrity of such generated implementations has received little systematic scrutiny. Syntactically valid code that misrepresents algorithmic logic or parameter semantics constitutes a form of hallucination specific to optimisation software — one that is difficult to detect without execution-level testing and that can produce catastrophic performance degradation without any visible error signal. This work introduces a five-category taxonomy of algorithmic hallucination covering parameter value hallucination, update-formula omission, boundaryhandling failure, reproducibility failure, and termination-logic error. A controlled audit is conducted by injecting each hallucination type into verified reference implementations of three widely used metaheuristics — particle swarm optimisation, differential evolution, and a genetic algorithm — and evaluating their behaviour across five standard continuous benchmark functions. Across twenty independent trials per condition, parameter hallucination and formula-omission errors produce best-fitness degradation exceeding 108-fold relative to the reference, while boundary-handling failures generate more than 4 000 out-of-bounds constraint violations per trial. Reproducibility failure inflates inter-trial variance by three to four orders of magnitude, and premature-termination errors induce a 100% trial failure rate. The results demonstrate that current LLM-generated metaheuristic code requires structured execution-level validation before deployment, and a practical audit protocol is proposed to support that process. Experimental artefacts, including all source code, generated data, and benchmark results, are released to support reproducible follow-on research.
Keywords
References
[1] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374, 2021.
[2] D. Zan, B. Chen, F. Zhang, D. Lu, B. Wu, B. Guan, W. Yongji, and J.-G. Lou, “Large language models meet NL2Code: A survey,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), Volume 1: Long Papers. Toronto, Canada: Association for Computational Linguistics, 2023, pp. 7443–7464.
[3] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” ACM Computing Surveys, vol. 55, no. 12, pp. 248:1–248:38, 2023.
[4] L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu, “A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,” ACM Transactions on Information Systems, vol. 43, no. 2, pp. 42:1–42:55, 2025.
[5] S. M. T. I. Tonmoy, S. M. M. Zaman, V. Jain, A. Rani, V. Rawte, A. Chadha, and A. Das, “A comprehensive survey of hallucination mitigation techniques in large language models,” arXiv preprint arXiv:2401.01313, 2024.
[6] B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. R. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi, P. Kohli, and A. Fawzi, “Mathematical discoveries from program search with large language models,” Nature, vol. 625, no. 7995, pp. 468–475, 2024.
[7] C. Yang, X. Wang, Y. Lu, H. Liu, Q. V. Le, D. Zhou, and X. Chen, “Large language models as optimizers,” in Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024), 2024.
[8] F. Liu, X. Tong, M. Yuan, X. Lin, F. Luo, Z. Wang, Z. Lu, and Q. Zhang, “Evolution of heuristics: Towards efficient automatic algorithm design using large language model,” in Proceedings of the Forty-First International Conference on Machine Learning (ICML 2024), ser. Proceedings of Machine Learning Research, vol. 235. PMLR, 2024, pp. 32 201–32 223.
[9] H. Ye, J. Wang, Z. Cao, F. Berto, C. Hua, H. Kim, J. Park, and G. Song, “ReEvo: Large language models as hyper-heuristics with reflective evolution,” in Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024, pp. 43 571–43 608.
[10] E. Meyerson, M. J. Nelson, H. Bradley, A. Gaier, A. Moradi, A. K. Hoover, and J. Lehman, “Language model crossover: Variation through few-shot prompting,” ACM Transactions on Evolutionary Learning and Optimization, vol. 4, no. 4, pp. 27:1–27:40, 2024.
[11] J. Lehman, J. Gordon, S. Jain, K. Ndousse, C. Yeh, and K. O. Stanley, “Evolution through large models,” in Handbook of Evolutionary Machine Learning, W. Banzhaf, P. Machado, and M. Zhang, Eds. Singapore: Springer, 2024, pp. 331–366.
[12] F. Liu, Y. Yao, P. Guo, Z. Yang, Z. Zhao, X. Lin, X. Tong, M. Yuan, Z. Lu, Z. Wang, and Q. Zhang, “A systematic survey on large language models for algorithm design,” arXiv preprint arXiv:2410.14716, 2024.
[13] A. Chen, D. M. Dohan, and D. R. So, “EvoPrompting: Language models for code-level neural architecture search,” in Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023, pp. 7787–7817.
[14] S. Liu, C. Chen, X. Qu, K. Tang, and Y.-S. Ong, “Large language models as evolutionary optimizers,” in 2024 IEEE Congress on Evolutionary Computation (CEC). IEEE, 2024, pp. 1–8.
[15] J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. V. Le, and C. Sutton, “Program synthesis with large language models,” arXiv preprint arXiv:2108.07732, 2021.
[16] J. Pineau, P. Vincent-Lamarre, K. Sinha, V. Larivière, A. Beygelzimer, F. d’Alché Buc, E. Fox, and H. Larochelle, “Improving reproducibility in machine learning research,” Journal of Machine Learning Research, vol. 22, no. 164, pp. 1–20, 2021.
[17] K. Rajwar, K. Deep, and S. Das, “An exhaustive review of the metaheuristic algorithms for search and optimization: taxonomy, applications, and open challenges,” Artificial Intelligence Review, vol. 56, no. 11, pp. 13 187–13 257, 2023.
[18] M. Clerc and J. Kennedy, “The particle swarm— explosion, stability, and convergence in a multidimensional complex space,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 1, pp. 58–73, 2002.
[19] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” in Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020, pp. 1877–1901.
[20] OpenAI, “GPT-4 technical report,” arXiv preprint arXiv:2303.08774, 2023.
[21] X. Chen, M. Lin, N. Sch"arli, and D. Zhou, “Teaching large language models to self-debug,” arXiv preprint arXiv:2304.05128, 2023.
Cite This Article
Choose your preferred format
Publisher's Note
The statements, opinions, and data presented in this article are solely those of the author(s) and do not necessarily represent those of ASPG, the journal, or its editors. ASPG and the editors disclaim responsibility for any harm arising from the use of any ideas, methods, instructions, or products described in this article, to the fullest extent permitted by applicable law.