Volume 17 , Issue 2 , PP: 146-166, 2026 | Cite this article as | XML | Html | PDF | Full Length Article
Deepak 1 * , Savita Sheoran 2
Doi: https://doi.org/10.54216/JCIM.170211
The growth of Large Language Models (LLMs) applications has intensified the demand for efficient vector database solutions capable of handling high-dimensional semantic search operations. Contemporary information retrieval systems face significant challenges in processing complex queries across vast knowledge repositories while maintaining contextual accuracy and computational efficiency. This research investigates the optimization potential of vector search implementations in LLMs through comprehensive evaluation using MongoDB Atlas as the primary vector database platform. Traditional keyword-based retrieval methods fail to capture semantic relationships and contextual nuances essential for accurate information extraction in modern AI applications. Vector-based query optimization enables semantic similarity matching, allowing systems to access contextually relevant data or information even when exact keyword matches are absent. But it significantly improving response quality and user experience. The study addresses critical performance bottlenecks in production-scale vector search deployments, where query latency and retrieval accuracy directly impact system usability. Through systematic comparison of traditional text-embedding-ada-002 against the advanced text-embedding-3-small model, we demonstrate substantial performance enhancements across multiple evaluation metrics. Results establish text-embedding-3-small as superior for semantic search applications, while GPT-4o-mini demonstrates optimal faithfulness performance (0.9067) for accuracy-critical deployments.
Vector Search , Large Language Models , MongoDB Atlas , Semantic Search , Natural Language Processing , Vector Databases , Embedding Models
[1] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” in Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901, 2020.
[2] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 4171–4186, 2019.
[3] K. Nassiri and M. A. Akhloufi, “Recent Advances in Large Language Models for Healthcare,” BioMed- Informatics, vol. 4, no. 2, pp. 1097–1143, Apr. 2024, doi: 10.3390/biomedinformatics4020062.
[4] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, pp. 5998–6008, 2017.
[5] L. Wang et al., “A survey on large language model based autonomous agents,” Frontiers in Computer Science, vol. 18, no. 6, p. 186345, Dec. 2024, doi: 10.1007/s11704-024-40231-1.
[6] A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training,” OpenAI Technical Report, 2018.
[7] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” OpenAI Technical Report, 2019.
[8] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023.
[9] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, P. Schuh, K. Shi, S. Tsvyashchenko, J. Maynez, A. Rao, P. Barnes, Y. Tay, N. Shazeer, V. Prabhakaran, E. Reif, N. Du, B. Hutchinson, R. Pope, J. Bradbury, J. Austin, M. Isard, G. Gur-Ari, P. Yin, T. Duke, A. Levskaya, S. Ghemawat, S. Dev, H. Michalewski, X. Garcia, V. Misra, K. Robinson, L. Fedus, D. Zhou, D. Ippolito, D. Luan, H. Lim, B. Zoph, A. Spiridonov, R. Sepassi, D. Dohan, S. Agrawal, M. Omernick, A. M. Dai, T. S. Pillai, M. Pellat, A. Lewkowycz, E. Moreira, R. Child, O. Polozov, K. Lee, Z. Zhou, X. Wang, B. Saeta, M. Diaz, O. Firat, M. Catasta, J. Wei, K. Meier- Hellstern, D. Eck, J. Dean, S. Petrov, and N. Fiedel, “PaLM: Scaling language modeling with pathways,” arXiv preprint arXiv:2204.02311, 2022.
[10] C. L¨ulf, D. M. L. Martins, M. A. V. Salles, Y. Zhou, and F. Gieseke, “Fast Search-by-Classification for Large-Scale Databases Using Index-Aware Decision Trees and Random Forests,” Proceedings of the VLDB Endowment, vol. 16, no. 11, pp. 2845–2857, Jul. 2023, doi: 10.14778/3611479.3611492.
[11] B. Sarmah, B. Hall, R. Rao, S. Patel, S. Pasquali, and D. Mehta, “HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction,” arXiv preprint arXiv:2408.04948, Aug. 2024, doi: 10.48550/arXiv.2408.04948.
[12] L. Huang et al., “A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions,” ACM Transactions on Information Systems, vol. 43, no. 2, pp. 1–55, Mar. 2025, doi: 10.1145/3703155.
[13] L. Tr¨umper, T. Ben-Nun, P. Schaad, A. Calotoiu, and T. Hoefler, “Performance Embeddings: A Similarity-Based Transfer Tuning Approach to Performance Optimization,” in Proceedings of the 37th International Conference on Supercomputing, ICS ’23. New York, NY, USA: Association for Computing Machinery, 2023, pp. 50–62, doi: 10.1145/3577193.3593714.
[14] S. Mohammadi, A. Balador, S. Sinaei, and F. Flammini, “Balancing privacy and performance in federated learning: A systematic literature review on methods and metrics,” Journal of Parallel and Distributed Computing, vol. 192, p. 104918, Oct. 2024, doi: 10.1016/j.jpdc.2024.104918.
[15] OpenAI, “New embedding models and API updates,” OpenAI Technical Blog, January 2024. [Online]. Available: https://openai.com/index/new-embedding-models-and-api-updates/
[16] S. Es, J. James, L. Espinosa-Anke, and S. Schockaert, “RAGAS: Automated Evaluation of Retrieval Augmented Generation,” arXiv preprint arXiv:2309.15217, 2023.
[17] Y. Hu and Y. Lu, “RAG and RAU: A Survey on Retrieval-Augmented Language Models in NLP,” arXiv preprint arXiv:2404.19543, 2024.
[18] S. Wu, “Retrieval-Augmented Generation for Natural Language Processing: A Survey,” arXiv preprint arXiv:2407.13193, 2024.
[19] J. Lin et al., “Vector Search with OpenAI Embeddings: Lucene Is All You Need,” arXiv preprint arXiv:2308.14963, 2023.
[20] AdaSci Team, “MongoDB Atlas Vector Search for RAG Powered LLM Applications,” Technical Article, AdaSci, 2024.
[21] Y. Han, X. Zhang, R. Chen, L. Wu, and Q. Liu, “When Large Language Models Meet Vector Databases: A Survey,” arXiv preprint arXiv:2402.01763, 2024.
[22] MongoDB Community, “Using OpenAI Latest Embeddings in a RAG System With MongoDB,” MongoDB Developer Blog, 2022.
[23] AA. S. Alzubaidi, M. A. A. Al-Khalidi, and S. A. H. Al-Tamimi, “Deep Learning Approaches for Sentiment Analysis: A Comprehensive Review,” Journal of Computer Science and Technology, vol. 39, no. 2, pp. 321–340, 2023, doi: 10.1007/s11390-023-00235-6.
[24] S. Ghane, R. Sawant, G. Supe, and C. Pichad, “LangchainIQ: Intelligent Content and Query Processing,” International Journal of Management, Technology, and Social Sciences, pp. 34–43, Aug. 2024, doi: 10.47992/IJMTS.2581.6012.0360.
[25] A. Hendawi et al., “Distributed NoSQL Data Stores: Performance Analysis and a Case Study,” in 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA: IEEE, Dec. 2018, pp. 1937– 1944, doi: 10.1109/BigData.2018.8622544.
[26] G. V. N. Priyanka, S. Vasavi, and A. Anu Gokhale, “Evaluation of Performance Metrics in GeoRediSpark Framework for GeoSpatial Query Processing,” in Advances in Decision Sciences, Image Processing, Security and Computer Vision, vol. 3, S. C. Satapathy, K. S. Raju, K. Shyamala, D. R. Krishna, and M. N. Favorskaya, Eds., Cham: Springer International Publishing, 2020, pp. 318–325, doi: 10.1007/978-3- 030-24322-7 41.
[27] M. Marountas, G. Drakopoulos, P. Mylonas, and S. Sioutas, “Recommending Database Architectures for Social Queries: A Twitter Case Study,” in Artificial Intelligence Applications and Innovations, vol. 627, I. Maglogiannis, J. Macintyre, and L. Iliadis, Eds., Cham: Springer International Publishing, 2021, pp. 715–728, doi: 10.1007/978-3-030-79150-6 56.
[28] A. Costea et al., “VectorH: Taking SQL-on-Hadoop to the Next Level,” in Proceedings of the 2016 International Conference on Management of Data, SIGMOD ’16. New York, NY, USA: Association for Computing Machinery, 2016, pp. 1105–1117, doi: 10.1145/2882903.2903742.
[29] Z. Liu et al., “Mitigating Privacy Risks in LLM Embeddings from Embedding Inversion,” arXiv preprint arXiv:2411.05034, 2024
[30]ragas-wikiqa dataset,” Hugging Face Datasets, 2024. [Online]. Available: https://huggingface.co/datasets/explodinggradients/ragas-wikiqa.
