Raqamli Transformatsiya va Sun’iy Intellekt
VEKTOR FAZO MODELI HAMDA JUMLALAR O‘XSHASHLIGI O‘LCHOVLARIGA ASOSLANGAN SAVOL - JAVOB TIZIMI ISHLAB CHIQISH
1 Muhammad al-Xorazmiy nomidagi TATU.
2 Muhammad al-Xorazmiy nomidagi TATU
Keywords
Abstract
Ushbu tadqiqot TF-IDF vektor fazo modeli va jumlalar o‘xshashligiga asoslangan yopiq domenli savol-javob (SJ) tizimini ishlab chiqishga bag‘ishlangan. Tizim Samarqand davlat xizmatlari markazi uchun foydalanuvchilarning savollariga avtomatik javob berish imkoniyatini taqdim etadi. 2000 ta savol-javob juftligidan iborat korpus yig‘ilib, ishlov berilgan va modelni o‘qitishda qo‘llanilgan. Metodologiya matnlarni TF-IDF yordamida vektorlashtirish va kosinus o‘xshashligi orqali semantik solishtirishni o‘z ichiga oladi. BLEU, ROUGE va METEOR kabi baholash metrikalari natijasida aniqlik TF-IDF modeli yordamida 95.2%, jumlalar o‘xshashligi ko‘rsatkichlari yordamida esa eng yaxshi natija 94.8% ga yetgani aniqlandi. Ushbu yondashuv o‘zbek tili kabi kam manbali tillar uchun yopiq domenli avtomatlashtirilgan SJ tizimlarini yaratishda samarali ekanligini ko‘rsatadi.
References
- Minghui Qiu, Feng-Lin Li, Siyu Wang, Xing Gao, Yan Chen, Weipeng Zhao, et al., "AliMe chat: A sequence to sequence and rerank based chatbot engine", Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 498-503, July 2017.
- Jurafsky, D., Martin, J. H. "Speech and Language Processing", Pearson, 3rd Edition, 2023.
- Philipp Cimiano; Christina Unger; John McCrae, Ontology-Based Interpretation of Natural Language, Morgan & Claypool Publishers, 2014. ISBN 978-1-60845-990-2.
- Zhao, K., & Mao, X.-L. Integrating ROUGE into Sentence Similarity Measures, Journal of Chinese Information Processing, 2017
- Rajpurkar, P., et al., "SQuAD: 100,000+ Questions for Machine Comprehension of Text", Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.
- Beel, J., Gipp, B., Langer, S., Breitinger, C. "Research-paper recommender systems: a literature survey", International Journal on Digital Libraries, 2016.
- Mannon Ochilov, Dilshod Jurayev, Oybek Narzullayev, Mаtndаgi tinish belgilаrini tiklаsh muаmmolаri vа yechimlаri, Computer linguistics: problems, solutions. 2024.
- Jurafsky, D., Martin, J. H. "Speech and Language Processing", Pearson, 3rd Edition, 2023.
- Ganesan, K. "ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks", arXiv preprint, 2018.
- Ilyos Xujayarov, Mannon Ochilov, Orzimurod Xolmatov, Dilshod Jurayev. Sun’iy intellekt algoritmlari asosida matn tilini avtomatik aniqlash. Международный Журнал Теоретических и Прикладных Вопросов Цифровых Технологий.
- Xolmatov O.A., Kamolov R.K., "NLPda savol-javob tizimlarini yaratish turlari va bosqichlari", Muhammad al-Xorazmiy nomidagi TATU Samarqand filiali, "O‘zbek tilining milliy korpusi: muammolar va vazifalar" xalqaro ilmiy-amaliy konferensiyasi materiallari, 25-aprel 2023-yil, Samarqand.
- Mukhammadjon Musaev, Dilshod Juraev, Mannon Ochilov, Malika Abdullaeva. Text processing technology in Uzbek speech to sign language translation systems. AIP Conference Proceedings