ОПРЕДЕЛЕНИЕ ИМЕНОВАННЫХ СУЩНОСТЕЙ В КАЗАХСКИХ ТЕКСТАХ С ИСПОЛЬЗОВАНИЕМ ТРАНСФОРМЕРНЫХ МОДЕЛЕЙ
DOI:
https://doi.org/10.52167/1609-1817-2025-137-2-296-309Ключевые слова:
Казахский язык, распознавание именованных сущности, трансформер, обработка естественного языка (NLP), глубокое обучениеАннотация
В данной работе рассматривался распознавание именованных сущностей в казахском языке с использованием трансформерных моделей. Распознавание именованных сущностей (NER) - это раздел обработки естественного языка, который автоматически идентифицирует именованные сущности в текстовых данных и классифицирует их по предопределённым категориям. В ходе анализа предыдущих исследований было отмечено, что по данной теме на казахском языке требуется ещё много исследований. В качестве текстовых данных были взяты научные статьи в области информационных технологий, а при подготовке набора данных имена людей, названия учреждений, местоположения, а также термины в данной области были размечены как именованные сущности в формате IOB2. Для распознавания именованных сущностей рассматривались трансформерная модель библиотеки spaCy и большая языковая модель Llama. Для решения поставленной задачи были полностью описаны внутренние структурные блоки обеих моделей, дано формальное описание реализации NER в этих моделях и выполнена их тонкая настройка. Модели были обучены на подготовленном наборе данных и проведены различные тестирования. Полученные результаты сопоставлены с показателями других исследований и представлен сравнительный анализ.
Библиографические ссылки
[1] Kalyani Pakhale, “Comprehensive Overview of Named Entity Recognition: Models, Domain-Specific Applications and Challenges”, Computer Science, Linguistics, Published in arXiv.org, 25 September 2023, 10.48550/arXiv.2309.14084
[2] Basra Jehangir, Saravanan Radhakrishnan, Rahul Agarwal, “A survey on Named Entity Recognition — datasets, tools, and methodologies”, Natural Language Processing Journal, №3 (2023), ttps://doi.org/10.1016/j.nlp.2023.100017
[3] Sanjib Narzarya, Anjali Brahmaa, Sukumar Nandib, Bidisha Somb, “Deep Learning based Named Entity Recognition for the Bodo Language”, Procedia Computer Science, 235 (2024), 2405–2421, https://doi.org/10.1016/j.procs.2024.04.228
[4] Davlatyor Mengliev, Vladimir Barakhnin, Nilufar Abdurakhmonova, Mukhriddin Eshkulov, “Developing named entity recognition algorithms for Uzbek: Dataset insights and implementation”, Data in Brief, 54 (2024), https://doi.org/10.1016/j.dib.2024.110413
[5] Warto, Supriadi Rustad, Guruh Fajar Shidik, Edi Noersasongko, Purwanto, Muljono, De Rosal Ignatius Moses Setiadi, Systematic Literature Review on Named Entity Recognition: Approach, Method, and Application, Statistics, Optimization & Information Computing, Vol 12 No 4 (2024), 907-942. https://doi.org/10.19139/soic-2310-5070-1631
[6] Tolegen Gulmira, Toleu Alymzhan, Xiaoqing Zheng, “Named Entity Recognition for Kazakh Using Conditional Random Fields”, The 4-th International Conference on Computer Processing of Turkic Languages TurkLang, 2016
[7] Gulmira Tolegen, Alymzhan Toleu, Orken Mamyrbayev, and Rustam Mussabayev, “Neural Named Entity Recognition for Kazakh. Computational Linguistics and Intelligent Text Processing”, 2023, 10.1007/978-3-031-24340-0_1
[8] G. Abdikalyk, A. Mukanova, A. Nazyrova, “Named entity recignition for kazakh language using CRF and Random Forest models: A comparative study”, News of the national acadamy of sciences of the republic of Kazakhstan Physico-mathematical series., ISSN 1991-346X, Volume 3. Number 347 (2023). 7–17, https://doi.org/10.32014/2023.2518-1726.200
[9] Akhmed-Zaki, D., Mansurova, M., Barakhnin, V., Kubis, M., Chikibayeva, D., and Kyrgyzbayeva, M., “Development of Kazakh named entity recognition models”, In International Conference on Computational Collective Intelligence (ICCCI), volume 12496 of Lecture Notes in Computer Science, pp. 697–708, Springer.
[10] Kuralbayev Aibek, Mukhsimbayev Bobur, Bekbaganbetov Abay, Fuad Hajiyev, “Named Entity Recognition Algorithms Comparison for Judicial Text Data”, IEEE 14th International Conference on Application of Information and Communication Technologies (AICT), Tashkent, Uzbekistan, 2020, 10.1109/AICT50176.2020.9368843
[11] Ж.М. Кожирбаев, Ж.А. Есенбаев, “Распознание именованных объектов для казахсткого языка”, MMCS. №3(107), ISSN 1563–0277, eISSN 2617–4871, 2020, https://doi.org/10.26577/JMMCS.2020.v107.i3.06
[12] Rustem Yeshpanov, Yerbolat Khassanov, Huseyin Atakan Varo, “KazNERD: Kazakh Named Entity Recognition Dataset”, Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 417–426 Marseille, France, 2022, https://aclanthology.org/2022.lrec-1.44/
[13] Gulizada Haisa, Gulila Altenbek, “Deep Learning with Word Embedding Improves Kazakh Named-Entity Recognition”, MDPI- Information 2022, 13, https://doi.org/10.3390/info13040180
[14] Gulizada Haisa, Gulila Altenbek, “Multi-Task Learning Model for Kazakh Query Understanding”, Sensors 2022, 22(24), 9810; https://doi.org/10.3390/s22249810
[15] A. Boldyrev, G. Weikum, C. Theobalt, Dictionary-based Named Entity Recognition, Computer Science, Published 2013, Corpus ID: 62736088
[16] Rafiullah Momand, Shakirullah Waseeb, Ahmad Masood Latif Rai, A Comparative Study of Dictionary-based and Machine Learning-based Named Entity Recognition in Pashto, Conference: NLPIR 2020: 4th International Conference on Natural Language Processing and Information Retrieval, 2020, DOI:10.1145/3443279.3443307
[17] Pablo Calleja, Raúl García-Castro, Guadalupe Aguado-de-Cea, Asunción Gómez-Pérez, Role-based model for Named Entity Recognition, Conference: RANLP 2017 - Recent Advances in Natural Language Processing Meet Deep Learning, 2017, DOI:10.26615/978-954-452-049-6_021
[18] Mai, Oudah, Khaled Shaalan, NERA 2.0: Improving coverage and performance of rule-based named entity recognition for Arabic, Natural Language Engineering, Published online by Cambridge University Press, Volume 23 , Issue 3 , May 2017 , pp. 441 – 472, DOI: https://doi.org/10.1017/S1351324916000097
[19] Vaishali P. Kadam, Mahender Namrata, A Named Entity Recognition System for the Marathi Language, JOURNAL OF ADVANCED APPLIED SCIENTIFIC RESEARCH, Volume 6, Issue-3, 2024, ISSN 2454-3225, DOI: 10.46947/joaasr632024937
[20] Jieren Cheng, Jingxin Liu, Xinbin Xu, Dongwan Xia, Le Liu,Victor S. Sheng, A review of Chinese named entity recognition, KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS VOL. 15, NO. 6, Jun. 2021, DOI: 10.3837/tiis.2021.06.004
[21] Gowri Prasad, K.K. Fousiya, Dr. M. Anand Kumar and Dr. K.P. Soman, Named Entity Recognition for Malayalam Language: A CRF based Approach, 2015 International Conference on Smart Technologies and Management for Computing, Communication, Controls, Energy and Materials (ICSTM), 2015, DOI:10.1109/ICSTM.2015.7225384
[22] Yu Zhang, Xuwen Wang, Zhen Hou, Jiao Li, Clinical Named Entity Recognition From Chinese Electronic Health Records via Machine Learning Methods, JMIR Medical Informatics, Volume 6, Issue 4, 2018, https://doi.org/10.2196/medinform.9965
[23] Andrey A. Gultiaev, Jenny V. Domashova, Developing a named entity recognition model for text documents in Russian to detect personal data using machine learning methods, Procedia Computer Science 213 (2022) 127–135, https://doi.org/10.1016/j.procs.2022.11.047
[24] Michelle Y. Chew, Yi J. Cheng, Oliver Mahan, Md Rakibul Islam, A comparative study of name entity recognition techniques in software engineering texts, SAC '22: Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing, Pages 1611 – 1614, https://doi.org/10.1145/3477314.3507200
[25] Gelin Ning and Yunli Baia, Biomedical named entity recognition based on Glove-BLSTM-CRF model, Journal of Computational Methods in Sciences and Engineering 21 (2021) 125–133 DOI 10.3233/JCM-204419
[26] Mohaimenul Azam Khan Raiaan, Md. Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most. Marufatul Jannat Mim, Jubaer Ahmad Mohammed Eunus Ali,and Sami Azam, A review on large language models: architectures, applications, taxonomies, open issues and challenges, IEEE Access, 2024, DOI: 10.36227/techrxiv.24171183
[27] Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu Richard Socher, Xavier Amatriain, Jianfeng Gao, Large Language Models: A Survey, Computation and Language, 2024, https://doi.org/10.48550/arXiv.2402.06196
[28] Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, Ajmal Mian, A Comprehensive Overview of Large Language Models, Computation and Language, 2024, https://doi.org/10.48550/arXiv.2307.06435
[29] “SpaCy libaray official documentation”, https://spacy.io/usage/processing-pipelines, қатынау уақыты: 10.02.2025
[30] Llama Team. The Llama 3 herd of models // AI@Meta. 2024. URL: https://llama.meta.com/ (қатынау уақыты 5.01.2025)
[31] Kumar A., Sharma R., Bedi P. Towards Optimal NLP Solutions: Analyzing GPT and LLaMA-2 Models Across Model Scale, Dataset Size, and Task Diversity // Engineering, Technology & Applied Science Research. 2024. Т. 14, № 3. С. 14219-14224. DOI: https://doi.org/10.48084/etasr.7200.
[32] Zhang, B., Sennrich, R. Root Mean Square Layer Normalization / B. Zhang, R. Sennrich // Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). – Vancouver, Canada, 2019. – URL: https://github.com/bzhangGo/rmsnorm
[33] Su J., Lu Y., Pan S., Murtadha A., Wen B., Liu Y. ROFORMER: Enhanced Transformer with Rotary Position Embedding // arXiv preprint. 2023. URL: https://arxiv.org/abs/2104.09864
[34] Shazeer, N. GLU Variants Improve Transformer / Noam Shazeer // arXiv preprint. – 2020. – 12 февр. – URL: http://arxiv.org/abs/2002.05202v1
[35] Ramachandran, P., Zoph, B., Le, Q. V. Searching for Activation Functions / P. Ramachandran, B. Zoph, Q. V. Le // arXiv preprint. – 2017. – 27 окт. – URL: http://arxiv.org/abs/1710.05941v2
[36] Li, W. J., Lyu, D. X., Wang, G., Hu, A., Xu, N., He, G. Hardware-oriented algorithms for softmax and layer normalization of large language models / W. J. Li, D. X. Lyu, G. Wang, A. Hu, N. Xu, G. He // Science China Information Sciences. – 2024. – Т. 67, № 10. – С. 200404:1–200404:15. – DOI: 10.1007/s11432-024-4137-4
Загрузки
Опубликован
Как цитировать
Выпуск
Раздел
Категории
Лицензия
Copyright (c) 2025 Нуржан Мукажанов, Жибек Алибиева, Тимур Картбаев, Бауыржан Аширбеков, Динара Касымова
Это произведение доступно по лицензии Creative Commons «Attribution-NonCommercial-NoDerivatives» («Атрибуция — Некоммерческое использование — Без производных произведений») 4.0 Всемирная.
