maisМоделирование и анализ информационных системModeling and Analysis of Information Systems1818-10152313-5417Yaroslavl State University10.18255/1818-1015-2022-3-266-279mais-1716Research ArticleTheory of DataКлассификация статей из средств массовой информации по категориям и релевантности предметной областиClassification of Articles from Mass Media by Categories and Relevance of the Subject Areahttps://orcid.org/0000-0002-5591-8332ЛарионовВладислав ДмитриевичLarionovVladislav Dmitrievichvladlarionov998@gmail.comhttps://orcid.org/0000-0003-3984-8423ПарамоновИлья ВячеславовичParamonovIlya Vyacheslavovichilya.paramonov@fruct.orgЯрославский государственный университет им. П. Г. ДемидоваРоссияP. G. Demidov Yaroslavl State UniversityRussian Federation202225092022293266279Copyright © Ларионов В.Д., Парамонов И.В., 20222022Ларионов В.Д., Парамонов И.В.Larionov V.D., Paramonov I.V.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.mais-journal.ru/jour/article/view/1716

Исследование посвященно классификации новостных статей о Ярославском государственном университете им. П. Г. Демидова (ЯрГУ) на 4 категории: общество, образование, наука и технологии, нерелевантная.Предложенные подходы основаны на нейронной сети BERT и методах машинного обучения SVM, Logistic Regression, K-Neighbors, Random Forest в сочетании с эмбеддингами различных видов: Word2Vec, FastText, TF-IDF, GPT-3. Также предложены способы предобработки текстов для достижения более высокого качества классификации. В ходе экспериментов установлено, что лучше всего с задачей справляется SVM-классификатор с эмбеддингом TF-IDF, обученный на полных текстах статей с заголовками. Его значения микро- и макро-F-меры достигают 0.8214 и 0.8308 соответственно. Сопоставимые результаты показывает нейронная сеть BERT, обученная на фрагментах абзацев с упоминанием ЯрГУ, из которых брались 128 слов из начала и 384 слова из конца. Её показатели микро- и макро-F-меры достигают 0.8304 и 0.8181 соответственно. Таким образом, установлено, что абзацев с упоминанием конкретной организации оказывается достаточно, чтобы классификация по категориям была эффективной.

The research is devoted to classification of news articles about P. G. Demidov Yaroslavl State University (YarSU) into 4 categories: “society”, “education”, “science and technologies”, “not relevant”.The proposed approaches are based on using the BERT neural network and methods of machine learning: SVM, Logistic Regression, K-Neighbors, Random Forest, in combination of different embedding types: Word2Vec, FastText, TF-IDF, GPT-3. Also approaches of text preprocessing are considered to achieve higher quality of the classification. The experiments showed that the SVM classifier with TF-IDF embedding and trained on full article texts with titles achieved the best result. Its micro-F-measure and macro-F-measure are 0.8214 and 0.8308 respectively. The BERT neural network trained on fragments of paragraphs with YarSU mentions, from which the first 128 words and the last 384 words were taken, showed comparable results. The resulting micro-F-measure and macro-F-measure are 0.8304 and 0.8181 respectively. Thus, using paragraphs with the target organisation mentions is enough to classify text by categories efficiently.

классификация по категориямавтоматическая обработка текстовпредметная областьрусский языкновостные статьиclassification by categoriesautomatic text processingsubject areaRussian languagenews articlesReferencesA. Hussain, G. Ali, F. Akhtar, Z. H. Khand, and A. Ali, “Design and analysis of news category predictor”, Engineering, Technology & Applied Science Research, vol. 10, no. 5, pp. 6380-6385, 2020.A. Hussain, G. Ali, F. Akhtar, Z. H. Khand, and A. Ali, “Design and analysis of news category predictor”, Engineering, Technology & Applied Science Research, vol. 10, no. 5, pp. 6380-6385, 2020.G. Kaur and K. Bajaj, “News classification using neural networks”, Communications on applied electronics, vol. 5, no. 1, pp. 42-45, 2016.G. Kaur and K. Bajaj, “News classification using neural networks”, Communications on applied electronics, vol. 5, no. 1, pp. 42-45, 2016.P. Semberecki and H. Maciejewski, “Deep learning methods for subject text classification of articles”, in 2017 Federated Conference on Computer Science and Information Systems (FedCSIS), IEEE, 2017, pp. 357-360.P. Semberecki and H. Maciejewski, “Deep learning methods for subject text classification of articles”, in 2017 Federated Conference on Computer Science and Information Systems (FedCSIS), IEEE, 2017, pp. 357-360.X. Luo, “Efficient English text classification using selected machine learning techniques”, Alexandria Engineering Journal, vol. 60, no. 3, pp. 3401-3409, 2021.X. Luo, “Efficient English text classification using selected machine learning techniques”, Alexandria Engineering Journal, vol. 60, no. 3, pp. 3401-3409, 2021.S. Vychegzhanin, E. Kotelnikov, and V. Milov, “Comparative analysis of machine learning methods for news categorization in Russian”, in CEUR Workshop Proceedings, vol. 2922, 2021, pp. 100-108.S. Vychegzhanin, E. Kotelnikov, and V. Milov, “Comparative analysis of machine learning methods for news categorization in Russian”, in CEUR Workshop Proceedings, vol. 2922, 2021, pp. 100-108.N. A. Gordienko, “Klassifikaciya novostej s primeneniem metodov mashinnogo obucheniya i obrabotki estestvennogo yazyka”, in Innovacionnye resheniya social’nyh, ekonomicheskih i tekhnologicheskih problem sovremennogo obshchestva, in Russian, 2021, pp. 63-65.N. A. Gordienko, “Klassifikaciya novostej s primeneniem metodov mashinnogo obucheniya i obrabotki estestvennogo yazyka”, in Innovacionnye resheniya social’nyh, ekonomicheskih i tekhnologicheskih problem sovremennogo obshchestva, in Russian, 2021, pp. 63-65.E. N. Karuna and P. V. Sokolov, “Comparison of methods for automatic classification of Russian-language texts”, in Journal of Physics: Conference Series, vol. 1864, 2021, p. 012 117.E. N. Karuna and P. V. Sokolov, “Comparison of methods for automatic classification of Russian-language texts”, in Journal of Physics: Conference Series, vol. 1864, 2021, p. 012 117.J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, 2018. arXiv: 1810.04805 [cs.CL].J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, 2018. arXiv: 1810.04805 [cs.CL].F. Pedregosa, G. Varoquaux, A. Gramfort, et al., “Scikit-learn: Machine learning in Python”, the Journal of machine Learning research, vol. 12, pp. 2825-2830, 2011.F. Pedregosa, G. Varoquaux, A. Gramfort, et al., “Scikit-learn: Machine learning in Python”, the Journal of machine Learning research, vol. 12, pp. 2825-2830, 2011.T. Mikolov, K. Chen, G. Corrado, and J. Dean, Efficient estimation of word representations in vector space, 2013. arXiv: 1301.3781v3 [cs.CL].T. Mikolov, K. Chen, G. Corrado, and J. Dean, Efficient estimation of word representations in vector space, 2013. arXiv: 1301.3781v3 [cs.CL].R. Rˇ ehu˚rˇek and P. Sojka, “Software framework for topic modelling with large corpora”, in Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, 2010, pp. 45-50.R. Rˇ ehu˚rˇek and P. Sojka, “Software framework for topic modelling with large corpora”, in Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, 2010, pp. 45-50.A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Je´gou, and T. Mikolov, Fasttext.zip: Compressing text classification models, 2016. arXiv: 1612.03651 [cs.CL].A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Je´gou, and T. Mikolov, Fasttext.zip: Compressing text classification models, 2016. arXiv: 1612.03651 [cs.CL].K. S. Jones, “A statistical interpretation of term specificity and its application in retrieval”, Journal of documentation, vol. 28, no. 1, pp. 11-22, 1972.K. S. Jones, “A statistical interpretation of term specificity and its application in retrieval”, Journal of documentation, vol. 28, no. 1, pp. 11-22, 1972.T. Brown, B. Mann, N. Ryder, et al., “Language models are few-shot learners”, Advances in neural information processing systems, vol. 33, pp. 1877-1901, 2020.T. Brown, B. Mann, N. Ryder, et al., “Language models are few-shot learners”, Advances in neural information processing systems, vol. 33, pp. 1877-1901, 2020.T. Wolf, L. Debut, V. Sanh, et al., “Transformers: State-of-the-art natural language processing”, in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38-45.T. Wolf, L. Debut, V. Sanh, et al., “Transformers: State-of-the-art natural language processing”, in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38-45.M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks”, Information Processing & Management, vol. 45, pp. 427-437, 2009.M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks”, Information Processing & Management, vol. 45, pp. 427-437, 2009.

The authors declare that there are no conflicts of interest present.