<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN" "JATS-journalpublishing1-3.dtd">
<article article-type="research-article" dtd-version="1.3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xml:lang="ru"><front><journal-meta><journal-id journal-id-type="publisher-id">mais</journal-id><journal-title-group><journal-title xml:lang="ru">Моделирование и анализ информационных систем</journal-title><trans-title-group xml:lang="en"><trans-title>Modeling and Analysis of Information Systems</trans-title></trans-title-group></journal-title-group><issn pub-type="ppub">1818-1015</issn><issn pub-type="epub">2313-5417</issn><publisher><publisher-name>Yaroslavl State University</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.18255/1818-1015-2022-2-116-133</article-id><article-id custom-type="elpub" pub-id-type="custom">mais-1650</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research Article</subject></subj-group><subj-group subj-group-type="section-heading" xml:lang="ru"><subject>Theory of Data</subject></subj-group></article-categories><title-group><article-title>Нейросетевая классификация русскоязычных предложений по тональности на четыре класса</article-title><trans-title-group xml:lang="en"><trans-title>Neural Network-Based Sentiment Classification of Russian Sentences into Four Classes</trans-title></trans-title-group></title-group><contrib-group><contrib contrib-type="author" corresp="yes"><contrib-id contrib-id-type="orcid">https://orcid.org/0000-0002-8298-3156</contrib-id><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Костерин</surname><given-names>Максим Алексеевич</given-names></name><name name-style="western" xml:lang="en"><surname>Kosterin</surname><given-names>Maksim A.</given-names></name></name-alternatives><email xlink:type="simple">makcost@gmail.com</email><xref ref-type="aff" rid="aff-1"/></contrib><contrib contrib-type="author" corresp="yes"><contrib-id contrib-id-type="orcid">https://orcid.org/0000-0003-3984-8423</contrib-id><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Парамонов</surname><given-names>Илья Вячеславович</given-names></name><name name-style="western" xml:lang="en"><surname>Paramonov</surname><given-names>Ilya V.</given-names></name></name-alternatives><email xlink:type="simple">ilya.paramonov@fruct.org</email><xref ref-type="aff" rid="aff-1"/></contrib></contrib-group><aff-alternatives id="aff-1"><aff xml:lang="ru"><institution>Ярославский государственный университет им. П. Г. Демидова</institution><country>Россия</country></aff><aff xml:lang="en"><institution>P. G. Demidov Yaroslavl State University</institution><country>Russian Federation</country></aff></aff-alternatives><pub-date pub-type="collection"><year>2022</year></pub-date><pub-date pub-type="epub"><day>17</day><month>06</month><year>2022</year></pub-date><volume>29</volume><issue>2</issue><fpage>116</fpage><lpage>133</lpage><permissions><copyright-statement>Copyright &amp;#x00A9; Костерин М.А., Парамонов И.В., 2022</copyright-statement><copyright-year>2022</copyright-year><copyright-holder xml:lang="ru">Костерин М.А., Парамонов И.В.</copyright-holder><copyright-holder xml:lang="en">Kosterin M.A., Paramonov I.V.</copyright-holder><license xml:lang="ru" license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/" xlink:type="simple"><license-p>Данная работа распространяется под лицензией Creative Commons Attribution 4.0.</license-p></license><license xml:lang="en" license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/" xlink:type="simple"><license-p>This work is licensed under a Creative Commons Attribution 4.0 License.</license-p></license></permissions><self-uri xlink:href="https://www.mais-journal.ru/jour/article/view/1650">https://www.mais-journal.ru/jour/article/view/1650</self-uri><abstract><p>Работа посвящена классификации русскоязычных предложений по тональности на четыре класса: положительный, отрицательный, смешанный и нейтральный. В отличие от большинства современных работ в этой области, вводится в рассмотрение класс предложений смешанной тональности. Предложения со смешанной тональностью содержат в себе одновременно и положительно, и отрицательно окрашенную речь. Для решения данной задачи были применены: нейронная сеть LSTM с механизмом внимания, нейронная сеть GRU с двойным механизмом внимания, нейронная сеть BERT с несколькими модификациями выходного слоя для обеспечения классификации на четыре класса. Эксперименты по сравнению эффективности различных нейронных сетей производилось на трёх корпусах русскоязычных предложений. Два корпуса составлены из пользовательских отзывов: один с отзывами на одежду, другой с отзывами на отели. Третий корпус составлен из новостных статей российских изданий. Лучшая средняя взвешенная F-мера в экспериментах, составляющая 0.90, была достигнута моделью BERT на корпусе отзывов на одежду. На этом же корпусе были отмечены лучшие F-меры для положительных и отрицательных предложений, составившие 0.92 и 0.93 соответственно. Наилучшие показатели классификации нейтральных и смешанных предложений достигаются на корпусе новостных статей. Для них F-мера составляет 0.72 и 0.58 соответственно. В результате экспериментов было продемонстрировано значительное превосходство трансферных нейронных сетей BERT над нейронными сетями предыдущего поколения LSTM и GRU, наиболее ярко выражающееся при классификации текстов со слабо выраженной эмоциональной окраской. Анализ ошибок показал, что на «смежные» классы тональности (положительный/отрицательный и смешанный) приходится большая доля ошибок при классификации с помощью BERT, чем в случае «противоположных» классов (положительный и отрицательный, нейтральный и смешанный).</p></abstract><trans-abstract xml:lang="en"><p>The paper is devoted to the classification of Russian sentences into four classes: positive, negative, mixed, and neutral. Unlike the majority of modern study in this area, the mixed sentiment class is introduced. Mixed sentiment sentences contain positive and negative sentiments simultaneously.To solve the problem, the following tools were applied: the attention-based LSTM neural network, the dual attention-based GRU neural network, the BERT neural network with several modifications of the output layer to provide classification into four classes. The experimental comparison of the efficiency of various neural networks were performed on three corpora of Russian sentences. Two of them consist of users’ reviews: one with wear reviews and another with hotel reviews. The third corpus contains news from Russian media. The highest weighted F-measure in experiments (0.90) was achieved when using BERT on the wear reviews corpus, as well as the highest weighted F-measure for positive and negative sentences (0.92 and 0.93, respectively). The best classification results for neutral and mixed sentences were achieved on the news corpus. For them F-measure was 0.72 and 0.58, respectively. As a result of experiments, the significant superiority of the BERT transfer network was demonstrated in comparison with older neural networks LTSM and GRU, especially for classification of sentences with weakly expressed sentiments. The error analysis showed that “adjacent” (positive/negative and mixed) classes are worse classified with BERT than “opposite” classes (positive and negative, neutral and mixed).</p></trans-abstract><kwd-group xml:lang="ru"><kwd>анализ тональности</kwd><kwd>нейросетевой классификатор</kwd><kwd>BERT</kwd><kwd>обработка естественного языка</kwd></kwd-group><kwd-group xml:lang="en"><kwd>sentiment analysis</kwd><kwd>neural network-based classifier</kwd><kwd>BERT</kwd><kwd>natural language processing</kwd></kwd-group></article-meta></front><back><ref-list><title>References</title><ref id="cit1"><label>1</label><citation-alternatives><mixed-citation xml:lang="ru">C. Potts, Z. Wu, A. Geiger, and D. Kiela, Dynasent: A dynamic benchmark for sentiment analysis, 2020. arXiv: 2012.15349 [cs.CL].</mixed-citation><mixed-citation xml:lang="en">C. Potts, Z. Wu, A. Geiger, and D. Kiela, Dynasent: A dynamic benchmark for sentiment analysis, 2020. arXiv: 2012.15349 [cs.CL].</mixed-citation></citation-alternatives></ref><ref id="cit2"><label>2</label><citation-alternatives><mixed-citation xml:lang="ru">F. Hamborg and K. Donnay, “NewsMTSC: a dataset for (multi-) target-dependent sentiment classification in political news articles”, in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), 2021, pp. 1663-1675.</mixed-citation><mixed-citation xml:lang="en">F. Hamborg and K. Donnay, “NewsMTSC: a dataset for (multi-) target-dependent sentiment classification in political news articles”, in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), 2021, pp. 1663-1675.</mixed-citation></citation-alternatives></ref><ref id="cit3"><label>3</label><citation-alternatives><mixed-citation xml:lang="ru">B. Liu, “Sentiment analysis and opinion mining”, Synthesis lectures on human language technologies, vol. 5, no. 1, pp. 1-167, 2012.</mixed-citation><mixed-citation xml:lang="en">B. Liu, “Sentiment analysis and opinion mining”, Synthesis lectures on human language technologies, vol. 5, no. 1, pp. 1-167, 2012.</mixed-citation></citation-alternatives></ref><ref id="cit4"><label>4</label><citation-alternatives><mixed-citation xml:lang="ru">O. Habimana, Y. Li, R. Li, X. Gu, and G. Yu, “Sentiment analysis using deep learning approaches: an overview”, Science China Information Sciences, vol. 63, no. 1, pp. 1-36, 2020.</mixed-citation><mixed-citation xml:lang="en">O. Habimana, Y. Li, R. Li, X. Gu, and G. Yu, “Sentiment analysis using deep learning approaches: an overview”, Science China Information Sciences, vol. 63, no. 1, pp. 1-36, 2020.</mixed-citation></citation-alternatives></ref><ref id="cit5"><label>5</label><citation-alternatives><mixed-citation xml:lang="ru">S. Smetanin and M. Komarov, “Deep transfer learning baselines for sentiment analysis in Russian”, Information Processing &amp; Management, vol. 58, no. 3, p. 102 484, 2021.</mixed-citation><mixed-citation xml:lang="en">S. Smetanin and M. Komarov, “Deep transfer learning baselines for sentiment analysis in Russian”, Information Processing &amp; Management, vol. 58, no. 3, p. 102 484, 2021.</mixed-citation></citation-alternatives></ref><ref id="cit6"><label>6</label><citation-alternatives><mixed-citation xml:lang="ru">A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners”, Technical report, OpenAI, 2019.</mixed-citation><mixed-citation xml:lang="en">A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners”, Technical report, OpenAI, 2019.</mixed-citation></citation-alternatives></ref><ref id="cit7"><label>7</label><citation-alternatives><mixed-citation xml:lang="ru">Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le, “XLNet: Generalized autoregressive pretraining for language understanding”, Advances in neural information processing systems, vol. 32, pp. 5754-5764, 2019.</mixed-citation><mixed-citation xml:lang="en">Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le, “XLNet: Generalized autoregressive pretraining for language understanding”, Advances in neural information processing systems, vol. 32, pp. 5754-5764, 2019.</mixed-citation></citation-alternatives></ref><ref id="cit8"><label>8</label><citation-alternatives><mixed-citation xml:lang="ru">J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2018. arXiv: 1810.04805v2 [cs.CL].</mixed-citation><mixed-citation xml:lang="en">J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2018. arXiv: 1810.04805v2 [cs.CL].</mixed-citation></citation-alternatives></ref><ref id="cit9"><label>9</label><citation-alternatives><mixed-citation xml:lang="ru">N. Kalchbrenner, E. Grefenstette, and P. Blunsom, A convolutional neural network for modelling sentences, 2014. arXiv: 1404.2188 [cs.CL].</mixed-citation><mixed-citation xml:lang="en">N. Kalchbrenner, E. Grefenstette, and P. Blunsom, A convolutional neural network for modelling sentences, 2014. arXiv: 1404.2188 [cs.CL].</mixed-citation></citation-alternatives></ref><ref id="cit10"><label>10</label><citation-alternatives><mixed-citation xml:lang="ru">I. Paramonov and A. Poletaev, “Adaptation of Semantic Rule-Based Sentiment Analysis Approach for Russian Language”, in Proceedings of 30th Conference of Open Innovations Association FRUCT, IEEE, 2021, pp. 155-164.</mixed-citation><mixed-citation xml:lang="en">I. Paramonov and A. Poletaev, “Adaptation of Semantic Rule-Based Sentiment Analysis Approach for Russian Language”, in Proceedings of 30th Conference of Open Innovations Association FRUCT, IEEE, 2021, pp. 155-164.</mixed-citation></citation-alternatives></ref><ref id="cit11"><label>11</label><citation-alternatives><mixed-citation xml:lang="ru">K. Kenyon-Dean, E. Ahmed, S. Fujimoto, J. Georges-Filteau, C. Glasz, B. Kaur, A. Lalande, S. Bhanderi, R. Belfer, N. Kanagasabai, et al., “Sentiment analysis: It’s complicated!”, in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 2018, pp. 1886-1895.</mixed-citation><mixed-citation xml:lang="en">K. Kenyon-Dean, E. Ahmed, S. Fujimoto, J. Georges-Filteau, C. Glasz, B. Kaur, A. Lalande, S. Bhanderi, R. Belfer, N. Kanagasabai, et al., “Sentiment analysis: It’s complicated!”, in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 2018, pp. 1886-1895.</mixed-citation></citation-alternatives></ref><ref id="cit12"><label>12</label><citation-alternatives><mixed-citation xml:lang="ru">X. Tan, Y. Cai, and C. Zhu, “Recognizing conflict opinions in aspect-level sentiment classification with dual attention networks”, in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3426-3431.</mixed-citation><mixed-citation xml:lang="en">X. Tan, Y. Cai, and C. Zhu, “Recognizing conflict opinions in aspect-level sentiment classification with dual attention networks”, in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3426-3431.</mixed-citation></citation-alternatives></ref><ref id="cit13"><label>13</label><citation-alternatives><mixed-citation xml:lang="ru">M. Soleymani, D. Garcia, B. Jou, B. Schuller, S.-F. Chang, and M. Pantic, “A survey of multimodal sentiment analysis”, Image and Vision Computing, vol. 65, pp. 3-14, 2017.</mixed-citation><mixed-citation xml:lang="en">M. Soleymani, D. Garcia, B. Jou, B. Schuller, S.-F. Chang, and M. Pantic, “A survey of multimodal sentiment analysis”, Image and Vision Computing, vol. 65, pp. 3-14, 2017.</mixed-citation></citation-alternatives></ref><ref id="cit14"><label>14</label><citation-alternatives><mixed-citation xml:lang="ru">L. A. M. Oberla¨nder and R. Klinger, “An analysis of annotated corpora for emotion classification in text”, in Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 2104-2119.</mixed-citation><mixed-citation xml:lang="en">L. A. M. Oberla¨nder and R. Klinger, “An analysis of annotated corpora for emotion classification in text”, in Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 2104-2119.</mixed-citation></citation-alternatives></ref><ref id="cit15"><label>15</label><citation-alternatives><mixed-citation xml:lang="ru">A. Radford, R. Jozefowicz, and I. Sutskever, Learning to generate reviews and discovering sentiment, 2017. arXiv: 1704.01444v2 [cs.LG].</mixed-citation><mixed-citation xml:lang="en">A. Radford, R. Jozefowicz, and I. Sutskever, Learning to generate reviews and discovering sentiment, 2017. arXiv: 1704.01444v2 [cs.LG].</mixed-citation></citation-alternatives></ref><ref id="cit16"><label>16</label><citation-alternatives><mixed-citation xml:lang="ru">Y. Wang, M. Huang, L. Zhao, and X. Zhu, “Attention-based LSTM for aspect-level sentiment classification”, in Proceedings of the 2016 conference on empirical methods in natural language processing, 2016, pp. 606-615. Neural Network-Based Sentiment Classification of Russian Sentences into Four Classes</mixed-citation><mixed-citation xml:lang="en">Y. Wang, M. Huang, L. Zhao, and X. Zhu, “Attention-based LSTM for aspect-level sentiment classification”, in Proceedings of the 2016 conference on empirical methods in natural language processing, 2016, pp. 606-615. Neural Network-Based Sentiment Classification of Russian Sentences into Four Classes</mixed-citation></citation-alternatives></ref><ref id="cit17"><label>17</label><citation-alternatives><mixed-citation xml:lang="ru">P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent attention network on memory for aspect sentiment analysis”, in Proceedings of the 2017 conference on empirical methods in natural language processing, 2017, pp. 452-461.</mixed-citation><mixed-citation xml:lang="en">P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent attention network on memory for aspect sentiment analysis”, in Proceedings of the 2017 conference on empirical methods in natural language processing, 2017, pp. 452-461.</mixed-citation></citation-alternatives></ref><ref id="cit18"><label>18</label><citation-alternatives><mixed-citation xml:lang="ru">A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need”, in Advances in neural information processing systems, 2017, pp. 5998-6008.</mixed-citation><mixed-citation xml:lang="en">A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need”, in Advances in neural information processing systems, 2017, pp. 5998-6008.</mixed-citation></citation-alternatives></ref><ref id="cit19"><label>19</label><citation-alternatives><mixed-citation xml:lang="ru">R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank”, in Proceedings of the 2013 conference on empirical methods in natural language processing, 2013, pp. 1631-1642.</mixed-citation><mixed-citation xml:lang="en">R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank”, in Proceedings of the 2013 conference on empirical methods in natural language processing, 2013, pp. 1631-1642.</mixed-citation></citation-alternatives></ref><ref id="cit20"><label>20</label><citation-alternatives><mixed-citation xml:lang="ru">S. Smetanin and M. Komarov, “Sentiment Analysis of Product Reviews in Russian using Convolutional Neural Networks”, in IEEE 21st Conference on Business Informatics (CBI), vol. 1, 2019, pp. 482-486.</mixed-citation><mixed-citation xml:lang="en">S. Smetanin and M. Komarov, “Sentiment Analysis of Product Reviews in Russian using Convolutional Neural Networks”, in IEEE 21st Conference on Business Informatics (CBI), vol. 1, 2019, pp. 482-486.</mixed-citation></citation-alternatives></ref></ref-list><fn-group><fn fn-type="conflict"><p>The authors declare that there are no conflicts of interest present.</p></fn></fn-group></back></article>
