maisМоделирование и анализ информационных системModeling and Analysis of Information Systems1818-10152313-5417Yaroslavl State University10.18255/1818-1015-2018-6-726-733mais-769Research ArticleОригинальные статьиArticlesВекторное представление слов с семантическими отношениями: экспериментальные наблюденияWord Embedding for Semantically Relative Words: an Experimental Studyhttps://orcid.org/0000-0003-4466-1735КаряеваМария СергеевнаKaryaevaMaria S.

аспирант

ул. Советская, 14, г. Ярославль, 150003

graduate student

14 Sovetskaya str., Yaroslavl 150003

mari.karyaeva@gmail.comhttps://orcid.org/0000-0002-6964-458XБраславскийПавел ИсааковичBraslavskiPavel I.

канд. техн. наук, доцент

г. Екатеринбург, ул. Мира, 19, 620002

PhD, Docent

19 Mira str., Ekaterinburg 620002

pbras@yandex.ruhttps://orcid.org/0000-0003-1427-4937СоколовВалерий АнатольевичSokolovValery A.

доктор физ.-мат. наук, профессор

ул. Советская, 14, г. Ярославль, 150003

Doctor, Professor

14 Sovetskaya str., Yaroslavl 150003

sokolov@uniyar.ac.ruЯрославский государственный университет им. П.Г. ДемидоваРоссияP.G. Demidov Yaroslavl State UniversityRussian FederationУральский федеральный университетРоссияUral Federal UniversityRussian Federation201819122018256726733Copyright © Каряева М.С., Браславский П.И., Соколов В.А., 20182018Каряева М.С., Браславский П.И., Соколов В.А.Karyaeva M.S., Braslavski P.I., Sokolov V.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.mais-journal.ru/jour/article/view/769

Возможность идентификации семантической близости между словами сделала модель word2vec широко используемой в NLP-задачах. Идея word2vec основана на контекстной близости слов. Каждое слово может быть представлено в виде вектора, близкие координаты векторов могут быть интерпретированы как близкие по смыслу слова. Таким образом, извлечение семантических отношений (отношение синонимии, родо-видовые отношения и другие) может быть автоматизировано. Установление семантических отношений вручную считается трудоемкой и необъективной задачей, требующей большого количества времени и привлечения экспертов. Но среди ассоциативных слов, сформированных с использованием модели word2vec, встречаются слова, не представляющие никаких отношений с главным словом, для которого был представлен ассоциативный ряд. В работе рассматриваются дополнительные критерии, которые могут быть применимы для решения данной проблемы. Наблюдения и проведенные эксперименты с общеизвестными характеристиками, такими как частота слов, позиция в ассоциативном ряду, могут быть использованы для улучшения результатов при работе с векторным представлением слов в части определения семантических отношений для русского языка. В экспериментах используется обученная на корпусах Флибусты модель word2vec и размеченные данные Викисловаря в качестве образцовых примеров, в которых отражены семантические отношения. Семантически связанные слова (или термины) нашли свое применение в тезаурусах, онтологиях, интеллектуальных системах для обработки естественного языка.

The ability to identify semantic relations between words has made a word2vec model widely used in NLP tasks. The idea of word2vec is based on a simple rule that a higher similarity can be reached if two words have a similar context. Each word can be represented as a vector, so the closest coordinates of vectors can be interpreted as similar words. It allows to establish semantic relations (synonymy, relations of hypernymy and hyponymy and other semantic relations) by applying an automatic extraction. The extraction of semantic relations by hand is considered as a time-consuming and biased task, requiring a large amount of time and some help of experts. Unfortunately, the word2vec model provides an associative list of words which does not consist of relative words only. In this paper, we show some additional criteria that may be applicable to solve this problem. Observations and experiments with well-known characteristics, such as word frequency, a position in an associative list, might be useful for improving results for the task of extraction of semantic relations for the Russian language by using word embedding. In the experiments, the word2vec model trained on the Flibusta and pairs from Wiktionary are used as examples with semantic relationships. Semantically related words are applicable to thesauri, ontologies and intelligent systems for natural language processing.

векторное представление словword2vecсемантические отношениятезаурусгипонимыгиперонимысинонимыword embeddingword2vecsemantic relationsthesaurushyponymyhypernymysynonymyИсследование выполнено при финансовой поддержке РФФИ в рамках научных проектов No16-07-01180 и No16-06-00497The reported study was funded by RFBR according to the research projects No16-07-01180 и No16-06-00497ReferencesMikolov T., Yih W., Zweig G., “Linguistic Regularities in Continuous Space Word Representations”, HLT-NAACL, 2013, 746–751.Mikolov T., Yih W., Zweig G., “Linguistic Regularities in Continuous Space Word Representations”, HLT-NAACL, 2013, 746–751.Sienˇcnik S.K., “Adapting word2vec to named entity recognition”, Proceedings of the 20th nordic conference of computational linguistics, 2015, 239–243.Sienˇcnik S.K., “Adapting word2vec to named entity recognition”, Proceedings of the 20th nordic conference of computational linguistics, 2015, 239–243.Lilleberg J., Zhu Y., Zhang Y., “Support vector machines and word2vec for text classification with semantic features”, Cognitive Informatics & Cognitive Computing, IEEE 14th International Conference, 2015, 136–140.Lilleberg J., Zhu Y., Zhang Y., “Support vector machines and word2vec for text classification with semantic features”, Cognitive Informatics & Cognitive Computing, IEEE 14th International Conference, 2015, 136–140.Ling W. et al., “Two/too simple adaptations of word2vec for syntax problems”, Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015, 1299–1304.Ling W. et al., “Two/too simple adaptations of word2vec for syntax problems”, Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015, 1299–1304.Najafabadi M.M. et al., “Deep learning applications and challenges in big data analytics”, Journal of Big Data, 2 (2015), 1.Najafabadi M.M. et al., “Deep learning applications and challenges in big data analytics”, Journal of Big Data, 2 (2015), 1.Kutuzov A., Andreev I., “Texts in, meaning out: neural language models in semantic similarity task for Russian”, 2015, https://arxiv.org/abs/1504.08183.Kutuzov A., Andreev I., “Texts in, meaning out: neural language models in semantic similarity task for Russian”, 2015, https://arxiv.org/abs/1504.08183.Hearst M. A., “Automatic acquisition of hyponyms from large text corpora”, Proceedings of the 14th conference on Computational linguistics – Association for Computational Linguistics, 2 (1992), 539–545.Hearst M. A., “Automatic acquisition of hyponyms from large text corpora”, Proceedings of the 14th conference on Computational linguistics – Association for Computational Linguistics, 2 (1992), 539–545.Klaussner C., Zhekova D., “Lexico-syntactic patterns for automatic ontology building”, Proceedings of the Second Student Research Workshop associated with RANLP, 2011, 109– 114.Klaussner C., Zhekova D., “Lexico-syntactic patterns for automatic ontology building”, Proceedings of the Second Student Research Workshop associated with RANLP, 2011, 109– 114.Maedche A., Pekar V., Staab S., “Ontology learning part one—on discovering taxonomic relations from the web”, Web Intelligence, 2003, 301–319.Maedche A., Pekar V., Staab S., “Ontology learning part one—on discovering taxonomic relations from the web”, Web Intelligence, 2003, 301–319.Snow R., Jurafsky D., Ng A. Y., “Learning syntactic patterns for automatic hypernym discovery”, Advances in Neural Information Processing Systems, 2005, 1297–1304.Snow R., Jurafsky D., Ng A. Y., “Learning syntactic patterns for automatic hypernym discovery”, Advances in Neural Information Processing Systems, 2005, 1297–1304.Panchenko A.,et al.,“Human and Machine Judgements for Russian Semantic Relatedness”, Analysis of Images, Social Networks and Texts: 5th International Conference, AIST 2016, (Yekaterinburg, Russia, April 7–9, 2016, Revised Selected Papers), 2017, 221–235.Panchenko A.,et al.,“Human and Machine Judgements for Russian Semantic Relatedness”, Analysis of Images, Social Networks and Texts: 5th International Conference, AIST 2016, (Yekaterinburg, Russia, April 7–9, 2016, Revised Selected Papers), 2017, 221–235.Fu R., et al., “Learning semantic hierarchies via word embeddings”, Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 1 (2014), 1199–1209.Fu R., et al., “Learning semantic hierarchies via word embeddings”, Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 1 (2014), 1199–1209.Ustalov D., Arefyev N., Biemann C., Panchenko A., “Negative sampling improves hypernymy extraction based on projection learning”, 2017, https://arxiv.org/pdf/ 1707.03903.pdf.Ustalov D., Arefyev N., Biemann C., Panchenko A., “Negative sampling improves hypernymy extraction based on projection learning”, 2017, https://arxiv.org/pdf/ 1707.03903.pdf.Wang C., Cao L., Zhou B., “Medical Synonym Extraction with Concept Space Models”, 2015, https://arxiv.org/pdf/1506.00528.pdf.Wang C., Cao L., Zhou B., “Medical Synonym Extraction with Concept Space Models”, 2015, https://arxiv.org/pdf/1506.00528.pdf.Rei M., Briscoe T., “Looking for hyponyms in vector space”, Proceedings of the Eighteenth Conference on Computational Natural Language Learning, 2014, 68–77.Rei M., Briscoe T., “Looking for hyponyms in vector space”, Proceedings of the Eighteenth Conference on Computational Natural Language Learning, 2014, 68–77.Turney P., Pantel P., “From frequency to meaning: Vector space models of semantics”, Journal of artificial intelligence research, 37 (2010), 141–188.Turney P., Pantel P., “From frequency to meaning: Vector space models of semantics”, Journal of artificial intelligence research, 37 (2010), 141–188.Matsuo Y., Ishizuka M., “Keyword extraction from a single document using word cooccurrence statistical information”, International Journal on Artificial Intelligence Tools, 13:1 (2004), 157–169.Matsuo Y., Ishizuka M., “Keyword extraction from a single document using word cooccurrence statistical information”, International Journal on Artificial Intelligence Tools, 13:1 (2004), 157–169.

The authors declare that there are no conflicts of interest present.