Applying large language models to Russian-English word alignment

This paper investigates the task of automatic word alignment in parallel texts, a fundamental step for training machine translation systems, conducting comparative linguistic studies, and creating linguistic resources. Given the scarcity of annotated data for many language pairs, the applicability of Large Language Models (LLMs) becomes particularly relevant due to their high generalization capabilities and ability to solve tasks without extensive fine-tuning on target datasets. This study presents a comparative analysis of the effectiveness of modern general-purpose LLMs versus specialized alignment algorithms using Russian-English parallel data. The research involved testing ten state-of-the-art models (including Gemini 3 Pro, GPT-5.2, and Claude Sonnet 4.5) using various prompting strategies (zero-shot, few-shot), alongside five baseline approaches ranging from statistical methods (fast-align, eflomal) to neural network architectures (AwesomeAlign, AccAlign, BinaryAlign). Performance was evaluated based on Precision, Recall, F-measure, and Alignment Error Rate (AER) metrics using annotated data from the Russian National Corpus. Experimental results indicated that the specialized BinaryAlign algorithm maintains the lead in overall alignment quality (F-measure 0.883, AER 0.113). However, leading LLMs, specifically Gemini 3 Pro Preview and GPT-5.2, demonstrated results surpassing those of most classic and early neural network baselines. Notably, for the most effective models, including in-context examples often reduced performance compared to the zero-shot setting. Thus, modern LLMs can serve as a reliable tool for high-quality alignment in the absence of training data, opening new perspectives for processing low-resource language pairs.

1. M. Baker, “Corpora in Translation Studies: An Overview and Some Suggestions for Future Research,” Target: International Journal of Translation Studies, vol. 7, no. 2, pp. 223–243, 1995, doi: 10.1075/target.7.2.03bak.

2. R. Dabre, C. Chu, and A. Kunchukuttan, “A Survey of Multilingual Neural Machine Translation,” ACM Computing Surveys, vol. 53, no. 5, p. 99, 2020, doi: 10.1145/3406095.

3. L. Tian, D. F. Wong, L. S. Chao, and F. Oliveira, “A Relationship: Word Alignment, Phrase Table, and Translation Quality,” The Scientific World Journal, vol. 2014, no. 1, p. 438106, 2014, doi: 10.1155/2014/438106.

4. P. F. Brown, S. A. Della Pietra, V. J. Della Pietra, and R. L. Mercer, “The Mathematics of Statistical Machine Translation: Parameter Estimation,” Computational Linguistics, vol. 19, no. 2, pp. 263–311, 1993.

5. F. J. Och and H. Ney, “A Systematic Comparison of Various Statistical Alignment Models,” Computational Linguistics, vol. 29, no. 1, pp. 19–51, 2003, doi: 10.1162/089120103321337421.

6. C. Dyer, V. Chahuneau, and N. A. Smith, “A Simple, Fast, and Effective Reparameterization of IBM Model 2,” in Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2013, pp. 644–648.

7. M. Jalili Sabet, P. Dufter, F. Yvon, and H. Sch"utze, “SimAlign: High Quality Word Alignments Without Parallel Training Data Using Static and Contextualized Embeddings,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 1627–1643, doi: 10.18653/v1/2020.findings-emnlp.147.

8. Z.-Y. Dou and G. Neubig, “Word Alignment by Fine-tuning Embeddings on Parallel Corpora,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021, pp. 2112–2128, doi: 10.18653/v1/2021.eacl-main.181.

9. W. Wang, G. Chen, H. Wang, Y. Han, and Y. Chen, “Multilingual Sentence Transformer as A Multilingual Word Aligner,” in Findings of the Association for Computational Linguistics: EMNLP 2022, 2022, pp. 2952–2963, doi: 10.18653/v1/2022.findings-emnlp.215.

10. Q. Wu, M. Nagata, and Y. Tsuruoka, “WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Prediction,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 11084–11099, doi: 10.18653/v1/2023.acl-long.621.

11. G. Latouche, M.-A. Carbonneau, and B. Swanson, “BinaryAlign: Word Alignment as Binary Sequence Labeling,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 10277–10288, doi: 10.18653/v1/2024.acl-long.553.

12. Z. Miao, Q. Wu, M. Nagata, and Y. Tsuruoka, “Improving Word Alignment Using Semi-Supervised Learning,” in Findings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 19871–19888, doi: 10.18653/v1/2025.findings-acl.1020.

13. K. Chousa, M. Nagata, and M. Nishino, “SpanAlign: Sentence Alignment Method based on Cross-Language Span Prediction and ILP,” in Proceedings of the 28th International Conference on Computational Linguistics, 2020, pp. 4750–4761, doi: 10.18653/v1/2020.coling-main.418.

14. R. "Ostling and J. Tiedemann, “Efficient word alignment with Markov Chain Monte Carlo,” Prague Bulletin of Mathematical Linguistics, vol. 106, pp. 125–146, 2016, doi: 10.1515/pralin-2016-0013.

15. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.

16. F. Feng, Y. Yang, D. Cer, N. Arivazhagan, and W. Wang, “Language-agnostic BERT Sentence Embedding,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 878–891, doi: 10.18653/v1/2022.acl-long.62.

17. A. Conneau et al., “Unsupervised Cross-lingual Representation Learning at Scale,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 8440–8451, doi: 10.18653/v1/2020.acl-main.747.

18. S. O. Savchuk et al., “Russian National Corpus 2.0: New opportunities and development prospects,” Voprosy Jazykoznanija, no. 2, pp. 7–34, 2024, doi: 10.31857/0373-658x.2024.2.7-34.

19. A. Politova, O. Bonetskaya, D. Dolgov, M. Frolova, and A. Pyrkova, “Word alignment in the Russian-Chinese parallel corpus,” in Corpus Use in Cross-linguistic Research: Paving the way for teaching, translation and professional communication, vol. 113, 2023, pp. 195–215.