Neural machine translation systems including the latest Transformer models represent translation units in the form of embeddings – vectors of real numbers. Such continuous representations of translation units lead to smoother translation results, but do not always guarantee better results due to wrong word translations, compared to statistical machine translation systems. Moreover, for low-resource language pairs, such as Russian-Vietnamese, the errors of word translations in neural machine translation systems are more aggravated. In order to solve the problem, we try different ways of con-catenating source word embeddings with embeddings of their corresponding word transla-tions, when building a Transformer-based translation system for the Russian-Vietnamese language pair. As a result, we create two novel Transformer models: Transformer with Long Encoder and Transformer with Short Encoder. In the Transformer with Long Encoder source word embedding and translation embedding of single size are concatenated to form a vector of double size. The Long Encoder reduces the size of the concatenated embedding to single size with a linear layer, and then adds it with positional embedding of the source word to create a final embedding. The Short Encoder resembles the Long Encoder except for the linear layer. Instead, the Short Encoder creates word embedding and translation embedding of half-size, and then concatenates them to form a concatenated embedding of single size. The experimental results show that the proposed models provide better translation quality compared to the baseline Transformer model.
Предметные области OECD FOS+WOS
- 1.02 КОМПЬЮТЕРНЫЕ И ИНФОРМАЦИОННЫЕ НАУКИ
- 50 АВТОМАТИКА. ВЫЧИСЛИТЕЛЬНАЯ ТЕХНИКА