Function marian::inits::sinusoidalPositionEmbeddings¶
Defined in File node_initializers.cpp
Function Documentation¶
-
Ptr<NodeInitializer>
marian::inits
::
sinusoidalPositionEmbeddings
(int start)¶ Computes Google’s sinusoidal position embeddings.
Computes Google’s Transformer-style sinusoidal position embeddings starting from position ‘start’ taking into account batch and time dimensions of the tensor. Expected tensor layout {-2: time, -1: model}. Usually gets later reshaped to {time, 1, model} and added with a broadcast to learned embeddings. Positional embeddings are the same for each batch entry and change over time steps.