Function marian::inits::glorotUniform

• Defined in File node_initializers.cpp

Function Documentation

Ptr<NodeInitializer> marian::inits::glorotUniform(bool fanIn = false, bool fanOut = false, float scale = 1.f)

Initialize tensor with random numbers from Glorot uniform distribution.

The Glorot uniform, also called Xavier uniform, is designed to keep the scale of the gradients roughly the same in all layers. This function offers three variants (modes). The values of the tensor is sampled from Uniform(-x*scale, x*scale):

• when fanIn=false and fanOut=false (by default): x = sqrt(6 / (in + out))

• when fanIn=true and fanOut=false (fanIn mode): x = sqrt(3 / in)

• when fanIn=false and fanOut=false (fanOut mode): x = sqrt(3 / out) where in is the number of input units in the tensor, out is the number of output units. scale is used to change the range of Uniform distribution.

  Return

  A NodeInitializer