Defined in File loss.h
public marian::CrossEntropyLoss(Class CrossEntropyLoss)
SequenceUnlikelihoodLoss: public marian::CrossEntropyLoss¶
Unlikelihood loss across last axis, summed up over batch and time dimensions.
This is an implementation of sequence-level unlikelihood loss from https://arxiv.org/abs/1908.04319. We rely on word-level label weights where 1 is correct and 0 is marking an error. If there are not zeros for a sentence it going to be trained with normal CE loss if there is at least one 0 it is going to flip over to use SUL for that sentence to penalize the selected word.
SUL is implemented as: -log(gather(1 - softmax(logits), -1, indices))
Factors are currently not supported.
SequenceUnlikelihoodLoss(float labelSmoothing, float factorWeight)¶