Struct PReLUNodeOp

Inheritance Relationships

Base Type

Struct Documentation

struct PReLUNodeOp : public marian::UnaryNodeOp

Represents a parametric rectified linear unit node in an expression graph.

For \( \alpha = 0.01 \) (the default value) it is equivalent to Leaky ReLU.

This node implements the activation function:

\[\begin{split} f(x, \alpha) = \begin{cases} \alpha x & \text{if } x \leq 0 \\ x & \text{if } x > 0 \end{cases} \end{split}\]

and its derivative:

\[\begin{split} f^\prime(x, \alpha) = \begin{cases} \alpha & \text{if } x \leq 0 \\ 1 & \text{if } x > 0 \end{cases} \end{split}\]

Public Functions

PReLUNodeOp(float alpha, Expr a)
NodeOps forwardOps()
NodeOps backwardOps()
const std::string type()
virtual size_t hash()
virtual bool equal(Expr node)