Struct SwishNodeOp

Inheritance Relationships

Base Type

Struct Documentation

struct SwishNodeOp : public marian::UnaryNodeOp

Represents a swish node in an expression graph.

This node implements the activation function \( f(x) = x \cdot \sigma(bx) \) and its derivative \( f^\prime(x) = bf(x) + \sigma(bx)(1 - bf(x)) \) .

Public Functions

SwishNodeOp(Expr a, float b = 1.f)
NodeOps forwardOps()
NodeOps backwardOps()
const std::string type()
virtual size_t hash()
virtual bool equal(Expr node)

Public Members

float b_