Enum cudnnRNNMode

cudnnRNNMode_t is an enumerated type used to specify the type of network used in the cudnnRNNForwardInference(), cudnnRNNForwardTraining(), cudnnRNNBackwardData() and cudnnRNNBackwardWeights() routines.

Namespace: ManagedCuda.CudaDNN

Assembly: CudaDNN.dll

Syntax

public enum cudnnRNNMode

Fields

Name	Description
GRU	A three-gate network consisting of Gated Recurrent Units. In the forward pass the output ht for a given iteration can be computed from the recurrent input ht-1 and the previous layer input xt given matrices W, R and biases bW, bR from the following equations: i_t = σ(W_i x_t + R_i h_(t-1) + b_Wi + b_Ru) r_t = σ(W_r x_t + R_r h_(t-1) + b_Wr + b_Rr) h_'t = tanh(W_h x_t + r_t◦R_h h_(t-1) + b_Wh + b_Rh) h_t = (1 - i_t◦h_'t) + i_t◦h_(t-1) Where σ is the sigmoid operator: σ(x) = 1 / (1 + e^-x), ◦ represents a point-wise multiplication and tanh is the hyperbolic tangent function. i_t, r_t, h_'t represent the input, reset, new gates respectively.
LSTM	A four-gate Long Short-Term Memory network with no peephole connections. In the forward pass the output ht and cell output c_t for a given iteration can be computed from the recurrent input h_(t-1), the cell input c_(t-1) and the previous layer input x_t given matrices W, R and biases b_W, b_R from the following equations: i_t = σ(W_i x_t + R_i h_(t-1) + b_Wi + b_Ri) f_t = σ(W_f x_t + R_f h_(t-1) + b_Wf + b_Rf) o_t = σ(W_o x_t + R_o h_(t-1) + b_Wo + b_Ro) c_'t = tanh(W_c x_t + R_c h_(t-1) + b_Wc + b_Rc) c_t = f_t◦c_'(t-1) + i_t◦c_'t h_t = o_t◦tanh(c_t) Where σ is the sigmoid operator: σ(x) = 1 / (1 + e^-x), ◦ represents a point-wise multiplication and tanh is the hyperbolic tangent function. i_t, f_t, o_t, c_'t represent the input, forget, output and new gates respectively.
RNNRelu	A single-gate recurrent neural network with a ReLU activation function. In the forward pass the output ht for a given iteration can be computed from the recurrent input ht-1 and the previous layer input xt given matrices W, R and biases bW, bR from the following equation: h_t = ReLU(W_i x_t + R_i h_(t-1) + b_Wi + b_Ri) Where ReLU(x) = max(x, 0).
RNNTanh	A single-gate recurrent neural network with a tanh activation function. In the forward pass the output ht for a given iteration can be computed from the recurrent input ht-1 and the previous layer input xt given matrices W, R and biases bW, bR from the following equation: h_t = tanh(W_i x_t + R_i h_(t-1) + b_Wi + b_Ri) Where tanh is the hyperbolic tangent function.