Class CudaDNNNativeMethods

Inheritance

System.Object

CudaDNNNativeMethods

Inherited Members

System.Object.Equals(System.Object)

System.Object.Equals(System.Object, System.Object)

System.Object.GetHashCode()

System.Object.GetType()

System.Object.MemberwiseClone()

System.Object.ReferenceEquals(System.Object, System.Object)

System.Object.ToString()

Namespace: ManagedCuda.CudaDNN

Assembly: CudaDNN.dll

Syntax

public static class CudaDNNNativeMethods

Properties

| Improve this Doc View Source

Version

Gives the version of the wrapped api

Declaration

public static Version Version { get; }

Property Value

Type	Description
System.Version

Methods

cudnnActivationBackward(cudnnHandle, cudnnActivationDescriptor, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This routine computes the gradient of a neuron activation function.

Declaration

public static cudnnStatus cudnnActivationBackward(cudnnHandle handle, cudnnActivationDescriptor activationDesc, ref double alpha, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnActivationDescriptor	activationDesc	Handle to the previously created activation descriptor object.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor srcDiffData.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the output tensor descriptor destDesc.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output differential tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor destDiffDesc.

Returns

Type	Description
cudnnStatus

cudnnActivationBackward(cudnnHandle, cudnnActivationDescriptor, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This routine computes the gradient of a neuron activation function.

Declaration

public static cudnnStatus cudnnActivationBackward(cudnnHandle handle, cudnnActivationDescriptor activationDesc, ref float alpha, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnActivationDescriptor	activationDesc	Handle to the previously created activation descriptor object.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor srcDiffData.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the output tensor descriptor destDesc.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output differential tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor destDiffDesc.

Returns

Type	Description
cudnnStatus

cudnnActivationForward(cudnnHandle, cudnnActivationDescriptor, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This routine applies a specified neuron activation function element-wise over each input value.

Declaration

public static cudnnStatus cudnnActivationForward(cudnnHandle handle, cudnnActivationDescriptor activationDesc, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnActivationDescriptor	activationDesc	Handle to the previously created activation descriptor object.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor destDesc.

Returns

Type	Description
cudnnStatus

cudnnActivationForward(cudnnHandle, cudnnActivationDescriptor, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This routine applies a specified neuron activation function element-wise over each input value.

Declaration

public static cudnnStatus cudnnActivationForward(cudnnHandle handle, cudnnActivationDescriptor activationDesc, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnActivationDescriptor	activationDesc	Handle to the previously created activation descriptor object.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor destDesc.

Returns

Type	Description
cudnnStatus

cudnnAddTensor(cudnnHandle, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function adds the scaled values of one bias tensor to another tensor. Each dimension of the bias tensor must match the coresponding dimension of the srcDest tensor or must be equal to 1. In the latter case, the same value from the bias tensor for thoses dimensions will be used to blend into the srcDest tensor.

Declaration

public static cudnnStatus cudnnAddTensor(cudnnHandle handle, ref double alpha, cudnnTensorDescriptor aDesc, CUdeviceptr a, ref double beta, cudnnTensorDescriptor cDesc, CUdeviceptr cData)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	aDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	a	Pointer to data of the tensor described by the biasDesc descriptor.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	cDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	cData	Pointer to data of the tensor described by the srcDestDesc descriptor.

Returns

Type	Description
cudnnStatus

cudnnAddTensor(cudnnHandle, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnAddTensor(cudnnHandle handle, ref float alpha, cudnnTensorDescriptor aDesc, CUdeviceptr a, ref float beta, cudnnTensorDescriptor cDesc, CUdeviceptr c)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	aDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	a	Pointer to data of the tensor described by the biasDesc descriptor.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	cDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	c	Pointer to data of the tensor described by the srcDestDesc descriptor.

Returns

Type	Description
cudnnStatus

cudnnBatchNormalizationBackward(cudnnHandle, cudnnBatchNormMode, ref Double, ref Double, ref Double, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, Double, CUdeviceptr, CUdeviceptr)

Declaration

public static cudnnStatus cudnnBatchNormalizationBackward(cudnnHandle handle, cudnnBatchNormMode mode, ref double alphaDataDiff, ref double betaDataDiff, ref double alphaParamDiff, ref double betaParamDiff, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnTensorDescriptor dxDesc, CUdeviceptr dx, cudnnTensorDescriptor dBnScaleBiasDesc, CUdeviceptr bnScale, CUdeviceptr dBnScaleResult, CUdeviceptr dBnBiasResult, double epsilon, CUdeviceptr savedMean, CUdeviceptr savedInvVariance)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnBatchNormMode	mode
System.Double	alphaDataDiff
System.Double	betaDataDiff
System.Double	alphaParamDiff
System.Double	betaParamDiff
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
cudnnTensorDescriptor	dyDesc
CUdeviceptr	dy
cudnnTensorDescriptor	dxDesc
CUdeviceptr	dx
cudnnTensorDescriptor	dBnScaleBiasDesc
CUdeviceptr	bnScale
CUdeviceptr	dBnScaleResult
CUdeviceptr	dBnBiasResult
System.Double	epsilon
CUdeviceptr	savedMean
CUdeviceptr	savedInvVariance

Returns

Type	Description
cudnnStatus

cudnnBatchNormalizationBackward(cudnnHandle, cudnnBatchNormMode, ref Single, ref Single, ref Single, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, Double, CUdeviceptr, CUdeviceptr)

Performs backward pass of Batch Normalization layer. Returns x gradient, bnScale gradient and bnBias gradient

Declaration

public static cudnnStatus cudnnBatchNormalizationBackward(cudnnHandle handle, cudnnBatchNormMode mode, ref float alphaDataDiff, ref float betaDataDiff, ref float alphaParamDiff, ref float betaParamDiff, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnTensorDescriptor dxDesc, CUdeviceptr dx, cudnnTensorDescriptor dBnScaleBiasDesc, CUdeviceptr bnScale, CUdeviceptr dBnScaleResult, CUdeviceptr dBnBiasResult, double epsilon, CUdeviceptr savedMean, CUdeviceptr savedInvVariance)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnBatchNormMode	mode
System.Single	alphaDataDiff
System.Single	betaDataDiff
System.Single	alphaParamDiff
System.Single	betaParamDiff
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
cudnnTensorDescriptor	dyDesc
CUdeviceptr	dy
cudnnTensorDescriptor	dxDesc
CUdeviceptr	dx
cudnnTensorDescriptor	dBnScaleBiasDesc
CUdeviceptr	bnScale
CUdeviceptr	dBnScaleResult
CUdeviceptr	dBnBiasResult
System.Double	epsilon
CUdeviceptr	savedMean
CUdeviceptr	savedInvVariance

Returns

Type	Description
cudnnStatus

cudnnBatchNormalizationForwardInference(cudnnHandle, cudnnBatchNormMode, ref Double, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr, Double)

Declaration

public static cudnnStatus cudnnBatchNormalizationForwardInference(cudnnHandle handle, cudnnBatchNormMode mode, ref double alpha, ref double beta, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor bnScaleBiasMeanVarDesc, CUdeviceptr bnScale, CUdeviceptr bnBias, CUdeviceptr estimatedMean, CUdeviceptr estimatedVariance, double epsilon)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnBatchNormMode	mode
System.Double	alpha
System.Double	beta
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
cudnnTensorDescriptor	yDesc
CUdeviceptr	y
cudnnTensorDescriptor	bnScaleBiasMeanVarDesc
CUdeviceptr	bnScale
CUdeviceptr	bnBias
CUdeviceptr	estimatedMean
CUdeviceptr	estimatedVariance
System.Double	epsilon

Returns

Type	Description
cudnnStatus

cudnnBatchNormalizationForwardInference(cudnnHandle, cudnnBatchNormMode, ref Single, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr, Double)

Performs Batch Normalization during Inference: y[i] = bnScale[k]*(x[i]-estimatedMean[k])*estimatedInvVariance[k] + bnBias[k] with bnScale, bnBias, runningMean, runningInvVariance tensors indexed according to spatial or per-activation mode. Refer to cudnnBatchNormalizationForwardTraining above for notes on function arguments.

Declaration

public static cudnnStatus cudnnBatchNormalizationForwardInference(cudnnHandle handle, cudnnBatchNormMode mode, ref float alpha, ref float beta, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor bnScaleBiasMeanVarDesc, CUdeviceptr bnScale, CUdeviceptr bnBias, CUdeviceptr estimatedMean, CUdeviceptr estimatedVariance, double epsilon)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnBatchNormMode	mode
System.Single	alpha
System.Single	beta
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
cudnnTensorDescriptor	yDesc
CUdeviceptr	y
cudnnTensorDescriptor	bnScaleBiasMeanVarDesc
CUdeviceptr	bnScale
CUdeviceptr	bnBias
CUdeviceptr	estimatedMean
CUdeviceptr	estimatedVariance
System.Double	epsilon

Returns

Type	Description
cudnnStatus

cudnnBatchNormalizationForwardTraining(cudnnHandle, cudnnBatchNormMode, ref Double, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, Double, CUdeviceptr, CUdeviceptr, Double, CUdeviceptr, CUdeviceptr)

Declaration

public static cudnnStatus cudnnBatchNormalizationForwardTraining(cudnnHandle handle, cudnnBatchNormMode mode, ref double alpha, ref double beta, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor bnScaleBiasMeanVarDesc, CUdeviceptr bnScale, CUdeviceptr bnBias, double exponentialAverageFactor, CUdeviceptr resultRunningMean, CUdeviceptr resultRunningVariance, double epsilon, CUdeviceptr resultSaveMean, CUdeviceptr resultSaveVariance)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnBatchNormMode	mode
System.Double	alpha
System.Double	beta
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
cudnnTensorDescriptor	yDesc
CUdeviceptr	y
cudnnTensorDescriptor	bnScaleBiasMeanVarDesc
CUdeviceptr	bnScale
CUdeviceptr	bnBias
System.Double	exponentialAverageFactor
CUdeviceptr	resultRunningMean
CUdeviceptr	resultRunningVariance
System.Double	epsilon
CUdeviceptr	resultSaveMean
CUdeviceptr	resultSaveVariance

Returns

Type	Description
cudnnStatus

cudnnBatchNormalizationForwardTraining(cudnnHandle, cudnnBatchNormMode, ref Single, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, Double, CUdeviceptr, CUdeviceptr, Double, CUdeviceptr, CUdeviceptr)

Computes y = BN(x). Also accumulates moving averages of mean and inverse variances

Declaration

public static cudnnStatus cudnnBatchNormalizationForwardTraining(cudnnHandle handle, cudnnBatchNormMode mode, ref float alpha, ref float beta, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor bnScaleBiasMeanVarDesc, CUdeviceptr bnScale, CUdeviceptr bnBias, double exponentialAverageFactor, CUdeviceptr resultRunningMean, CUdeviceptr resultRunningVariance, double epsilon, CUdeviceptr resultSaveMean, CUdeviceptr resultSaveVariance)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnBatchNormMode	mode
System.Single	alpha
System.Single	beta
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
cudnnTensorDescriptor	yDesc
CUdeviceptr	y
cudnnTensorDescriptor	bnScaleBiasMeanVarDesc
CUdeviceptr	bnScale
CUdeviceptr	bnBias
System.Double	exponentialAverageFactor
CUdeviceptr	resultRunningMean
CUdeviceptr	resultRunningVariance
System.Double	epsilon
CUdeviceptr	resultSaveMean
CUdeviceptr	resultSaveVariance

Returns

Type	Description
cudnnStatus

cudnnConvolutionBackwardBias(cudnnHandle, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function computes the convolution gradient with respect to the bias, which is the sum of every element belonging to the same feature map across all of the images of the input tensor. Therefore, the number of elements produced is equal to the number of features maps of the input tensor.

Declaration

public static cudnnStatus cudnnConvolutionBackwardBias(cudnnHandle handle, ref double alpha, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, ref double beta, cudnnTensorDescriptor dbDesc, CUdeviceptr db)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dbDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	db	Data pointer to GPU memory associated with the output tensor descriptor destDesc.

Returns

Type	Description
cudnnStatus

cudnnConvolutionBackwardBias(cudnnHandle, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnConvolutionBackwardBias(cudnnHandle handle, ref float alpha, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, ref float beta, cudnnTensorDescriptor dbDesc, CUdeviceptr db)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dbDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	db	Data pointer to GPU memory associated with the output tensor descriptor destDesc.

Returns

Type	Description
cudnnStatus

cudnnConvolutionBackwardData(cudnnHandle, ref Double, cudnnFilterDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnConvolutionBwdDataAlgo, CUdeviceptr, SizeT, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function computes the convolution gradient with respect to the output tensor using the specified algo, returning results in gradDesc. Scaling factors alpha and beta can be used to scale the input tensor and the output tensor respectively.

Declaration

public static cudnnStatus cudnnConvolutionBackwardData(cudnnHandle handle, ref double alpha, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnConvolutionDescriptor convDesc, cudnnConvolutionBwdDataAlgo algo, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes, ref double beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor filterDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the input differential tensor descriptor diffDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnConvolutionBwdDataAlgo	algo	Enumerant that specifies which backward data convolution algorithm shoud be used to compute the results
CUdeviceptr	workSpace	Data pointer to GPU memory to a workspace needed to able to execute the specified algorithm. If no workspace is needed for a particular algorithm, that pointer can be nil
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor gradDesc that carries the result.

Returns

Type	Description
cudnnStatus

cudnnConvolutionBackwardData(cudnnHandle, ref Single, cudnnFilterDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnConvolutionBwdDataAlgo, CUdeviceptr, SizeT, ref Single, cudnnTensorDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnConvolutionBackwardData(cudnnHandle handle, ref float alpha, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnConvolutionDescriptor convDesc, cudnnConvolutionBwdDataAlgo algo, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes, ref float beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor filterDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the input differential tensor descriptor diffDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnConvolutionBwdDataAlgo	algo	Enumerant that specifies which backward data convolution algorithm shoud be used to compute the results
CUdeviceptr	workSpace	Data pointer to GPU memory to a workspace needed to able to execute the specified algorithm. If no workspace is needed for a particular algorithm, that pointer can be nil
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor gradDesc that carries the result.

Returns

Type	Description
cudnnStatus

cudnnConvolutionBackwardFilter(cudnnHandle, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnConvolutionBwdFilterAlgo, CUdeviceptr, SizeT, ref Double, cudnnFilterDescriptor, CUdeviceptr)

This function computes the convolution gradient with respect to filter coefficients using the specified algo, returning results in gradDesc.Scaling factors alpha and beta can be used to scale the input tensor and the output tensor respectively.

Declaration

public static cudnnStatus cudnnConvolutionBackwardFilter(cudnnHandle handle, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnConvolutionDescriptor convDesc, cudnnConvolutionBwdFilterAlgo algo, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes, ref double beta, cudnnFilterDescriptor dwDesc, CUdeviceptr dw)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the input differential tensor descriptor diffDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnConvolutionBwdFilterAlgo	algo	Enumerant that specifies which convolution algorithm shoud be used to compute the results
CUdeviceptr	workSpace	Data pointer to GPU memory to a workspace needed to able to execute the specified algorithm. If no workspace is needed for a particular algorithm, that pointer can be nil
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnFilterDescriptor	dwDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	dw	Data pointer to GPU memory associated with the filter descriptor gradDesc that carries the result.

Returns

Type	Description
cudnnStatus

cudnnConvolutionBackwardFilter(cudnnHandle, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnConvolutionBwdFilterAlgo, CUdeviceptr, SizeT, ref Single, cudnnFilterDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnConvolutionBackwardFilter(cudnnHandle handle, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnConvolutionDescriptor convDesc, cudnnConvolutionBwdFilterAlgo algo, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes, ref float beta, cudnnFilterDescriptor dwDesc, CUdeviceptr dw)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the input differential tensor descriptor diffDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnConvolutionBwdFilterAlgo	algo	Enumerant that specifies which convolution algorithm shoud be used to compute the results
CUdeviceptr	workSpace	Data pointer to GPU memory to a workspace needed to able to execute the specified algorithm. If no workspace is needed for a particular algorithm, that pointer can be nil
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnFilterDescriptor	dwDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	dw	Data pointer to GPU memory associated with the filter descriptor gradDesc that carries the result.

Returns

Type	Description
cudnnStatus

cudnnConvolutionBiasActivationForward(cudnnHandle, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnConvolutionFwdAlgo, CUdeviceptr, SizeT, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnActivationDescriptor, cudnnTensorDescriptor, CUdeviceptr)

This function applies a bias and then an activation to the convolutions or crosscorrelations of cudnnConvolutionForward(), returning results in y.The full computation follows the equation y = act(alpha1* conv(x) + alpha2* z + bias ).

The routine cudnnGetConvolution2dForwardOutputDim or cudnnGetConvolutionNdForwardOutputDim can be used to determine the proper dimensions of the output tensor descriptor yDesc with respect to xDesc, convDesc and wDesc.

Declaration

public static cudnnStatus cudnnConvolutionBiasActivationForward(cudnnHandle handle, ref double alpha1, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnConvolutionDescriptor convDesc, cudnnConvolutionFwdAlgo algo, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes, ref double alpha2, cudnnTensorDescriptor zDesc, CUdeviceptr z, cudnnTensorDescriptor biasDesc, CUdeviceptr bias, cudnnActivationDescriptor activationDesc, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Double	alpha1	Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as described by the above equation.
cudnnTensorDescriptor	xDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor xDesc.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor wDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnConvolutionFwdAlgo	algo	Enumerant that specifies which convolution algorithm shoud be used to compute the results
CUdeviceptr	workSpace	Data pointer to GPU memory to a workspace needed to able to execute the specified algorithm.If no workspace is needed for a particular algorithm, that pointer can be nil
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace
System.Double	alpha2	Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as described by the above equation.
cudnnTensorDescriptor	zDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	z	Data pointer to GPU memory associated with the tensor descriptor zDesc.
cudnnTensorDescriptor	biasDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	bias	Data pointer to GPU memory associated with the tensor descriptor biasDesc.
cudnnActivationDescriptor	activationDesc	Handle to a previously initialized activation descriptor.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor yDesc that carries the result of the convolution.

Returns

Type	Description
cudnnStatus

cudnnConvolutionBiasActivationForward(cudnnHandle, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnConvolutionFwdAlgo, CUdeviceptr, SizeT, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnActivationDescriptor, cudnnTensorDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnConvolutionBiasActivationForward(cudnnHandle handle, ref float alpha1, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnConvolutionDescriptor convDesc, cudnnConvolutionFwdAlgo algo, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes, ref float alpha2, cudnnTensorDescriptor zDesc, CUdeviceptr z, cudnnTensorDescriptor biasDesc, CUdeviceptr bias, cudnnActivationDescriptor activationDesc, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Single	alpha1	Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as described by the above equation.
cudnnTensorDescriptor	xDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor xDesc.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor wDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnConvolutionFwdAlgo	algo	Enumerant that specifies which convolution algorithm shoud be used to compute the results
CUdeviceptr	workSpace	Data pointer to GPU memory to a workspace needed to able to execute the specified algorithm.If no workspace is needed for a particular algorithm, that pointer can be nil
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace
System.Single	alpha2	Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as described by the above equation.
cudnnTensorDescriptor	zDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	z	Data pointer to GPU memory associated with the tensor descriptor zDesc.
cudnnTensorDescriptor	biasDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	bias	Data pointer to GPU memory associated with the tensor descriptor biasDesc.
cudnnActivationDescriptor	activationDesc	Handle to a previously initialized activation descriptor.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor yDesc that carries the result of the convolution.

Returns

Type	Description
cudnnStatus

cudnnConvolutionForward(cudnnHandle, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnConvolutionFwdAlgo, CUdeviceptr, SizeT, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function executes convolutions or cross-correlations over src using the specified filters, returning results in dest. Scaling factors alpha and beta can be used to scale the input tensor and the output tensor respectively.

Declaration

public static cudnnStatus cudnnConvolutionForward(cudnnHandle handle, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnConvolutionDescriptor convDesc, cudnnConvolutionFwdAlgo algo, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes, ref double beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor filterDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnConvolutionFwdAlgo	algo	Enumerant that specifies which convolution algorithm shoud be used to compute the results
CUdeviceptr	workSpace	Data pointer to GPU memory to a workspace needed to able to execute the specified algorithm. If no workspace is needed for a particular algorithm, that pointer can be nil
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor destDesc that carries the result of the convolution.

Returns

Type	Description
cudnnStatus

cudnnConvolutionForward(cudnnHandle, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnConvolutionFwdAlgo, CUdeviceptr, SizeT, ref Single, cudnnTensorDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnConvolutionForward(cudnnHandle handle, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnConvolutionDescriptor convDesc, cudnnConvolutionFwdAlgo algo, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes, ref float beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor filterDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnConvolutionFwdAlgo	algo	Enumerant that specifies which convolution algorithm shoud be used to compute the results
CUdeviceptr	workSpace	Data pointer to GPU memory to a workspace needed to able to execute the specified algorithm. If no workspace is needed for a particular algorithm, that pointer can be nil
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor destDesc that carries the result of the convolution.

Returns

Type	Description
cudnnStatus

cudnnCreate(ref cudnnHandle)

This function initializes the cuDNN library and creates a handle to an opaque structure holding the cuDNN library context. It allocates hardware resources on the host and device and must be called prior to making any other cuDNN library calls. The cuDNN library context is tied to the current CUDA device. To use the library on multiple devices, one cuDNN handle needs to be created for each device. For a given device, multiple cuDNN handles with different configurations (e.g., different current CUDA streams) may be created. Because cudnnCreate allocates some internal resources, the release of those resources by calling cudnnDestroy will implicitly call cudaDeviceSynchronize; therefore, the recommended best practice is to call cudnnCreate/cudnnDestroy outside of performance-critical code paths. For multithreaded applications that use the same device from different threads, the

Declaration

public static cudnnStatus cudnnCreate(ref cudnnHandle handle)

Parameters

Type	Name	Description
cudnnHandle	handle

Returns

Type	Description
cudnnStatus

cudnnCreateActivationDescriptor(ref cudnnActivationDescriptor)

This function creates a activation descriptor object by allocating the memory needed to hold its opaque structure.

Declaration

public static cudnnStatus cudnnCreateActivationDescriptor(ref cudnnActivationDescriptor activationDesc)

Parameters

Type	Name	Description
cudnnActivationDescriptor	activationDesc

Returns

Type	Description
cudnnStatus

cudnnCreateConvolutionDescriptor(ref cudnnConvolutionDescriptor)

This function creates a convolution descriptor object by allocating the memory needed to hold its opaque structure

Declaration

public static cudnnStatus cudnnCreateConvolutionDescriptor(ref cudnnConvolutionDescriptor convDesc)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc

Returns

Type	Description
cudnnStatus

cudnnCreateCTCLossDescriptor(ref cudnnCTCLossDescriptor)

Create an instance of a CTC (Connectionist Temporal Classification) loss descriptor

Declaration

public static cudnnStatus cudnnCreateCTCLossDescriptor(ref cudnnCTCLossDescriptor ctcLossDesc)

Parameters

Type	Name	Description
cudnnCTCLossDescriptor	ctcLossDesc

Returns

Type	Description
cudnnStatus

cudnnCreateDropoutDescriptor(ref cudnnDropoutDescriptor)

This function creates a generic dropout descriptor object by allocating the memory needed to hold its opaque structure.

Declaration

public static cudnnStatus cudnnCreateDropoutDescriptor(ref cudnnDropoutDescriptor dropoutDesc)

Parameters

Type	Name	Description
cudnnDropoutDescriptor	dropoutDesc

Returns

Type	Description
cudnnStatus

cudnnCreateFilterDescriptor(ref cudnnFilterDescriptor)

This function creates a filter descriptor object by allocating the memory needed to hold its opaque structure.

Declaration

public static cudnnStatus cudnnCreateFilterDescriptor(ref cudnnFilterDescriptor filterDesc)

Parameters

Type	Name	Description
cudnnFilterDescriptor	filterDesc

Returns

Type	Description
cudnnStatus

cudnnCreateLRNDescriptor(ref cudnnLRNDescriptor)

Create an instance of LRN (Local Response Normalization) descriptor

This function will set lrnN=5, lrnAlpha=1e-4, lrnBeta=0.75, lrnK=2.0 as defaults from Krizhevsky'12 ImageNet paper

Declaration

public static cudnnStatus cudnnCreateLRNDescriptor(ref cudnnLRNDescriptor normDesc)

Parameters

Type	Name	Description
cudnnLRNDescriptor	normDesc

Returns

Type	Description
cudnnStatus

cudnnCreateOpTensorDescriptor(ref cudnnOpTensorDescriptor)

Declaration

public static cudnnStatus cudnnCreateOpTensorDescriptor(ref cudnnOpTensorDescriptor opTensorDesc)

Parameters

Type	Name	Description
cudnnOpTensorDescriptor	opTensorDesc

Returns

Type	Description
cudnnStatus

cudnnCreatePersistentRNNPlan(cudnnRNNDescriptor, Int32, cudnnDataType, ref cudnnPersistentRNNPlan)

This function creates a plan to execute persistent RNNs when using the CUDNN_RNN_ALGO_PERSIST_DYNAMIC algo.This plan is tailored to the current GPU and problem hyperparemeters. This function call is expected to be expensive in terms of runtime, and should be used infrequently.

Declaration

public static cudnnStatus cudnnCreatePersistentRNNPlan(cudnnRNNDescriptor rnnDesc, int minibatch, cudnnDataType dataType, ref cudnnPersistentRNNPlan plan)

Parameters

Type	Name	Description
cudnnRNNDescriptor	rnnDesc
System.Int32	minibatch
cudnnDataType	dataType
cudnnPersistentRNNPlan	plan

Returns

Type	Description
cudnnStatus

cudnnCreatePoolingDescriptor(ref cudnnPoolingDescriptor)

This function creates a pooling descriptor object by allocating the memory needed to hold its opaque structure

Declaration

public static cudnnStatus cudnnCreatePoolingDescriptor(ref cudnnPoolingDescriptor poolingDesc)

Parameters

Type	Name	Description
cudnnPoolingDescriptor	poolingDesc

Returns

Type	Description
cudnnStatus

cudnnCreateReduceTensorDescriptor(ref cudnnReduceTensorDescriptor)

Declaration

public static cudnnStatus cudnnCreateReduceTensorDescriptor(ref cudnnReduceTensorDescriptor reduceTensorDesc)

Parameters

Type	Name	Description
cudnnReduceTensorDescriptor	reduceTensorDesc

Returns

Type	Description
cudnnStatus

cudnnCreateRNNDescriptor(ref cudnnRNNDescriptor)

This function creates a generic RNN descriptor object by allocating the memory needed to hold its opaque structure.

Declaration

public static cudnnStatus cudnnCreateRNNDescriptor(ref cudnnRNNDescriptor rnnDesc)

Parameters

Type	Name	Description
cudnnRNNDescriptor	rnnDesc

Returns

Type	Description
cudnnStatus

cudnnCreateSpatialTransformerDescriptor(ref cudnnSpatialTransformerDescriptor)

This function creates a generic spatial transformer descriptor object by allocating the memory needed to hold its opaque structure.

Declaration

public static cudnnStatus cudnnCreateSpatialTransformerDescriptor(ref cudnnSpatialTransformerDescriptor stDesc)

Parameters

Type	Name	Description
cudnnSpatialTransformerDescriptor	stDesc

Returns

Type	Description
cudnnStatus

cudnnCreateTensorDescriptor(ref cudnnTensorDescriptor)

This function creates a generic Tensor descriptor object by allocating the memory needed to hold its opaque structure. The data is initialized to be all zero.

Declaration

public static cudnnStatus cudnnCreateTensorDescriptor(ref cudnnTensorDescriptor tensorDesc)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc

Returns

Type	Description
cudnnStatus

cudnnCTCLoss(cudnnHandle, cudnnTensorDescriptor, CUdeviceptr, Int32[], Int32[], Int32[], CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnCTCLossAlgo, cudnnCTCLossDescriptor, CUdeviceptr, SizeT)

return the ctc costs and gradients, given the probabilities and labels

Declaration

public static cudnnStatus cudnnCTCLoss(cudnnHandle handle, cudnnTensorDescriptor probsDesc, CUdeviceptr probs, int[] labels, int[] labelLengths, int[] inputLengths, CUdeviceptr costs, cudnnTensorDescriptor gradientsDesc, CUdeviceptr gradients, cudnnCTCLossAlgo algo, cudnnCTCLossDescriptor ctcLossDesc, CUdeviceptr workspace, SizeT workSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnTensorDescriptor	probsDesc
CUdeviceptr	probs
System.Int32[]	labels
System.Int32[]	labelLengths
System.Int32[]	inputLengths
CUdeviceptr	costs
cudnnTensorDescriptor	gradientsDesc
CUdeviceptr	gradients
cudnnCTCLossAlgo	algo
cudnnCTCLossDescriptor	ctcLossDesc
CUdeviceptr	workspace
SizeT	workSpaceSizeInBytes

Returns

Type	Description
cudnnStatus

cudnnDeriveBNTensorDescriptor(cudnnTensorDescriptor, cudnnTensorDescriptor, cudnnBatchNormMode)

Derives a tensor descriptor from layer data descriptor for BatchNormalization scale, invVariance, bnBias, bnScale tensors. Use this tensor desc for bnScaleBiasMeanVarDesc and bnScaleBiasDiffDesc in Batch Normalization forward and backward functions.

Declaration

public static cudnnStatus cudnnDeriveBNTensorDescriptor(cudnnTensorDescriptor derivedBnDesc, cudnnTensorDescriptor xDesc, cudnnBatchNormMode mode)

Parameters

Type	Name	Description
cudnnTensorDescriptor	derivedBnDesc
cudnnTensorDescriptor	xDesc
cudnnBatchNormMode	mode

Returns

Type	Description
cudnnStatus

cudnnDestroy(cudnnHandle)

This function releases hardware resources used by the cuDNN library. This function is usually the last call with a particular handle to the cuDNN library. Because cudnnCreate allocates some internal resources, the release of those resources by calling cudnnDestroy will implicitly call cudaDeviceSynchronize; therefore, the recommended best practice is to call cudnnCreate/cudnnDestroy outside of performance-critical code paths.

Declaration

public static cudnnStatus cudnnDestroy(cudnnHandle handle)

Parameters

Type	Name	Description
cudnnHandle	handle

Returns

Type	Description
cudnnStatus

cudnnDestroyActivationDescriptor(cudnnActivationDescriptor)

This function destroys a previously created activation descriptor object.

Declaration

public static cudnnStatus cudnnDestroyActivationDescriptor(cudnnActivationDescriptor activationDesc)

Parameters

Type	Name	Description
cudnnActivationDescriptor	activationDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyConvolutionDescriptor(cudnnConvolutionDescriptor)

This function destroys a previously created convolution descriptor object.

Declaration

public static cudnnStatus cudnnDestroyConvolutionDescriptor(cudnnConvolutionDescriptor convDesc)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyCTCLossDescriptor(cudnnCTCLossDescriptor)

Declaration

public static cudnnStatus cudnnDestroyCTCLossDescriptor(cudnnCTCLossDescriptor ctcLossDesc)

Parameters

Type	Name	Description
cudnnCTCLossDescriptor	ctcLossDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyDropoutDescriptor(cudnnDropoutDescriptor)

This function destroys a previously created dropout descriptor object.

Declaration

public static cudnnStatus cudnnDestroyDropoutDescriptor(cudnnDropoutDescriptor dropoutDesc)

Parameters

Type	Name	Description
cudnnDropoutDescriptor	dropoutDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyFilterDescriptor(cudnnFilterDescriptor)

This function destroys a previously created Tensor4D descriptor object.

Declaration

public static cudnnStatus cudnnDestroyFilterDescriptor(cudnnFilterDescriptor filterDesc)

Parameters

Type	Name	Description
cudnnFilterDescriptor	filterDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyLRNDescriptor(cudnnLRNDescriptor)

This function destroys a previously created LRN descriptor object.

Declaration

public static cudnnStatus cudnnDestroyLRNDescriptor(cudnnLRNDescriptor lrnDesc)

Parameters

Type	Name	Description
cudnnLRNDescriptor	lrnDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyOpTensorDescriptor(cudnnOpTensorDescriptor)

Declaration

public static cudnnStatus cudnnDestroyOpTensorDescriptor(cudnnOpTensorDescriptor opTensorDesc)

Parameters

Type	Name	Description
cudnnOpTensorDescriptor	opTensorDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyPersistentRNNPlan(cudnnPersistentRNNPlan)

This function destroys a previously created persistent RNN plan object.

Declaration

public static cudnnStatus cudnnDestroyPersistentRNNPlan(cudnnPersistentRNNPlan plan)

Parameters

Type	Name	Description
cudnnPersistentRNNPlan	plan

Returns

Type	Description
cudnnStatus

cudnnDestroyPoolingDescriptor(cudnnPoolingDescriptor)

This function destroys a previously created pooling descriptor object.

Declaration

public static cudnnStatus cudnnDestroyPoolingDescriptor(cudnnPoolingDescriptor poolingDesc)

Parameters

Type	Name	Description
cudnnPoolingDescriptor	poolingDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyReduceTensorDescriptor(cudnnReduceTensorDescriptor)

Declaration

public static cudnnStatus cudnnDestroyReduceTensorDescriptor(cudnnReduceTensorDescriptor reduceTensorDesc)

Parameters

Type	Name	Description
cudnnReduceTensorDescriptor	reduceTensorDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyRNNDescriptor(cudnnRNNDescriptor)

This function destroys a previously created RNN descriptor object.

Declaration

public static cudnnStatus cudnnDestroyRNNDescriptor(cudnnRNNDescriptor rnnDesc)

Parameters

Type	Name	Description
cudnnRNNDescriptor	rnnDesc

Returns

Type	Description
cudnnStatus

cudnnDestroySpatialTransformerDescriptor(cudnnSpatialTransformerDescriptor)

This function destroys a previously created spatial transformer descriptor object.

Declaration

public static cudnnStatus cudnnDestroySpatialTransformerDescriptor(cudnnSpatialTransformerDescriptor stDesc)

Parameters

Type	Name	Description
cudnnSpatialTransformerDescriptor	stDesc

Returns

Type	Description
cudnnStatus

cudnnDestroyTensorDescriptor(cudnnTensorDescriptor)

This function destroys a previously created Tensor descriptor object.

Declaration

public static cudnnStatus cudnnDestroyTensorDescriptor(cudnnTensorDescriptor tensorDesc)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc

Returns

Type	Description
cudnnStatus

cudnnDivisiveNormalizationBackward(cudnnHandle, cudnnLRNDescriptor, cudnnDivNormMode, ref Double, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr)

This function performs the backward DivisiveNormalization layer computation.

Declaration

public static cudnnStatus cudnnDivisiveNormalizationBackward(cudnnHandle handle, cudnnLRNDescriptor normDesc, cudnnDivNormMode mode, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, CUdeviceptr means, CUdeviceptr dy, CUdeviceptr temp, CUdeviceptr temp2, ref double beta, cudnnTensorDescriptor dXdMeansDesc, CUdeviceptr dx, CUdeviceptr dMeans)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnLRNDescriptor	normDesc	Handle to a previously intialized LRN parameter descriptor (this descriptor is used for both LRN and DivisiveNormalization layers).
cudnnDivNormMode	mode	DivisiveNormalization layer mode of operation. Currently only CUDNN_DIVNORM_PRECOMPUTED_MEANS is implemented. Normalization is performed using the means input tensor that is expected to be precomputed by the user.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Tensor descriptor and pointers in device memory for the bottom layer's data and means. (Bottom layer is the earlier layer in the computation graph during inference). Note: the means tensor is expected to be precomputed by the user. It can also contain any valid values (not required to be actual means, and can be for instance a result of a convolution with a Gaussian kernel).
CUdeviceptr	x	Tensor descriptor and pointers in device memory for the bottom layer's data and means. (Bottom layer is the earlier layer in the computation graph during inference). Note: the means tensor is expected to be precomputed by the user. It can also contain any valid values (not required to be actual means, and can be for instance a result of a convolution with a Gaussian kernel).
CUdeviceptr	means	Tensor descriptor and pointers in device memory for the bottom layer's data and means. (Bottom layer is the earlier layer in the computation graph during inference). Note: the means tensor is expected to be precomputed by the user. It can also contain any valid values (not required to be actual means, and can be for instance a result of a convolution with a Gaussian kernel).
CUdeviceptr	dy	Tensor pointer in device memory for the top layer's cumulative loss differential data (error backpropagation). (Top layer is the later layer in the computation graph during inference).
CUdeviceptr	temp	Temporary tensors in device memory. These are used for computing intermediate values during the backward pass. These tensors do not have to be preserved from forward to backward pass. Both use srcDesc as a descriptor.
CUdeviceptr	temp2	Temporary tensors in device memory. These are used for computing intermediate values during the backward pass. These tensors do not have to be preserved from forward to backward pass. Both use srcDesc as a descriptor.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dXdMeansDesc	Tensor descriptor for destDataDiff and destMeansDiff.
CUdeviceptr	dx	Tensor pointers (in device memory) for the bottom layer's resulting differentials (data and means). Both share the same descriptor.
CUdeviceptr	dMeans	Tensor pointers (in device memory) for the bottom layer's resulting differentials (data and means). Both share the same descriptor.

Returns

Type	Description
cudnnStatus

cudnnDivisiveNormalizationBackward(cudnnHandle, cudnnLRNDescriptor, cudnnDivNormMode, ref Single, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr)

This function performs the backward DivisiveNormalization layer computation.

Declaration

public static cudnnStatus cudnnDivisiveNormalizationBackward(cudnnHandle handle, cudnnLRNDescriptor normDesc, cudnnDivNormMode mode, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, CUdeviceptr means, CUdeviceptr dy, CUdeviceptr temp, CUdeviceptr temp2, ref float beta, cudnnTensorDescriptor dXdMeansDesc, CUdeviceptr dx, CUdeviceptr dMeans)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnLRNDescriptor	normDesc	Handle to a previously intialized LRN parameter descriptor (this descriptor is used for both LRN and DivisiveNormalization layers).
cudnnDivNormMode	mode	DivisiveNormalization layer mode of operation. Currently only CUDNN_DIVNORM_PRECOMPUTED_MEANS is implemented. Normalization is performed using the means input tensor that is expected to be precomputed by the user.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Tensor descriptor and pointers in device memory for the bottom layer's data and means. (Bottom layer is the earlier layer in the computation graph during inference). Note: the means tensor is expected to be precomputed by the user. It can also contain any valid values (not required to be actual means, and can be for instance a result of a convolution with a Gaussian kernel).
CUdeviceptr	x	Tensor descriptor and pointers in device memory for the bottom layer's data and means. (Bottom layer is the earlier layer in the computation graph during inference). Note: the means tensor is expected to be precomputed by the user. It can also contain any valid values (not required to be actual means, and can be for instance a result of a convolution with a Gaussian kernel).
CUdeviceptr	means	Tensor descriptor and pointers in device memory for the bottom layer's data and means. (Bottom layer is the earlier layer in the computation graph during inference). Note: the means tensor is expected to be precomputed by the user. It can also contain any valid values (not required to be actual means, and can be for instance a result of a convolution with a Gaussian kernel).
CUdeviceptr	dy	Tensor pointer in device memory for the top layer's cumulative loss differential data (error backpropagation). (Top layer is the later layer in the computation graph during inference).
CUdeviceptr	temp	Temporary tensors in device memory. These are used for computing intermediate values during the backward pass. These tensors do not have to be preserved from forward to backward pass. Both use srcDesc as a descriptor.
CUdeviceptr	temp2	Temporary tensors in device memory. These are used for computing intermediate values during the backward pass. These tensors do not have to be preserved from forward to backward pass. Both use srcDesc as a descriptor.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dXdMeansDesc	Tensor descriptor for destDataDiff and destMeansDiff.
CUdeviceptr	dx	Tensor pointers (in device memory) for the bottom layer's resulting differentials (data and means). Both share the same descriptor.
CUdeviceptr	dMeans	Tensor pointers (in device memory) for the bottom layer's resulting differentials (data and means). Both share the same descriptor.

Returns

Type	Description
cudnnStatus

cudnnDivisiveNormalizationForward(cudnnHandle, cudnnLRNDescriptor, cudnnDivNormMode, ref Double, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function performs the forward DivisiveNormalization layer computation.

Declaration

public static cudnnStatus cudnnDivisiveNormalizationForward(cudnnHandle handle, cudnnLRNDescriptor normDesc, cudnnDivNormMode mode, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, CUdeviceptr means, CUdeviceptr temp, CUdeviceptr temp2, ref double beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnLRNDescriptor	normDesc	Handle to a previously intialized LRN parameter descriptor. This descriptor is used for both LRN and DivisiveNormalization layers.
cudnnDivNormMode	mode	DivisiveNormalization layer mode of operation. Currently only CUDNN_DIVNORM_PRECOMPUTED_MEANS is implemented. Normalization is performed using the means input tensor that is expected to be precomputed by the user.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Tensor descriptor objects for the input and output tensors. Note that srcDesc is shared between srcData, srcMeansData, tempData, tempData2 tensors.
CUdeviceptr	x	Input tensor data pointer in device memory.
CUdeviceptr	means	Input means tensor data pointer in device memory. Note that this tensor can be NULL (in that case it's values are assumed to be zero during the computation). This tensor also doesn't have to contain means, these can be any values, a frequently used variation is a result of convolution with a normalized positive kernel (such as Gaussian).
CUdeviceptr	temp	Temporary tensors in device memory. These are used for computing intermediate values during the forward pass. These tensors do not have to be preserved as inputs from forward to the backward pass. Both use srcDesc as a descriptor.
CUdeviceptr	temp2	Temporary tensors in device memory. These are used for computing intermediate values during the forward pass. These tensors do not have to be preserved as inputs from forward to the backward pass. Both use srcDesc as a descriptor.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Tensor descriptor objects for the input and output tensors. Note that srcDesc is shared between srcData, srcMeansData, tempData, tempData2 tensors.
CUdeviceptr	y	Pointer in device memory to a tensor for the result of the forward DivisiveNormalization pass.

Returns

Type	Description
cudnnStatus

cudnnDivisiveNormalizationForward(cudnnHandle, cudnnLRNDescriptor, cudnnDivNormMode, ref Single, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This function performs the forward DivisiveNormalization layer computation.

Declaration

public static cudnnStatus cudnnDivisiveNormalizationForward(cudnnHandle handle, cudnnLRNDescriptor normDesc, cudnnDivNormMode mode, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, CUdeviceptr means, CUdeviceptr temp, CUdeviceptr temp2, ref float beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnLRNDescriptor	normDesc	Handle to a previously intialized LRN parameter descriptor. This descriptor is used for both LRN and DivisiveNormalization layers.
cudnnDivNormMode	mode	DivisiveNormalization layer mode of operation. Currently only CUDNN_DIVNORM_PRECOMPUTED_MEANS is implemented. Normalization is performed using the means input tensor that is expected to be precomputed by the user.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Tensor descriptor objects for the input and output tensors. Note that srcDesc is shared between srcData, srcMeansData, tempData, tempData2 tensors.
CUdeviceptr	x	Input tensor data pointer in device memory.
CUdeviceptr	means	Input means tensor data pointer in device memory. Note that this tensor can be NULL (in that case it's values are assumed to be zero during the computation). This tensor also doesn't have to contain means, these can be any values, a frequently used variation is a result of convolution with a normalized positive kernel (such as Gaussian).
CUdeviceptr	temp	Temporary tensors in device memory. These are used for computing intermediate values during the forward pass. These tensors do not have to be preserved as inputs from forward to the backward pass. Both use srcDesc as a descriptor.
CUdeviceptr	temp2	Temporary tensors in device memory. These are used for computing intermediate values during the forward pass. These tensors do not have to be preserved as inputs from forward to the backward pass. Both use srcDesc as a descriptor.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Tensor descriptor objects for the input and output tensors. Note that srcDesc is shared between srcData, srcMeansData, tempData, tempData2 tensors.
CUdeviceptr	y	Pointer in device memory to a tensor for the result of the forward DivisiveNormalization pass.

Returns

Type	Description
cudnnStatus

cudnnDropoutBackward(cudnnHandle, cudnnDropoutDescriptor, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, SizeT)

This function performs backward dropout operation over dy returning results in dx. If during forward dropout operation value from x was propagated to y then during backward operation value from dy will be propagated to dx, otherwise, dx value will be set to 0.

Declaration

public static cudnnStatus cudnnDropoutBackward(cudnnHandle handle, cudnnDropoutDescriptor dropoutDesc, cudnnTensorDescriptor dydesc, CUdeviceptr dy, cudnnTensorDescriptor dxdesc, CUdeviceptr dx, CUdeviceptr reserveSpace, SizeT reserveSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnDropoutDescriptor	dropoutDesc	Previously created dropout descriptor object.
cudnnTensorDescriptor	dydesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	dy	Pointer to data of the tensor described by the dyDesc descriptor.
cudnnTensorDescriptor	dxdesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	dx	Pointer to data of the tensor described by the dxDesc descriptor.
CUdeviceptr	reserveSpace	Pointer to user-allocated GPU memory used by this function. It is expected that reserveSpace was populated during a call to cudnnDropoutForward and has not been changed.
SizeT	reserveSpaceSizeInBytes	Specifies size in bytes of the provided memory for the reserve space.

Returns

Type	Description
cudnnStatus

cudnnDropoutForward(cudnnHandle, cudnnDropoutDescriptor, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, SizeT)

This function performs forward dropout operation over x returning results in y. If dropout was used as a parameter to cudnnSetDropoutDescriptor, the approximately dropout fraction of x values will be replaces by 0, and the rest will be scaled by 1/(1-dropout) This function should not be running concurrently with another cudnnDropoutForward function using the same states.

Declaration

public static cudnnStatus cudnnDropoutForward(cudnnHandle handle, cudnnDropoutDescriptor dropoutDesc, cudnnTensorDescriptor xdesc, CUdeviceptr x, cudnnTensorDescriptor ydesc, CUdeviceptr y, CUdeviceptr reserveSpace, SizeT reserveSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnDropoutDescriptor	dropoutDesc	Previously created dropout descriptor object.
cudnnTensorDescriptor	xdesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Pointer to data of the tensor described by the xDesc descriptor.
cudnnTensorDescriptor	ydesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Pointer to data of the tensor described by the yDesc descriptor.
CUdeviceptr	reserveSpace	Pointer to user-allocated GPU memory used by this function. It is expected that contents of reserveSpace doe not change between cudnnDropoutForward and cudnnDropoutBackward calls.
SizeT	reserveSpaceSizeInBytes	Specifies size in bytes of the provided memory for the reserve space.

Returns

Type	Description
cudnnStatus

cudnnDropoutGetReserveSpaceSize(cudnnTensorDescriptor, ref SizeT)

This function is used to query the amount of reserve needed to run dropout with the input dimensions given by xDesc. The same reserve space is expected to be passed to cudnnDropoutForward and cudnnDropoutBackward, and its contents is expected to remain unchanged between cudnnDropoutForward and cudnnDropoutBackward calls.

Declaration

public static cudnnStatus cudnnDropoutGetReserveSpaceSize(cudnnTensorDescriptor xdesc, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnTensorDescriptor	xdesc	Handle to a previously initialized tensor descriptor, describing input to a dropout operation.
SizeT	sizeInBytes	Amount of GPU memory needed as reserve space to be able to run dropout with an input tensor descriptor specified by xDesc.

Returns

Type	Description
cudnnStatus

cudnnDropoutGetStatesSize(cudnnHandle, ref SizeT)

This function is used to query the amount of space required to store the states of the random number generators used by cudnnDropoutForward function.

Declaration

public static cudnnStatus cudnnDropoutGetStatesSize(cudnnHandle handle, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle
SizeT	sizeInBytes

Returns

Type	Description
cudnnStatus

cudnnFindConvolutionBackwardDataAlgorithm(cudnnHandle, cudnnFilterDescriptor, cudnnTensorDescriptor, cudnnConvolutionDescriptor, cudnnTensorDescriptor, Int32, ref Int32, cudnnConvolutionBwdDataAlgoPerf[])

This function attempts all cuDNN algorithms for cudnnConvolutionBackwardData and outputs performance metrics to a user- allocated array of cudnnConvolutionBwdDataAlgoPerf_t. These metrics are written in sorted fashion where the first element has the lowest compute time.

Declaration

public static cudnnStatus cudnnFindConvolutionBackwardDataAlgorithm(cudnnHandle handle, cudnnFilterDescriptor wDesc, cudnnTensorDescriptor dyDesc, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor dxDesc, int requestedAlgoCount, ref int returnedAlgoCount, cudnnConvolutionBwdDataAlgoPerf[] perfResults)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output tensor descriptor.
System.Int32	requestedAlgoCount	The maximum number of elements to be stored in perfResults.
System.Int32	returnedAlgoCount	The number of output elements stored in perfResults.
cudnnConvolutionBwdDataAlgoPerf[]	perfResults	A user-allocated array to store performance metrics sorted ascending by compute time.

Returns

Type	Description
cudnnStatus

cudnnFindConvolutionBackwardFilterAlgorithm(cudnnHandle, cudnnTensorDescriptor, cudnnTensorDescriptor, cudnnConvolutionDescriptor, cudnnFilterDescriptor, Int32, ref Int32, cudnnConvolutionBwdFilterAlgoPerf[])

This function attempts all cuDNN algorithms for cudnnConvolutionBackwardFilter_v3 and outputs performance metrics to a user- allocated array of cudnnConvolutionBwdFilterAlgoPerf_t. These metrics are written in sorted fashion where the first element has the lowest compute time.

Declaration

public static cudnnStatus cudnnFindConvolutionBackwardFilterAlgorithm(cudnnHandle handle, cudnnTensorDescriptor xDesc, cudnnTensorDescriptor dyDesc, cudnnConvolutionDescriptor convDesc, cudnnFilterDescriptor dwDesc, int requestedAlgoCount, ref int returnedAlgoCount, cudnnConvolutionBwdFilterAlgoPerf[] perfResults)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnFilterDescriptor	dwDesc	Handle to a previously initialized filter descriptor.
System.Int32	requestedAlgoCount	The maximum number of elements to be stored in perfResults.
System.Int32	returnedAlgoCount	The number of output elements stored in perfResults.
cudnnConvolutionBwdFilterAlgoPerf[]	perfResults	A user-allocated array to store performance metrics sorted ascending by compute time.

Returns

Type	Description
cudnnStatus

cudnnFindConvolutionBackwardFilterAlgorithmEx(cudnnHandle, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnFilterDescriptor, CUdeviceptr, Int32, ref Int32, ref cudnnConvolutionBwdFilterAlgoPerf, CUdeviceptr, SizeT)

This function attempts all cuDNN algorithms for cudnnConvolutionBackwardFilter, using user-allocated GPU memory, and outputs performance metrics to a user-allocated array of cudnnConvolutionBwdFilterAlgoPerf_t. These metrics are written in sorted fashion where the first element has the lowest compute time. The workspace size should be the largest workspace you can spare in device memory; the size of this workspace will determine the availablity of convolution algorithms.

Declaration

public static cudnnStatus cudnnFindConvolutionBackwardFilterAlgorithmEx(cudnnHandle handle, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnConvolutionDescriptor convDesc, cudnnFilterDescriptor dwDesc, CUdeviceptr dw, int requestedAlgoCount, ref int returnedAlgoCount, ref cudnnConvolutionBwdFilterAlgoPerf perfResults, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the filter descriptor xDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor dyDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnFilterDescriptor	dwDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	dw	Data pointer to GPU memory associated with the filter descriptor dwDesc.The content of this tensor will be overwritten with arbitary values.
System.Int32	requestedAlgoCount	The maximum number of elements to be stored in perfResults.
System.Int32	returnedAlgoCount	The number of output elements stored in perfResults.
cudnnConvolutionBwdFilterAlgoPerf	perfResults	A user-allocated array to store performance metrics sorted ascending by compute time.
CUdeviceptr	workSpace	Data pointer to GPU memory that is a necessary workspace for some algorithms. The size of this workspace will determine the availabilty of algorithms. A nil pointer is considered a workSpace of 0 bytes.
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace.

Returns

Type	Description
cudnnStatus

cudnnFindConvolutionForwardAlgorithm(cudnnHandle, cudnnTensorDescriptor, cudnnFilterDescriptor, cudnnConvolutionDescriptor, cudnnTensorDescriptor, Int32, ref Int32, cudnnConvolutionFwdAlgoPerf[])

This function attempts all cuDNN algorithms and outputs performance metrics to a user-allocated array of cudnnConvolutionFwdAlgoPerf_t. These metrics are written in sorted fashion where the first element has the lowest compute time.

Declaration

public static cudnnStatus cudnnFindConvolutionForwardAlgorithm(cudnnHandle handle, cudnnTensorDescriptor srcDesc, cudnnFilterDescriptor filterDesc, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor destDesc, int requestedAlgoCount, ref int returnedAlgoCount, cudnnConvolutionFwdAlgoPerf[] perfResults)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	srcDesc	Handle to the previously initialized input tensor descriptor.
cudnnFilterDescriptor	filterDesc	Handle to a previously initialized filter descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnTensorDescriptor	destDesc	Handle to the previously initialized output tensor descriptor.
System.Int32	requestedAlgoCount	The maximum number of elements to be stored in perfResults.
System.Int32	returnedAlgoCount	The number of output elements stored in perfResults.
cudnnConvolutionFwdAlgoPerf[]	perfResults	A user-allocated array to store performance metrics sorted ascending by compute time.

Returns

Type	Description
cudnnStatus

cudnnFindConvolutionForwardAlgorithmEx(cudnnHandle, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, CUdeviceptr, cudnnConvolutionDescriptor, cudnnTensorDescriptor, CUdeviceptr, Int32, ref Int32, ref cudnnConvolutionFwdAlgoPerf, CUdeviceptr, SizeT)

This function attempts all available cuDNN algorithms for cudnnConvolutionForward, using user-allocated GPU memory, and outputs performance metrics to a user-allocated array of cudnnConvolutionFwdAlgoPerf_t. These metrics are written in sorted fashion where the first element has the lowest compute time. The workspace size should be the largest workspace you can spare in device memory; the size of this workspace will determine the availablity of the convolution algorithms.

Declaration

public static cudnnStatus cudnnFindConvolutionForwardAlgorithmEx(cudnnHandle handle, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor yDesc, CUdeviceptr y, int requestedAlgoCount, ref int returnedAlgoCount, ref cudnnConvolutionFwdAlgoPerf perfResults, CUdeviceptr workSpace, SizeT workSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor xDesc.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor wDesc.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor yDesc. The content of this tensor will be overwritten with arbitary values.
System.Int32	requestedAlgoCount	The maximum number of elements to be stored in perfResults.
System.Int32	returnedAlgoCount	The number of output elements stored in perfResults.
cudnnConvolutionFwdAlgoPerf	perfResults	A user-allocated array to store performance metrics sorted ascending by compute time.
CUdeviceptr	workSpace	Data pointer to GPU memory that is a necessary workspace for some algorithms. The size of this workspace will determine the availability of algorithms. A nil pointer is considered a workSpace of 0 bytes.
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workSpace.

Returns

Type	Description
cudnnStatus

cudnnGetActivationDescriptor(cudnnActivationDescriptor, ref cudnnActivationMode, ref cudnnNanPropagation, ref Double)

This function queries the parameters of the previouly initialized activation descriptor object.

Declaration

public static cudnnStatus cudnnGetActivationDescriptor(cudnnActivationDescriptor activationDesc, ref cudnnActivationMode mode, ref cudnnNanPropagation reluNanOpt, ref double coef)

Parameters

Type	Name	Description
cudnnActivationDescriptor	activationDesc	Handle to the previously created activation descriptor object.
cudnnActivationMode	mode	Enumerant to specify the activation mode.
cudnnNanPropagation	reluNanOpt	Nan propagation option for the relu.
System.Double	coef	floating point number to specify the clipping threashold when the activation mode is set to CUDNN_ACTIVATION_CLIPPED_RELU or to specify the alpha coefficient when the activation mode is set to CUDNN_ACTIVATION_ELU.

Returns

Type	Description
cudnnStatus

cudnnGetConvolution2dDescriptor(cudnnConvolutionDescriptor, ref Int32, ref Int32, ref Int32, ref Int32, ref Int32, ref Int32, ref cudnnConvolutionMode, ref cudnnDataType)

This function queries a previously initialized 2D convolution descriptor object.

Declaration

public static cudnnStatus cudnnGetConvolution2dDescriptor(cudnnConvolutionDescriptor convDesc, ref int pad_h, ref int pad_w, ref int u, ref int v, ref int dilation_h, ref int dilation_w, ref cudnnConvolutionMode mode, ref cudnnDataType dataType)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc	Handle to a previously created convolution descriptor.
System.Int32	pad_h	zero-padding height: number of rows of zeros implicitly concatenated onto the top and onto the bottom of input images.
System.Int32	pad_w	zero-padding width: number of columns of zeros implicitly concatenated onto the left and onto the right of input images.
System.Int32	u	Vertical filter stride.
System.Int32	v	Horizontal filter stride.
System.Int32	dilation_h	Filter height dilation.
System.Int32	dilation_w	Filter width dilation.
cudnnConvolutionMode	mode	Selects between CUDNN_CONVOLUTION and CUDNN_CROSS_CORRELATION.
cudnnDataType	dataType	Data type.

Returns

Type	Description
cudnnStatus

cudnnGetConvolution2dForwardOutputDim(cudnnConvolutionDescriptor, cudnnTensorDescriptor, cudnnFilterDescriptor, ref Int32, ref Int32, ref Int32, ref Int32)

This function returns the dimensions of the resulting 4D tensor of a 2D convolution, given the convolution descriptor, the input tensor descriptor and the filter descriptor This function can help to setup the output tensor and allocate the proper amount of memory prior to launch the actual convolution.

Each dimension h and w of the output images is computed as followed:

outputDim = 1 + (inputDim + 2*pad - filterDim)/convolutionStride;

Declaration

public static cudnnStatus cudnnGetConvolution2dForwardOutputDim(cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor inputTensorDesc, cudnnFilterDescriptor filterDesc, ref int n, ref int c, ref int h, ref int w)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc	Handle to a previously created convolution descriptor.
cudnnTensorDescriptor	inputTensorDesc	Handle to a previously initialized tensor descriptor.
cudnnFilterDescriptor	filterDesc	Handle to a previously initialized filter descriptor.
System.Int32	n	Number of output images.
System.Int32	c	Number of output feature maps per image.
System.Int32	h	Height of each output feature map.
System.Int32	w	Width of each output feature map.

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionBackwardDataAlgorithm(cudnnHandle, cudnnFilterDescriptor, cudnnTensorDescriptor, cudnnConvolutionDescriptor, cudnnTensorDescriptor, cudnnConvolutionBwdDataPreference, SizeT, ref cudnnConvolutionBwdDataAlgo)

This function serves as a heuristic for obtaining the best suited algorithm for cudnnConvolutionBackwardData for the given layer specifications. Based on the input preference, this function will either return the fastest algorithm or the fastest algorithm within a given memory limit. For an exhaustive search for the fastest algorithm, please use cudnnFindConvolutionBackwardDataAlgorithm.

Declaration

public static cudnnStatus cudnnGetConvolutionBackwardDataAlgorithm(cudnnHandle handle, cudnnFilterDescriptor wDesc, cudnnTensorDescriptor dyDesc, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor dxDesc, cudnnConvolutionBwdDataPreference preference, SizeT memoryLimitInbytes, ref cudnnConvolutionBwdDataAlgo algo)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output tensor descriptor.
cudnnConvolutionBwdDataPreference	preference	Enumerant to express the preference criteria in terms of memory requirement and speed.
SizeT	memoryLimitInbytes	It is to specify the maximum amount of GPU memory the user is willing to use as a workspace. This is currently a placeholder and is not used.
cudnnConvolutionBwdDataAlgo	algo	Enumerant that specifies which convolution algorithm should be used to compute the results according to the specified preference

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionBackwardDataAlgorithm_v7(cudnnHandle, cudnnFilterDescriptor, cudnnTensorDescriptor, cudnnConvolutionDescriptor, cudnnTensorDescriptor, Int32, ref Int32, cudnnConvolutionBwdDataAlgoPerf[])

Declaration

public static cudnnStatus cudnnGetConvolutionBackwardDataAlgorithm_v7(cudnnHandle handle, cudnnFilterDescriptor filterDesc, cudnnTensorDescriptor diffDesc, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor gradDesc, int requestedAlgoCount, ref int returnedAlgoCount, cudnnConvolutionBwdDataAlgoPerf[] perfResults)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnFilterDescriptor	filterDesc
cudnnTensorDescriptor	diffDesc
cudnnConvolutionDescriptor	convDesc
cudnnTensorDescriptor	gradDesc
System.Int32	requestedAlgoCount
System.Int32	returnedAlgoCount
cudnnConvolutionBwdDataAlgoPerf[]	perfResults

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionBackwardDataAlgorithmMaxCount(cudnnHandle, ref Int32)

Declaration

public static cudnnStatus cudnnGetConvolutionBackwardDataAlgorithmMaxCount(cudnnHandle handle, ref int count)

Parameters

Type	Name	Description
cudnnHandle	handle
System.Int32	count

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionBackwardDataWorkspaceSize(cudnnHandle, cudnnFilterDescriptor, cudnnTensorDescriptor, cudnnConvolutionDescriptor, cudnnTensorDescriptor, cudnnConvolutionBwdDataAlgo, ref SizeT)

This function returns the amount of GPU memory workspace the user needs to allocate to be able to call cudnnConvolutionBackwardData with the specified algorithm. The workspace allocated will then be passed to the routine cudnnConvolutionBackwardData. The specified algorithm can be the result of the call to cudnnGetConvolutionBackwardDataAlgorithm or can be chosen arbitrarily by the user. Note that not every algorithm is available for every configuration of the input tensor and/or every configuration of the convolution descriptor.

Declaration

public static cudnnStatus cudnnGetConvolutionBackwardDataWorkspaceSize(cudnnHandle handle, cudnnFilterDescriptor wDesc, cudnnTensorDescriptor dyDesc, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor dxDesc, cudnnConvolutionBwdDataAlgo algo, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output tensor descriptor.
cudnnConvolutionBwdDataAlgo	algo	Enumerant that specifies the chosen convolution algorithm
SizeT	sizeInBytes	Amount of GPU memory needed as workspace to be able to execute a forward convolution with the specified algo

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionBackwardFilterAlgorithm(cudnnHandle, cudnnTensorDescriptor, cudnnTensorDescriptor, cudnnConvolutionDescriptor, cudnnFilterDescriptor, cudnnConvolutionBwdFilterPreference, SizeT, ref cudnnConvolutionBwdFilterAlgo)

This function serves as a heuristic for obtaining the best suited algorithm for cudnnConvolutionBackwardFilter for the given layer specifications. Based on the input preference, this function will either return the fastest algorithm or the fastest algorithm within a given memory limit. For an exhaustive search for the fastest algorithm, please use cudnnFindConvolutionBackwardFilterAlgorithm.

Declaration

public static cudnnStatus cudnnGetConvolutionBackwardFilterAlgorithm(cudnnHandle handle, cudnnTensorDescriptor xDesc, cudnnTensorDescriptor dyDesc, cudnnConvolutionDescriptor convDesc, cudnnFilterDescriptor dwDesc, cudnnConvolutionBwdFilterPreference preference, SizeT memoryLimitInbytes, ref cudnnConvolutionBwdFilterAlgo algo)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnFilterDescriptor	dwDesc	Handle to a previously initialized filter descriptor.
cudnnConvolutionBwdFilterPreference	preference	Enumerant to express the preference criteria in terms of memory requirement and speed.
SizeT	memoryLimitInbytes	It is to specify the maximum amount of GPU memory the user is willing to use as a workspace. This is currently a placeholder and is not used.
cudnnConvolutionBwdFilterAlgo	algo	Enumerant that specifies which convolution algorithm should be used to compute the results according to the specified preference

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionBackwardFilterAlgorithm_v7(cudnnHandle, cudnnTensorDescriptor, cudnnTensorDescriptor, cudnnConvolutionDescriptor, cudnnFilterDescriptor, Int32, ref Int32, cudnnConvolutionBwdFilterAlgoPerf[])

Declaration

public static cudnnStatus cudnnGetConvolutionBackwardFilterAlgorithm_v7(cudnnHandle handle, cudnnTensorDescriptor srcDesc, cudnnTensorDescriptor diffDesc, cudnnConvolutionDescriptor convDesc, cudnnFilterDescriptor gradDesc, int requestedAlgoCount, ref int returnedAlgoCount, cudnnConvolutionBwdFilterAlgoPerf[] perfResults)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnTensorDescriptor	srcDesc
cudnnTensorDescriptor	diffDesc
cudnnConvolutionDescriptor	convDesc
cudnnFilterDescriptor	gradDesc
System.Int32	requestedAlgoCount
System.Int32	returnedAlgoCount
cudnnConvolutionBwdFilterAlgoPerf[]	perfResults

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionBackwardFilterAlgorithmMaxCount(cudnnHandle, ref Int32)

Declaration

public static cudnnStatus cudnnGetConvolutionBackwardFilterAlgorithmMaxCount(cudnnHandle handle, ref int count)

Parameters

Type	Name	Description
cudnnHandle	handle
System.Int32	count

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionBackwardFilterWorkspaceSize(cudnnHandle, cudnnTensorDescriptor, cudnnTensorDescriptor, cudnnConvolutionDescriptor, cudnnFilterDescriptor, cudnnConvolutionBwdFilterAlgo, ref SizeT)

This function returns the amount of GPU memory workspace the user needs to allocate to be able to call cudnnConvolutionBackwardFilter_v3 with the specified algorithm. The workspace allocated will then be passed to the routine cudnnConvolutionBackwardFilter. The specified algorithm can be the result of the call to cudnnGetConvolutionBackwardFilterAlgorithm or can be chosen arbitrarily by the user. Note that not every algorithm is available for every configuration of the input tensor and/or every configuration of the convolution descriptor.

Declaration

public static cudnnStatus cudnnGetConvolutionBackwardFilterWorkspaceSize(cudnnHandle handle, cudnnTensorDescriptor xDesc, cudnnTensorDescriptor dyDesc, cudnnConvolutionDescriptor convDesc, cudnnFilterDescriptor gradDesc, cudnnConvolutionBwdFilterAlgo algo, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnFilterDescriptor	gradDesc	Handle to a previously initialized filter descriptor.
cudnnConvolutionBwdFilterAlgo	algo	Enumerant that specifies the chosen convolution algorithm sizeInBytes output Amount of GPU memory needed as workspace to be able to execute
SizeT	sizeInBytes	Amount of GPU memory needed as workspace to be able to execute a forward convolution with the specified algo

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionForwardAlgorithm(cudnnHandle, cudnnTensorDescriptor, cudnnFilterDescriptor, cudnnConvolutionDescriptor, cudnnTensorDescriptor, cudnnConvolutionFwdPreference, SizeT, ref cudnnConvolutionFwdAlgo)

This function serves as a heuristic for obtaining the best suited algorithm for cudnnConvolutionForward for the given layer specifications. Based on the input preference, this function will either return the fastest algorithm or the fastest algorithm within a given memory limit. For an exhaustive search for the fastest algorithm, please use cudnnFindConvolutionForwardAlgorithm.

Declaration

public static cudnnStatus cudnnGetConvolutionForwardAlgorithm(cudnnHandle handle, cudnnTensorDescriptor xDesc, cudnnFilterDescriptor filterDesc, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor yDesc, cudnnConvolutionFwdPreference preference, SizeT memoryLimitInbytes, ref cudnnConvolutionFwdAlgo algo)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
cudnnFilterDescriptor	filterDesc	Handle to a previously initialized filter descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
cudnnConvolutionFwdPreference	preference	Enumerant to express the preference criteria in terms of memory requirement and speed.
SizeT	memoryLimitInbytes	It is used when enumerant preference is set to CUDNN_CONVOLUTION_FWD_SPECIFY_WORKSPACE_LIMIT to specify the maximum amount of GPU memory the user is willing to use as a workspace
cudnnConvolutionFwdAlgo	algo	Enumerant that specifies which convolution algorithm should be used to compute the results according to the specified preference

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionForwardAlgorithm_v7(cudnnHandle, cudnnTensorDescriptor, cudnnFilterDescriptor, cudnnConvolutionDescriptor, cudnnTensorDescriptor, Int32, ref Int32, cudnnConvolutionFwdAlgoPerf[])

Declaration

public static cudnnStatus cudnnGetConvolutionForwardAlgorithm_v7(cudnnHandle handle, cudnnTensorDescriptor srcDesc, cudnnFilterDescriptor filterDesc, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor destDesc, int requestedAlgoCount, ref int returnedAlgoCount, cudnnConvolutionFwdAlgoPerf[] perfResults)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnTensorDescriptor	srcDesc
cudnnFilterDescriptor	filterDesc
cudnnConvolutionDescriptor	convDesc
cudnnTensorDescriptor	destDesc
System.Int32	requestedAlgoCount
System.Int32	returnedAlgoCount
cudnnConvolutionFwdAlgoPerf[]	perfResults

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionForwardAlgorithmMaxCount(cudnnHandle, ref Int32)

Declaration

public static cudnnStatus cudnnGetConvolutionForwardAlgorithmMaxCount(cudnnHandle handle, ref int count)

Parameters

Type	Name	Description
cudnnHandle	handle
System.Int32	count

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionForwardWorkspaceSize(cudnnHandle, cudnnTensorDescriptor, cudnnFilterDescriptor, cudnnConvolutionDescriptor, cudnnTensorDescriptor, cudnnConvolutionFwdAlgo, ref SizeT)

This function returns the amount of GPU memory workspace the user needs to allocate to be able to call cudnnConvolutionForward with the specified algorithm. The workspace allocated will then be passed to the routine cudnnConvolutionForward. The specified algorithm can be the result of the call to cudnnGetConvolutionForwardAlgorithm or can be chosen arbitrarily by the user. Note that not every algorithm is available for every configuration of the input tensor and/or every configuration of the convolution descriptor.

Declaration

public static cudnnStatus cudnnGetConvolutionForwardWorkspaceSize(cudnnHandle handle, cudnnTensorDescriptor xDesc, cudnnFilterDescriptor filterDesc, cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor yDesc, cudnnConvolutionFwdAlgo algo, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
cudnnFilterDescriptor	filterDesc	Handle to a previously initialized filter descriptor.
cudnnConvolutionDescriptor	convDesc	Previously initialized convolution descriptor.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
cudnnConvolutionFwdAlgo	algo	Enumerant that specifies the chosen convolution algorithm
SizeT	sizeInBytes	Amount of GPU memory needed as workspace to be able to execute a forward convolution with the specified algo

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionGroupCount(cudnnConvolutionDescriptor, ref Int32)

Declaration

public static cudnnStatus cudnnGetConvolutionGroupCount(cudnnConvolutionDescriptor convDesc, ref int groupCount)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc
System.Int32	groupCount

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionMathType(cudnnConvolutionDescriptor, ref cudnnMathType)

Declaration

public static cudnnStatus cudnnGetConvolutionMathType(cudnnConvolutionDescriptor convDesc, ref cudnnMathType mathType)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc
cudnnMathType	mathType

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionNdDescriptor(cudnnConvolutionDescriptor, Int32, ref Int32, Int32[], Int32[], Int32[], ref cudnnConvolutionMode, ref cudnnDataType)

This function queries a previously initialized convolution descriptor object.

Declaration

public static cudnnStatus cudnnGetConvolutionNdDescriptor(cudnnConvolutionDescriptor convDesc, int arrayLengthRequested, ref int arrayLength, int[] padA, int[] strideA, int[] dilationA, ref cudnnConvolutionMode mode, ref cudnnDataType computeType)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc	Handle to a previously created convolution descriptor.
System.Int32	arrayLengthRequested	Dimension of the expected convolution descriptor. It is also the minimum size of the arrays padA, filterStrideA and upsacleA in order to be able to hold the results
System.Int32	arrayLength	actual dimension of the convolution descriptor.
System.Int32[]	padA	Array of dimension of at least arrayLengthRequested that will be filled with the padding parameters from the provided convolution descriptor.
System.Int32[]	strideA	Array of dimension of at least arrayLengthRequested that will be filled with the filter stride from the provided convolution descriptor.
System.Int32[]	dilationA	Array of dimension at least arrayLengthRequested that will be filled with the dilation parameters from the provided convolution descriptor.
cudnnConvolutionMode	mode	convolution mode of the provided descriptor.
cudnnDataType	computeType	datatype of the provided descriptor.

Returns

Type	Description
cudnnStatus

cudnnGetConvolutionNdForwardOutputDim(cudnnConvolutionDescriptor, cudnnTensorDescriptor, cudnnFilterDescriptor, Int32, Int32[])

This function returns the dimensions of the resulting n-D tensor of a nbDims-2-D convolution, given the convolution descriptor, the input tensor descriptor and the filter descriptor This function can help to setup the output tensor and allocate the proper amount of memory prior to launch the actual convolution.

Each dimension of the (nbDims-2)-D images of the output tensor is computed as followed:

outputDim = 1 + (inputDim + 2*pad - filterDim)/convolutionStride;

Declaration

public static cudnnStatus cudnnGetConvolutionNdForwardOutputDim(cudnnConvolutionDescriptor convDesc, cudnnTensorDescriptor inputTensorDesc, cudnnFilterDescriptor filterDesc, int nbDims, int[] tensorOuputDimA)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc	Handle to a previously created convolution descriptor.
cudnnTensorDescriptor	inputTensorDesc	Handle to a previously initialized tensor descriptor.
cudnnFilterDescriptor	filterDesc	Handle to a previously initialized filter descriptor.
System.Int32	nbDims	Dimension of the output tensor
System.Int32[]	tensorOuputDimA	Array of dimensions nbDims that contains on exit of this routine the sizes of the output tensor

Returns

Type	Description
cudnnStatus

cudnnGetCTCLossDescriptor(cudnnCTCLossDescriptor, ref cudnnDataType)

Declaration

public static cudnnStatus cudnnGetCTCLossDescriptor(cudnnCTCLossDescriptor ctcLossDesc, ref cudnnDataType compType)

Parameters

Type	Name	Description
cudnnCTCLossDescriptor	ctcLossDesc
cudnnDataType	compType

Returns

Type	Description
cudnnStatus

cudnnGetCTCLossWorkspaceSize(cudnnHandle, cudnnTensorDescriptor, cudnnTensorDescriptor, Int32[], Int32[], Int32[], cudnnCTCLossAlgo, cudnnCTCLossDescriptor, ref SizeT)

return the workspace size needed for ctc

Declaration

public static cudnnStatus cudnnGetCTCLossWorkspaceSize(cudnnHandle handle, cudnnTensorDescriptor probsDesc, cudnnTensorDescriptor gradientsDesc, int[] labels, int[] labelLengths, int[] inputLengths, cudnnCTCLossAlgo algo, cudnnCTCLossDescriptor ctcLossDesc, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnTensorDescriptor	probsDesc
cudnnTensorDescriptor	gradientsDesc
System.Int32[]	labels
System.Int32[]	labelLengths
System.Int32[]	inputLengths
cudnnCTCLossAlgo	algo
cudnnCTCLossDescriptor	ctcLossDesc
SizeT	sizeInBytes

Returns

Type	Description
cudnnStatus

cudnnGetDropoutDescriptor(cudnnDropoutDescriptor, cudnnHandle, ref Single, ref CUdeviceptr, ref UInt64)

Declaration

public static cudnnStatus cudnnGetDropoutDescriptor(cudnnDropoutDescriptor dropoutDesc, cudnnHandle handle, ref float dropout, ref CUdeviceptr states, ref ulong seed)

Parameters

Type	Name	Description
cudnnDropoutDescriptor	dropoutDesc
cudnnHandle	handle
System.Single	dropout
CUdeviceptr	states
System.UInt64	seed

Returns

Type	Description
cudnnStatus

| Improve this Doc View Source

cudnnGetErrorString(cudnnStatus)

This function returns a human-readable character string describing the cudnnStatus enumerate passed as input parameter.

Declaration

public static string cudnnGetErrorString(cudnnStatus status)

Parameters

Type	Name	Description
cudnnStatus	status

Returns

Type	Description
System.String

cudnnGetFilter4dDescriptor(cudnnFilterDescriptor, ref cudnnDataType, ref cudnnTensorFormat, ref Int32, ref Int32, ref Int32, ref Int32)

This function queries the parameters of the previouly initialized filter descriptor object. v4 version of the function also has the format parameter.

Declaration

public static cudnnStatus cudnnGetFilter4dDescriptor(cudnnFilterDescriptor filterDesc, ref cudnnDataType dataType, ref cudnnTensorFormat format, ref int k, ref int c, ref int h, ref int w)

Parameters

Type	Name	Description
cudnnFilterDescriptor	filterDesc	Handle to a previously created filter descriptor.
cudnnDataType	dataType	Data type.
cudnnTensorFormat	format	Layout format.
System.Int32	k	Number of output feature maps.
System.Int32	c	Number of input feature maps.
System.Int32	h	Height of each filter.
System.Int32	w	Width of each filter.

Returns

Type	Description
cudnnStatus

cudnnGetFilterNdDescriptor(cudnnFilterDescriptor, Int32, ref cudnnDataType, ref cudnnTensorFormat, ref Int32, Int32[])

This function queries a previously initialized filter descriptor object. v4 version of the function also has the format parameter.

Declaration

public static cudnnStatus cudnnGetFilterNdDescriptor(cudnnFilterDescriptor filterDesc, int nbDimsRequested, ref cudnnDataType dataType, ref cudnnTensorFormat format, ref int nbDims, int[] filterDimA)

Parameters

Type	Name	Description
cudnnFilterDescriptor	filterDesc	Handle to a previously initialized filter descriptor.
System.Int32	nbDimsRequested	Dimension of the expected filter descriptor. It is also the minimum size of the arrays filterDimA in order to be able to hold the results
cudnnDataType	dataType	Data type.
cudnnTensorFormat	format	Layout format.
System.Int32	nbDims	Actual dimension of the filter.
System.Int32[]	filterDimA	Array of dimension of at least nbDimsRequested that will be filled with the filter parameters from the provided filter descriptor.

Returns

Type	Description
cudnnStatus

cudnnGetLRNDescriptor(cudnnLRNDescriptor, ref UInt32, ref Double, ref Double, ref Double)

This function retrieves values stored in the previously initialized LRN descriptor object.

Declaration

public static cudnnStatus cudnnGetLRNDescriptor(cudnnLRNDescriptor normDesc, ref uint lrnN, ref double lrnAlpha, ref double lrnBeta, ref double lrnK)

Parameters

Type	Name	Description
cudnnLRNDescriptor	normDesc	Handle to a previously created LRN descriptor.
System.UInt32	lrnN	Pointers to receive values of parameters stored in the descriptor object. See cudnnSetLRNDescriptor for more details. Any of these pointers can be NULL (no value is returned for the corresponding parameter).
System.Double	lrnAlpha	Pointers to receive values of parameters stored in the descriptor object. See cudnnSetLRNDescriptor for more details. Any of these pointers can be NULL (no value is returned for the corresponding parameter).
System.Double	lrnBeta	Pointers to receive values of parameters stored in the descriptor object. See cudnnSetLRNDescriptor for more details. Any of these pointers can be NULL (no value is returned for the corresponding parameter).
System.Double	lrnK	Pointers to receive values of parameters stored in the descriptor object. See cudnnSetLRNDescriptor for more details. Any of these pointers can be NULL (no value is returned for the corresponding parameter).

Returns

Type	Description
cudnnStatus

cudnnGetOpTensorDescriptor(cudnnOpTensorDescriptor, ref cudnnOpTensorOp, ref cudnnDataType, ref cudnnNanPropagation)

Declaration

public static cudnnStatus cudnnGetOpTensorDescriptor(cudnnOpTensorDescriptor opTensorDesc, ref cudnnOpTensorOp opTensorOp, ref cudnnDataType opTensorCompType, ref cudnnNanPropagation opTensorNanOpt)

Parameters

Type	Name	Description
cudnnOpTensorDescriptor	opTensorDesc
cudnnOpTensorOp	opTensorOp
cudnnDataType	opTensorCompType
cudnnNanPropagation	opTensorNanOpt

Returns

Type	Description
cudnnStatus

cudnnGetPooling2dDescriptor(cudnnPoolingDescriptor, ref cudnnPoolingMode, ref cudnnNanPropagation, ref Int32, ref Int32, ref Int32, ref Int32, ref Int32, ref Int32)

This function queries a previously created 2D pooling descriptor object.

Declaration

public static cudnnStatus cudnnGetPooling2dDescriptor(cudnnPoolingDescriptor poolingDesc, ref cudnnPoolingMode mode, ref cudnnNanPropagation maxpoolingNanOpt, ref int windowHeight, ref int windowWidth, ref int verticalPadding, ref int horizontalPadding, ref int verticalStride, ref int horizontalStride)

Parameters

Type	Name	Description
cudnnPoolingDescriptor	poolingDesc	Handle to a previously created pooling descriptor.
cudnnPoolingMode	mode	Enumerant to specify the pooling mode.
cudnnNanPropagation	maxpoolingNanOpt	Nan propagation option for max pooling.
System.Int32	windowHeight	Height of the pooling window.
System.Int32	windowWidth	Width of the pooling window.
System.Int32	verticalPadding	Size of vertical padding.
System.Int32	horizontalPadding	Size of horizontal padding.
System.Int32	verticalStride	Pooling vertical stride.
System.Int32	horizontalStride	Pooling horizontal stride.

Returns

Type	Description
cudnnStatus

cudnnGetPooling2dForwardOutputDim(cudnnPoolingDescriptor, cudnnTensorDescriptor, ref Int32, ref Int32, ref Int32, ref Int32)

This function provides the output dimensions of a tensor after 2d pooling has been applied

Declaration

public static cudnnStatus cudnnGetPooling2dForwardOutputDim(cudnnPoolingDescriptor poolingDesc, cudnnTensorDescriptor inputTensorDesc, ref int n, ref int c, ref int h, ref int w)

Parameters

Type	Name	Description
cudnnPoolingDescriptor	poolingDesc	Handle to a previously inititalized pooling descriptor.
cudnnTensorDescriptor	inputTensorDesc	Handle to the previously initialized input tensor descriptor.
System.Int32	n	Number of images in the output
System.Int32	c	Number of channels in the output
System.Int32	h	Height of images in the output
System.Int32	w	Width of images in the output

Returns

Type	Description
cudnnStatus

cudnnGetPoolingNdDescriptor(cudnnPoolingDescriptor, Int32, ref cudnnPoolingMode, ref cudnnNanPropagation, ref Int32, Int32[], Int32[], Int32[])

This function queries a previously initialized generic pooling descriptor object.

Declaration

public static cudnnStatus cudnnGetPoolingNdDescriptor(cudnnPoolingDescriptor poolingDesc, int nbDimsRequested, ref cudnnPoolingMode mode, ref cudnnNanPropagation maxpoolingNanOpt, ref int nbDims, int[] windowDimA, int[] paddingA, int[] strideA)

Parameters

Type	Name	Description
cudnnPoolingDescriptor	poolingDesc	Handle to a previously created pooling descriptor.
System.Int32	nbDimsRequested	Dimension of the expected pooling descriptor. It is also the minimum size of the arrays windowDimA, paddingA and strideA in order to be able to hold the results
cudnnPoolingMode	mode	Enumerant to specify the pooling mode.
cudnnNanPropagation	maxpoolingNanOpt	Nan propagation option for max pooling.
System.Int32	nbDims	Actual dimension of the pooling descriptor.
System.Int32[]	windowDimA	Array of dimension of at least nbDimsRequested that will be filled with the window parameters from the provided pooling descriptor.
System.Int32[]	paddingA	Array of dimension of at least nbDimsRequested that will be filled with the padding parameters from the provided pooling descriptor.
System.Int32[]	strideA	Array of dimension at least nbDimsRequested that will be filled with the stride parameters from the provided pooling descriptor.

Returns

Type	Description
cudnnStatus

cudnnGetPoolingNdForwardOutputDim(cudnnPoolingDescriptor, cudnnTensorDescriptor, Int32, Int32[])

This function provides the output dimensions of a tensor after Nd pooling has been applied

Declaration

public static cudnnStatus cudnnGetPoolingNdForwardOutputDim(cudnnPoolingDescriptor poolingDesc, cudnnTensorDescriptor inputTensorDesc, int nbDims, int[] outputTensorDimA)

Parameters

Type	Name	Description
cudnnPoolingDescriptor	poolingDesc	Handle to a previously inititalized pooling descriptor.
cudnnTensorDescriptor	inputTensorDesc	Handle to the previously initialized input tensor descriptor.
System.Int32	nbDims	Number of dimensions in which pooling is to be applied.
System.Int32[]	outputTensorDimA	Array of nbDims output dimensions

Returns

Type	Description
cudnnStatus

cudnnGetProperty(libraryPropertyType, ref Int32)

Declaration

public static void cudnnGetProperty(libraryPropertyType type, ref int value)

Parameters

Type	Name	Description
libraryPropertyType	type
System.Int32	value

cudnnGetReduceTensorDescriptor(cudnnReduceTensorDescriptor, ref cudnnReduceTensorOp, ref cudnnDataType, ref cudnnNanPropagation, ref cudnnReduceTensorIndices, ref cudnnIndicesType)

Declaration

public static cudnnStatus cudnnGetReduceTensorDescriptor(cudnnReduceTensorDescriptor reduceTensorDesc, ref cudnnReduceTensorOp reduceTensorOp, ref cudnnDataType reduceTensorCompType, ref cudnnNanPropagation reduceTensorNanOpt, ref cudnnReduceTensorIndices reduceTensorIndices, ref cudnnIndicesType reduceTensorIndicesType)

Parameters

Type	Name	Description
cudnnReduceTensorDescriptor	reduceTensorDesc
cudnnReduceTensorOp	reduceTensorOp
cudnnDataType	reduceTensorCompType
cudnnNanPropagation	reduceTensorNanOpt
cudnnReduceTensorIndices	reduceTensorIndices
cudnnIndicesType	reduceTensorIndicesType

Returns

Type	Description
cudnnStatus

cudnnGetReductionIndicesSize(cudnnHandle, cudnnReduceTensorDescriptor, cudnnTensorDescriptor, cudnnTensorDescriptor, ref SizeT)

Helper function to return the minimum size of the index space to be passed to the reduction given the input and output tensors

Declaration

public static cudnnStatus cudnnGetReductionIndicesSize(cudnnHandle handle, cudnnReduceTensorDescriptor reduceTensorDesc, cudnnTensorDescriptor aDesc, cudnnTensorDescriptor cDesc, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnReduceTensorDescriptor	reduceTensorDesc
cudnnTensorDescriptor	aDesc
cudnnTensorDescriptor	cDesc
SizeT	sizeInBytes

Returns

Type	Description
cudnnStatus

cudnnGetReductionWorkspaceSize(cudnnHandle, cudnnReduceTensorDescriptor, cudnnTensorDescriptor, cudnnTensorDescriptor, ref SizeT)

Helper function to return the minimum size of the workspace to be passed to the reduction given the input and output tensors

Declaration

public static cudnnStatus cudnnGetReductionWorkspaceSize(cudnnHandle handle, cudnnReduceTensorDescriptor reduceTensorDesc, cudnnTensorDescriptor aDesc, cudnnTensorDescriptor cDesc, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnReduceTensorDescriptor	reduceTensorDesc
cudnnTensorDescriptor	aDesc
cudnnTensorDescriptor	cDesc
SizeT	sizeInBytes

Returns

Type	Description
cudnnStatus

cudnnGetRNNDescriptor(cudnnHandle, cudnnRNNDescriptor, ref Int32, ref Int32, ref cudnnDropoutDescriptor, ref cudnnRNNInputMode, ref cudnnDirectionMode, ref cudnnRNNMode, ref cudnnRNNAlgo, ref cudnnDataType)

Declaration

public static cudnnStatus cudnnGetRNNDescriptor(cudnnHandle cudnnHandle, cudnnRNNDescriptor rnnDesc, ref int hiddenSize, ref int numLayers, ref cudnnDropoutDescriptor dropoutDesc, ref cudnnRNNInputMode inputMode, ref cudnnDirectionMode direction, ref cudnnRNNMode mode, ref cudnnRNNAlgo algo, ref cudnnDataType dataType)

Parameters

Type	Name	Description
cudnnHandle	cudnnHandle
cudnnRNNDescriptor	rnnDesc
System.Int32	hiddenSize
System.Int32	numLayers
cudnnDropoutDescriptor	dropoutDesc
cudnnRNNInputMode	inputMode
cudnnDirectionMode	direction
cudnnRNNMode	mode
cudnnRNNAlgo	algo
cudnnDataType	dataType

Returns

Type	Description
cudnnStatus

cudnnGetRNNLinLayerBiasParams(cudnnHandle, cudnnRNNDescriptor, Int32, cudnnTensorDescriptor, cudnnFilterDescriptor, CUdeviceptr, Int32, cudnnFilterDescriptor, CUdeviceptr)

This function is used to obtain a pointer and descriptor for the bias parameters in layer within the RNN described by rnnDesc with inputs dimensions defined by xDesc.

Declaration

public static cudnnStatus cudnnGetRNNLinLayerBiasParams(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, int layer, cudnnTensorDescriptor xDesc, cudnnFilterDescriptor wDesc, CUdeviceptr w, int linLayerID, cudnnFilterDescriptor linLayerBiasDesc, CUdeviceptr linLayerBias)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
System.Int32	layer	The layer to query.
cudnnTensorDescriptor	xDesc	A fully packed tensor descriptor describing the input to one recurrent iteration.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor describing the weights for the RNN.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor wDesc.
System.Int32	linLayerID	The linear layer to obtain information about: If mode in rnnDesc was set to CUDNN_RNN_RELU or CUDNN_RNN_TANH a value of 0 references the bias applied to the input from the previous layer, a value of 1 references the bias applied to the recurrent input. If mode in rnnDesc was set to CUDNN_LSTM values of 0, 1, 2 and 3 reference bias applied to the input from the previous layer, value of 4, 5, 6 and 7 reference bias applied to the recurrent input. ‣ Values 0 and 4 reference the input gate. ‣ Values 1 and 5 reference the forget gate. ‣ Values 2 and 6 reference the new memory gate. ‣ Values 3 and 7 reference the output gate. If mode in rnnDesc was set to CUDNN_GRU values of 0, 1 and 2 reference bias applied to the input from the previous layer, value of 3, 4 and 5 reference bias applied to the recurrent input. ‣ Values 0 and 3 reference the reset gate. ‣ Values 1 and 4 reference the update gate. ‣ Values 2 and 5 reference the new memory gate.
cudnnFilterDescriptor	linLayerBiasDesc	Handle to a previously created filter descriptor.
CUdeviceptr	linLayerBias	Data pointer to GPU memory associated with the filter descriptor linLayerMatDesc.

Returns

Type	Description
cudnnStatus

cudnnGetRNNLinLayerMatrixParams(cudnnHandle, cudnnRNNDescriptor, Int32, cudnnTensorDescriptor, cudnnFilterDescriptor, CUdeviceptr, Int32, cudnnFilterDescriptor, CUdeviceptr)

This function is used to obtain a pointer and descriptor for the matrix parameters in layer within the RNN described by rnnDesc with inputs dimensions defined by xDesc.

Declaration

public static cudnnStatus cudnnGetRNNLinLayerMatrixParams(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, int layer, cudnnTensorDescriptor xDesc, cudnnFilterDescriptor wDesc, CUdeviceptr w, int linLayerID, cudnnFilterDescriptor linLayerMatDesc, CUdeviceptr linLayerMat)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
System.Int32	layer	The layer to query.
cudnnTensorDescriptor	xDesc	A fully packed tensor descriptor describing the input to one recurrent iteration.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor describing the weights for the RNN.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor wDesc.
System.Int32	linLayerID	The linear layer to obtain information about: If mode in rnnDesc was set to CUDNN_RNN_RELU or CUDNN_RNN_TANH a value of 0 references the matrix multiplication applied to the input from the previous layer, a value of 1 references the matrix multiplication applied to the recurrent input. If mode in rnnDesc was set to CUDNN_LSTM values of 0-3 reference matrix multiplications applied to the input from the previous layer, value of 4-7 reference matrix multiplications applied to the recurrent input. ‣ Values 0 and 4 reference the input gate. ‣ Values 1 and 5 reference the forget gate. ‣ Values 2 and 6 reference the new memory gate. ‣ Values 3 and 7 reference the output gate. If mode in rnnDesc was set to CUDNN_GRU values of 0-2 reference matrix multiplications applied to the input from the previous layer, value of 3-5 reference matrix multiplications applied to the recurrent input. ‣ Values 0 and 3 reference the reset gate. ‣ Values 1 and 4 reference the update gate. ‣ Values 2 and 5 reference the new memory gate.
cudnnFilterDescriptor	linLayerMatDesc	Handle to a previously created filter descriptor.
CUdeviceptr	linLayerMat	Data pointer to GPU memory associated with the filter descriptor linLayerMatDesc.

Returns

Type	Description
cudnnStatus

cudnnGetRNNParamsSize(cudnnHandle, cudnnRNNDescriptor, cudnnTensorDescriptor, ref SizeT, cudnnDataType)

This function is used to query the amount of parameter space required to execute the RNN described by rnnDesc with inputs dimensions defined by xDesc.

Declaration

public static cudnnStatus cudnnGetRNNParamsSize(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, cudnnTensorDescriptor xDesc, ref SizeT sizeInBytes, cudnnDataType dataType)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
cudnnTensorDescriptor	xDesc	A fully packed tensor descriptor describing the input to one recurrent iteration.
SizeT	sizeInBytes	Minimum amount of GPU memory needed as parameter space to be able to execute an RNN with the specified descriptor and input tensors.
cudnnDataType	dataType	The data type of the parameters.

Returns

Type	Description
cudnnStatus

cudnnGetRNNTrainingReserveSize(cudnnHandle, cudnnRNNDescriptor, Int32, cudnnTensorDescriptor[], ref SizeT)

This function is used to query the amount of reserved space required for training the RNN described by rnnDesc with inputs dimensions defined by xDesc. The same reserve space must be passed to cudnnRNNForwardTraining, cudnnRNNBackwardData and cudnnRNNBackwardWeights.

Declaration

public static cudnnStatus cudnnGetRNNTrainingReserveSize(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, int seqLength, cudnnTensorDescriptor[] xDesc, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
System.Int32	seqLength	Number of iterations to unroll over.
cudnnTensorDescriptor[]	xDesc	An array of tensor descriptors describing the input to each recurrent iteration.
SizeT	sizeInBytes	Minimum amount of GPU memory needed as reserve space to be able to train an RNN with the specified descriptor and input tensors.

Returns

Type	Description
cudnnStatus

cudnnGetRNNWorkspaceSize(cudnnHandle, cudnnRNNDescriptor, Int32, cudnnTensorDescriptor[], ref SizeT)

This function is used to query the amount of work space required to execute the RNN described by rnnDesc with inputs dimensions defined by xDesc.

Declaration

public static cudnnStatus cudnnGetRNNWorkspaceSize(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, int seqLength, cudnnTensorDescriptor[] xDesc, ref SizeT sizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
System.Int32	seqLength	Number of iterations to unroll over.
cudnnTensorDescriptor[]	xDesc	An array of tensor descriptors describing the input to each recurrent iteration.
SizeT	sizeInBytes	Minimum amount of GPU memory needed as workspace to be able to execute an RNN with the specified descriptor and input tensors.

Returns

Type	Description
cudnnStatus

cudnnGetStream(cudnnHandle, ref CUstream)

This function gets the cuDNN library stream, which is being used to execute all calls to the cuDNN library functions. If the cuDNN library stream is not set, all kernels use the default NULL stream.

Declaration

public static cudnnStatus cudnnGetStream(cudnnHandle handle, ref CUstream streamId)

Parameters

Type	Name	Description
cudnnHandle	handle
CUstream	streamId

Returns

Type	Description
cudnnStatus

cudnnGetTensor4dDescriptor(cudnnTensorDescriptor, ref cudnnDataType, ref Int32, ref Int32, ref Int32, ref Int32, ref Int32, ref Int32, ref Int32, ref Int32)

This function queries the parameters of the previouly initialized Tensor4D descriptor object.

Declaration

public static cudnnStatus cudnnGetTensor4dDescriptor(cudnnTensorDescriptor tensorDesc, ref cudnnDataType dataType, ref int n, ref int c, ref int h, ref int w, ref int nStride, ref int cStride, ref int hStride, ref int wStride)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc	Handle to a previously insitialized tensor descriptor.
cudnnDataType	dataType	Data type.
System.Int32	n	Number of images.
System.Int32	c	Number of feature maps per image.
System.Int32	h	Height of each feature map.
System.Int32	w	Width of each feature map.
System.Int32	nStride	Stride between two consecutive images.
System.Int32	cStride	Stride between two consecutive feature maps.
System.Int32	hStride	Stride between two consecutive rows.
System.Int32	wStride	Stride between two consecutive columns.

Returns

Type	Description
cudnnStatus

cudnnGetTensorNdDescriptor(cudnnTensorDescriptor, Int32, ref cudnnDataType, ref Int32, Int32[], Int32[])

This function retrieves values stored in a previously initialized Tensor descriptor object.

Declaration

public static cudnnStatus cudnnGetTensorNdDescriptor(cudnnTensorDescriptor tensorDesc, int nbDimsRequested, ref cudnnDataType dataType, ref int nbDims, int[] dimA, int[] strideA)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc	Handle to a previously initialized tensor descriptor.
System.Int32	nbDimsRequested	Number of dimensions to extract from a given tensor descriptor. It is also the minimum size of the arrays dimA and strideA. If this number is greater than the resulting nbDims[0], only nbDims[0] dimensions will be returned.
cudnnDataType	dataType	Data type.
System.Int32	nbDims	Actual number of dimensions of the tensor will be returned in nbDims[0].
System.Int32[]	dimA	Array of dimension of at least nbDimsRequested that will be filled with the dimensions from the provided tensor descriptor.
System.Int32[]	strideA	Array of dimension of at least nbDimsRequested that will be filled with the strides from the provided tensor descriptor.

Returns

Type	Description
cudnnStatus

cudnnGetTensorSizeInBytes(cudnnTensorDescriptor, ref SizeT)

This function returns the size of the tensor in memory in respect to the given descriptor. This function can be used to know the amount of GPU memory to be allocated to hold that tensor.

Declaration

public static cudnnStatus cudnnGetTensorSizeInBytes(cudnnTensorDescriptor tensorDesc, ref SizeT size)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc	Handle to a previously initialized tensor descriptor.
SizeT	size	Size in bytes needed to hold the tensor in GPU memory.

Returns

Type	Description
cudnnStatus

| Improve this Doc View Source

cudnnGetVersion()

This function returns the version number of the cuDNN Library. It returns the CUDNN_VERSION define present in the cudnn.h header file. Starting with release R2, the routine can be used to identify dynamically the current cuDNN Library used by the application. The define CUDNN_VERSION can be used to have the same application linked against different cuDNN versions using conditional compilation statements.

Declaration

public static Version cudnnGetVersion()

Returns

Type	Description
System.Version

cudnnIm2Col(cudnnHandle, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, cudnnConvolutionDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnIm2Col(cudnnHandle handle, cudnnTensorDescriptor xDesc, CUdeviceptr x, cudnnFilterDescriptor wDesc, cudnnConvolutionDescriptor convDesc, CUdeviceptr colBuffer)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
cudnnFilterDescriptor	wDesc
cudnnConvolutionDescriptor	convDesc
CUdeviceptr	colBuffer

Returns

Type	Description
cudnnStatus

cudnnLRNCrossChannelBackward(cudnnHandle, cudnnLRNDescriptor, cudnnLRNMode, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function performs the backward LRN layer computation.

Declaration

public static cudnnStatus cudnnLRNCrossChannelBackward(cudnnHandle handle, cudnnLRNDescriptor normDesc, cudnnLRNMode lrnMode, ref double alpha, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnLRNDescriptor	normDesc	Handle to a previously intialized LRN parameter descriptor.
cudnnLRNMode	lrnMode	LRN layer mode of operation. Currently only CUDNN_LRN_CROSS_CHANNEL_DIM1 is implemented. Normalization is performed along the tensor's dimA[1].
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Tensor descriptor and pointer in device memory for the bottom layer's data. (Bottom layer is the earlier layer in the computation graph during inference).
CUdeviceptr	y	Tensor descriptor and pointer in device memory for the bottom layer's data. (Bottom layer is the earlier layer in the computation graph during inference).
cudnnTensorDescriptor	dyDesc	Tensor descriptor and pointer in device memory for the top layer's cumulative loss differential data (error backpropagation). (Top layer is the later layer in the computation graph during inference).
CUdeviceptr	dy	Tensor descriptor and pointer in device memory for the top layer's cumulative loss differential data (error backpropagation). (Top layer is the later layer in the computation graph during inference).
cudnnTensorDescriptor	xDesc	Tensor descriptor and pointer in device memory for the bottom layer's data. (Bottom layer is the earlier layer in the computation graph during inference). Note that these values are not modified during backpropagation.
CUdeviceptr	x	Tensor descriptor and pointer in device memory for the bottom layer's data. (Bottom layer is the earlier layer in the computation graph during inference). Note that these values are not modified during backpropagation.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Tensor descriptor and pointer in device memory for the bottom layer's cumulative loss differential data (error backpropagation). (Bottom layer is the earlier layer in the computation graph during inference).
CUdeviceptr	dx	Tensor descriptor and pointer in device memory for the bottom layer's cumulative loss differential data (error backpropagation). (Bottom layer is the earlier layer in the computation graph during inference).

Returns

Type	Description
cudnnStatus

cudnnLRNCrossChannelBackward(cudnnHandle, cudnnLRNDescriptor, cudnnLRNMode, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This function performs the backward LRN layer computation.

Declaration

public static cudnnStatus cudnnLRNCrossChannelBackward(cudnnHandle handle, cudnnLRNDescriptor normDesc, cudnnLRNMode lrnMode, ref float alpha, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnLRNDescriptor	normDesc	Handle to a previously intialized LRN parameter descriptor.
cudnnLRNMode	lrnMode	LRN layer mode of operation. Currently only CUDNN_LRN_CROSS_CHANNEL_DIM1 is implemented. Normalization is performed along the tensor's dimA[1].
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Tensor descriptor and pointer in device memory for the bottom layer's data. (Bottom layer is the earlier layer in the computation graph during inference).
CUdeviceptr	y	Tensor descriptor and pointer in device memory for the bottom layer's data. (Bottom layer is the earlier layer in the computation graph during inference).
cudnnTensorDescriptor	dyDesc	Tensor descriptor and pointer in device memory for the top layer's cumulative loss differential data (error backpropagation). (Top layer is the later layer in the computation graph during inference).
CUdeviceptr	dy	Tensor descriptor and pointer in device memory for the top layer's cumulative loss differential data (error backpropagation). (Top layer is the later layer in the computation graph during inference).
cudnnTensorDescriptor	xDesc	Tensor descriptor and pointer in device memory for the bottom layer's data. (Bottom layer is the earlier layer in the computation graph during inference). Note that these values are not modified during backpropagation.
CUdeviceptr	x	Tensor descriptor and pointer in device memory for the bottom layer's data. (Bottom layer is the earlier layer in the computation graph during inference). Note that these values are not modified during backpropagation.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Tensor descriptor and pointer in device memory for the bottom layer's cumulative loss differential data (error backpropagation). (Bottom layer is the earlier layer in the computation graph during inference).
CUdeviceptr	dx	Tensor descriptor and pointer in device memory for the bottom layer's cumulative loss differential data (error backpropagation). (Bottom layer is the earlier layer in the computation graph during inference).

Returns

Type	Description
cudnnStatus

cudnnLRNCrossChannelForward(cudnnHandle, cudnnLRNDescriptor, cudnnLRNMode, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function performs the forward LRN layer computation.

Declaration

public static cudnnStatus cudnnLRNCrossChannelForward(cudnnHandle handle, cudnnLRNDescriptor normDesc, cudnnLRNMode lrnMode, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnLRNDescriptor	normDesc	Handle to a previously intialized LRN parameter descriptor.
cudnnLRNMode	lrnMode	LRN layer mode of operation. Currently only CUDNN_LRN_CROSS_CHANNEL_DIM1 is implemented. Normalization is performed along the tensor's dimA[1].
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Tensor descriptor objects for the input and output tensors.
CUdeviceptr	x	Input tensor data pointer in device memory.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Tensor descriptor objects for the input and output tensors.
CUdeviceptr	y	Output tensor data pointer in device memory.

Returns

Type	Description
cudnnStatus

cudnnLRNCrossChannelForward(cudnnHandle, cudnnLRNDescriptor, cudnnLRNMode, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This function performs the forward LRN layer computation.

Declaration

public static cudnnStatus cudnnLRNCrossChannelForward(cudnnHandle handle, cudnnLRNDescriptor normDesc, cudnnLRNMode lrnMode, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnLRNDescriptor	normDesc	Handle to a previously intialized LRN parameter descriptor.
cudnnLRNMode	lrnMode	LRN layer mode of operation. Currently only CUDNN_LRN_CROSS_CHANNEL_DIM1 is implemented. Normalization is performed along the tensor's dimA[1].
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Tensor descriptor objects for the input and output tensors.
CUdeviceptr	x	Input tensor data pointer in device memory.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows: dstValue = alpha[0]resultValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Tensor descriptor objects for the input and output tensors.
CUdeviceptr	y	Output tensor data pointer in device memory.

Returns

Type	Description
cudnnStatus

cudnnOpTensor(cudnnHandle, cudnnOpTensorDescriptor, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function implements the equation C = op ( alpha1[0] * A, alpha2[0] * B ) + beta[0] * C, given tensors A, B, and C and scaling factors alpha1, alpha2, and beta. The op to use is indicated by the descriptor opTensorDesc. Currently-supported ops are listed by the cudnnOpTensorOp_t enum. Each dimension of the input tensor A must match the corresponding dimension of the destination tensor C, and each dimension of the input tensor B must match the corresponding dimension of the destination tensor C or must be equal to 1. In the latter case, the same value from the input tensor B for those dimensions will be used to blend into the C tensor. The data types of the input tensors A and B must match. If the data type of the destination tensor C is double, then the data type of the input tensors also must be double. If the data type of the destination tensor C is double, then opTensorCompType in opTensorDesc must be double. Else opTensorCompType must be float. If the input tensor B is the same tensor as the destination tensor C, then the input tensor A also must be the same tensor as the destination tensor C.

Declaration

public static cudnnStatus cudnnOpTensor(cudnnHandle handle, cudnnOpTensorDescriptor opTensorDesc, ref double alpha1, cudnnTensorDescriptor aDesc, CUdeviceptr A, ref double alpha2, cudnnTensorDescriptor bDesc, CUdeviceptr B, ref double beta, cudnnTensorDescriptor cDesc, CUdeviceptr C)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnOpTensorDescriptor	opTensorDesc	Handle to a previously initialized op tensor descriptor.
System.Double	alpha1	Pointer to scaling factors(in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	aDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	A	Pointer to data of the tensors described by the aDesc descriptor.
System.Double	alpha2	Pointer to scaling factors(in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	bDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	B	Pointer to data of the tensors described by the bDesc descriptor.
System.Double	beta	Pointer to scaling factors(in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	cDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	C	Pointer to data of the tensor described by the cDesc descriptor.

Returns

Type	Description
cudnnStatus

cudnnOpTensor(cudnnHandle, cudnnOpTensorDescriptor, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnOpTensor(cudnnHandle handle, cudnnOpTensorDescriptor opTensorDesc, ref float alpha1, cudnnTensorDescriptor aDesc, CUdeviceptr A, ref float alpha2, cudnnTensorDescriptor bDesc, CUdeviceptr B, ref float beta, cudnnTensorDescriptor cDesc, CUdeviceptr C)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnOpTensorDescriptor	opTensorDesc	Handle to a previously initialized op tensor descriptor.
System.Single	alpha1	Pointer to scaling factors(in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	aDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	A	Pointer to data of the tensors described by the aDesc descriptor.
System.Single	alpha2	Pointer to scaling factors(in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	bDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	B	Pointer to data of the tensors described by the bDesc descriptor.
System.Single	beta	Pointer to scaling factors(in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	cDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	C	Pointer to data of the tensor described by the cDesc descriptor.

Returns

Type	Description
cudnnStatus

cudnnPoolingBackward(cudnnHandle, cudnnPoolingDescriptor, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function computes the gradient of a pooling operation.

Declaration

public static cudnnStatus cudnnPoolingBackward(cudnnHandle handle, cudnnPoolingDescriptor poolingDesc, ref double alpha, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnPoolingDescriptor	poolingDesc	Handle to the previously initialized pooling descriptor.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor srcDiffData.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the output tensor descriptor destDesc.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output differential tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor destDiffDesc.

Returns

Type	Description
cudnnStatus

cudnnPoolingBackward(cudnnHandle, cudnnPoolingDescriptor, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This function computes the gradient of a pooling operation.

Declaration

public static cudnnStatus cudnnPoolingBackward(cudnnHandle handle, cudnnPoolingDescriptor poolingDesc, ref float alpha, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnPoolingDescriptor	poolingDesc	Handle to the previously initialized pooling descriptor.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor srcDiffData.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the output tensor descriptor destDesc.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output differential tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor destDiffDesc.

Returns

Type	Description
cudnnStatus

cudnnPoolingForward(cudnnHandle, cudnnPoolingDescriptor, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function computes pooling of input values (i.e., the maximum or average of several adjacent values) to produce an output with smaller height and/or width.

Declaration

public static cudnnStatus cudnnPoolingForward(cudnnHandle handle, cudnnPoolingDescriptor poolingDesc, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnPoolingDescriptor	poolingDesc	Handle to a previously initialized pooling descriptor.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor destDesc.

Returns

Type	Description
cudnnStatus

cudnnPoolingForward(cudnnHandle, cudnnPoolingDescriptor, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This function computes pooling of input values (i.e., the maximum or average of several adjacent values) to produce an output with smaller height and/or width.

Declaration

public static cudnnStatus cudnnPoolingForward(cudnnHandle handle, cudnnPoolingDescriptor poolingDesc, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnPoolingDescriptor	poolingDesc	Handle to a previously initialized pooling descriptor.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor destDesc.

Returns

Type	Description
cudnnStatus

cudnnQueryRuntimeError(cudnnHandle, ref cudnnStatus, cudnnErrQueryMode, cudnnRuntimeTag)

Declaration

public static cudnnStatus cudnnQueryRuntimeError(cudnnHandle handle, ref cudnnStatus rstatus, cudnnErrQueryMode mode, cudnnRuntimeTag tag)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnStatus	rstatus
cudnnErrQueryMode	mode
cudnnRuntimeTag	tag

Returns

Type	Description
cudnnStatus

cudnnReduceTensor(cudnnHandle, cudnnReduceTensorDescriptor, CUdeviceptr, SizeT, CUdeviceptr, SizeT, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function reduces tensor A by implementing the equation C = alpha * reduce op ( A )

beta* C, given tensors A and C and scaling factors alpha and beta.The reduction op to use is indicated by the descriptor reduceTensorDesc.Currently-supported ops are listed by the cudnnReduceTensorOp_t enum.

Declaration

public static cudnnStatus cudnnReduceTensor(cudnnHandle handle, cudnnReduceTensorDescriptor reduceTensorDesc, CUdeviceptr indices, SizeT indicesSizeInBytes, CUdeviceptr workspace, SizeT workspaceSizeInBytes, ref double alpha, cudnnTensorDescriptor aDesc, CUdeviceptr A, ref double beta, cudnnTensorDescriptor cDesc, CUdeviceptr C)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnReduceTensorDescriptor	reduceTensorDesc	Handle to a previously initialized reduce tensor descriptor.
CUdeviceptr	indices	Handle to a previously allocated space for writing indices.
SizeT	indicesSizeInBytes	Size of the above previously allocated space.
CUdeviceptr	workspace	Handle to a previously allocated space for the reduction implementation.
SizeT	workspaceSizeInBytes	Size of the above previously allocated space.
System.Double	alpha	Pointer to scaling factor (in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	aDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	A	Pointer to data of the tensor described by the aDesc descriptor.
System.Double	beta	Pointer to scaling factor (in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	cDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	C	Pointer to data of the tensor described by the cDesc descriptor.

Returns

Type	Description
cudnnStatus

cudnnReduceTensor(cudnnHandle, cudnnReduceTensorDescriptor, CUdeviceptr, SizeT, CUdeviceptr, SizeT, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This function reduces tensor A by implementing the equation C = alpha * reduce op ( A )

beta* C, given tensors A and C and scaling factors alpha and beta.The reduction op to use is indicated by the descriptor reduceTensorDesc.Currently-supported ops are listed by the cudnnReduceTensorOp_t enum.

Declaration

public static cudnnStatus cudnnReduceTensor(cudnnHandle handle, cudnnReduceTensorDescriptor reduceTensorDesc, CUdeviceptr indices, SizeT indicesSizeInBytes, CUdeviceptr workspace, SizeT workspaceSizeInBytes, ref float alpha, cudnnTensorDescriptor aDesc, CUdeviceptr A, ref float beta, cudnnTensorDescriptor cDesc, CUdeviceptr C)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnReduceTensorDescriptor	reduceTensorDesc	Handle to a previously initialized reduce tensor descriptor.
CUdeviceptr	indices	Handle to a previously allocated space for writing indices.
SizeT	indicesSizeInBytes	Size of the above previously allocated space.
CUdeviceptr	workspace	Handle to a previously allocated space for the reduction implementation.
SizeT	workspaceSizeInBytes	Size of the above previously allocated space.
System.Single	alpha	Pointer to scaling factor (in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	aDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	A	Pointer to data of the tensor described by the aDesc descriptor.
System.Single	beta	Pointer to scaling factor (in host memory) used to blend the source value with prior value in the destination tensor as indicated by the above op equation.
cudnnTensorDescriptor	cDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	C	Pointer to data of the tensor described by the cDesc descriptor.

Returns

Type	Description
cudnnStatus

cudnnRestoreDropoutDescriptor(cudnnDropoutDescriptor, cudnnHandle, Single, CUdeviceptr, SizeT, UInt64)

Restores the dropout descriptor to a previously saved-off state

Declaration

public static cudnnStatus cudnnRestoreDropoutDescriptor(cudnnDropoutDescriptor dropoutDesc, cudnnHandle handle, float dropout, CUdeviceptr states, SizeT stateSizeInBytes, ulong seed)

Parameters

Type	Name	Description
cudnnDropoutDescriptor	dropoutDesc
cudnnHandle	handle
System.Single	dropout
CUdeviceptr	states
SizeT	stateSizeInBytes
System.UInt64	seed

Returns

Type	Description
cudnnStatus

cudnnRNNBackwardData(cudnnHandle, cudnnRNNDescriptor, cudnnTensorDescriptor[], CUdeviceptr, cudnnTensorDescriptor[], CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor[], CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, SizeT, CUdeviceptr, SizeT)

This routine executes the recurrent neural network described by rnnDesc with output gradients dy, dhy, dhc, weights w and input gradients dx, dhx, dcx. workspace is required for intermediate storage. The data in reserveSpace must have previously been generated by cudnnRNNForwardTraining. The same reserveSpace data must be used for future calls to cudnnRNNBackwardWeights if they execute on the same input data.

Declaration

public static cudnnStatus cudnnRNNBackwardData(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, cudnnTensorDescriptor[] yDesc, CUdeviceptr y, cudnnTensorDescriptor[] dyDesc, CUdeviceptr dy, cudnnTensorDescriptor dhyDesc, CUdeviceptr dhy, cudnnTensorDescriptor dcyDesc, CUdeviceptr dcy, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnTensorDescriptor hxDesc, CUdeviceptr hx, cudnnTensorDescriptor cxDesc, CUdeviceptr cx, cudnnTensorDescriptor[] dxDesc, CUdeviceptr dx, cudnnTensorDescriptor dhxDesc, CUdeviceptr dhx, cudnnTensorDescriptor dcxDesc, CUdeviceptr dcx, CUdeviceptr workspace, SizeT workSpaceSizeInBytes, CUdeviceptr reserveSpace, SizeT reserveSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
cudnnTensorDescriptor[]	yDesc	An array of tensor descriptors describing the output from each recurrent iteration. The first dimension of the tensor depends on the direction argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc: If direction is CUDNN_UNIDIRECTIONAL the first dimension should match the hiddenSize argument passed to cudnnSetRNNDescriptor. If direction is CUDNN_BIDIRECTIONAL the first dimension should match double the hiddenSize argument passed to cudnnSetRNNDescriptor. The second dimension of the tensor n must match the second dimension of the tensor n in dyDesc. The tensor must be fully packed.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor yDesc.
cudnnTensorDescriptor[]	dyDesc	An array of tensor descriptors describing the gradient at the output from each recurrent iteration. The first dimension of the tensor depends on the direction argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc: If direction is CUDNN_UNIDIRECTIONAL the first dimension should match the hiddenSize argument passed to cudnnSetRNNDescriptor. If direction is CUDNN_BIDIRECTIONAL the first dimension should match double the hiddenSize argument passed to cudnnSetRNNDescriptor. The second dimension of the tensor n must match the second dimension of the tensor n in dxDesc. The tensor must be fully packed.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptors in the array dyDesc.
cudnnTensorDescriptor	dhyDesc	Handle to a previously initialized tensor descriptor describing the gradients at the final hidden state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in dyDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	dhy	Data pointer to GPU memory associated with the tensor descriptor dhyDesc. If a NULL pointer is passed, the gradients at the final hidden state of the network will be initialized to zero.
cudnnTensorDescriptor	dcyDesc	Handle to a previously initialized tensor descriptor describing the gradients at the final cell state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in dyDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	dcy	Data pointer to GPU memory associated with the tensor descriptor dcyDesc. If a NULL pointer is passed, the gradients at the final cell state of the network will be initialized to zero.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor describing the weights for the RNN.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor wDesc.
cudnnTensorDescriptor	hxDesc	Handle to a previously initialized tensor descriptor describing the initial hidden state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	hx	Data pointer to GPU memory associated with the tensor descriptor hxDesc. If a NULL pointer is passed, the initial hidden state of the network will be initialized to zero.
cudnnTensorDescriptor	cxDesc	Handle to a previously initialized tensor descriptor describing the initial cell state for LSTM networks. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	cx	Data pointer to GPU memory associated with the tensor descriptor cxDesc. If a NULL pointer is passed, the initial cell state of the network will be initialized to zero.
cudnnTensorDescriptor[]	dxDesc	An array of tensor descriptors describing the gradient at the input of each recurrent iteration. Each tensor descriptor must have the same first dimension. The second dimension of the tensors may decrease from element n to element n+1 but may not increase. The tensor must be fully packed.
CUdeviceptr	dx	Data pointer to GPU memory associated with the tensor descriptors in the array dxDesc.
cudnnTensorDescriptor	dhxDesc	Handle to a previously initialized tensor descriptor describing the gradient at the initial hidden state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	dhx	Data pointer to GPU memory associated with the tensor descriptor dhxDesc. If a NULL pointer is passed, the gradient at the hidden input of the network will not be set.
cudnnTensorDescriptor	dcxDesc	Handle to a previously initialized tensor descriptor describing the gradient at the initial cell state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	dcx	Data pointer to GPU memory associated with the tensor descriptor dcxDesc. If a NULL pointer is passed, the gradient at the cell input of the network will not be set.
CUdeviceptr	workspace	Data pointer to GPU memory to be used as a workspace for this call.
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workspace.
CUdeviceptr	reserveSpace	Data pointer to GPU memory to be used as a reserve space for this call.
SizeT	reserveSpaceSizeInBytes	Specifies the size in bytes of the provided reserveSpace.

Returns

Type	Description
cudnnStatus

cudnnRNNBackwardWeights(cudnnHandle, cudnnRNNDescriptor, cudnnTensorDescriptor[], CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor[], CUdeviceptr, CUdeviceptr, SizeT, cudnnFilterDescriptor, CUdeviceptr, CUdeviceptr, SizeT)

This routine accumulates weight gradients dw from the recurrent neural network described by rnnDesc with inputs x, hx, and outputs y. The mode of operation in this case is additive, the weight gradients calculated will be added to those already existing in dw. workspace is required for intermediate storage. The data in reserveSpace must have previously been generated by cudnnRNNBackwardData.

Declaration

public static cudnnStatus cudnnRNNBackwardWeights(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, cudnnTensorDescriptor[] xDesc, CUdeviceptr x, cudnnTensorDescriptor hxDesc, CUdeviceptr hx, cudnnTensorDescriptor[] yDesc, CUdeviceptr y, CUdeviceptr workspace, SizeT workSpaceSizeInBytes, cudnnFilterDescriptor dwDesc, CUdeviceptr dw, CUdeviceptr reserveSpace, SizeT reserveSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
cudnnTensorDescriptor[]	xDesc	An array of tensor descriptors describing the input to each recurrent iteration. Each tensor descriptor must have the same first dimension. The second dimension of the tensors may decrease from element n to element n+1 but may not increase. The tensor must be fully packed.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptors in the array xDesc.
cudnnTensorDescriptor	hxDesc	Handle to a previously initialized tensor descriptor describing the initial hidden state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	hx	Data pointer to GPU memory associated with the tensor descriptor hxDesc. If a NULL pointer is passed, the initial hidden state of the network will be initialized to zero.
cudnnTensorDescriptor[]	yDesc	An array of tensor descriptors describing the output from each recurrent iteration. The first dimension of the tensor depends on the direction argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc: If direction is CUDNN_UNIDIRECTIONAL the first dimension should match the hiddenSize argument passed to cudnnSetRNNDescriptor. If direction is CUDNN_BIDIRECTIONAL the first dimension should match double the hiddenSize argument passed to cudnnSetRNNDescriptor. The second dimension of the tensor n must match the second dimension of the tensor n in dyDesc. The tensor must be fully packed.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor yDesc.
CUdeviceptr	workspace	Data pointer to GPU memory to be used as a workspace for this call.
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workspace.
cudnnFilterDescriptor	dwDesc	Handle to a previously initialized filter descriptor describing the gradients of the weights for the RNN.
CUdeviceptr	dw	Data pointer to GPU memory associated with the filter descriptor dwDesc.
CUdeviceptr	reserveSpace	Data pointer to GPU memory to be used as a reserve space for this call.
SizeT	reserveSpaceSizeInBytes	Specifies the size in bytes of the provided reserveSpace.

Returns

Type	Description
cudnnStatus

cudnnRNNForwardInference(cudnnHandle, cudnnRNNDescriptor, cudnnTensorDescriptor[], CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, CUdeviceptr, cudnnTensorDescriptor[], CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, SizeT)

This routine executes the recurrent neural network described by rnnDesc with inputs x, hx, cx, weights w and outputs y, hy, cy. workspace is required for intermediate storage. This function does not store data required for training; cudnnRNNForwardTraining should be used for that purpose.

Declaration

public static cudnnStatus cudnnRNNForwardInference(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, cudnnTensorDescriptor[] xDesc, CUdeviceptr x, cudnnTensorDescriptor hxDesc, CUdeviceptr hx, cudnnTensorDescriptor cxDesc, CUdeviceptr cx, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnTensorDescriptor[] yDesc, CUdeviceptr y, cudnnTensorDescriptor hyDesc, CUdeviceptr hy, cudnnTensorDescriptor cyDesc, CUdeviceptr cy, CUdeviceptr workspace, SizeT workSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
cudnnTensorDescriptor[]	xDesc	An array of tensor descriptors describing the input to each recurrent iteration. Each tensor descriptor must have the same first dimension. The second dimension of the tensors may decrease from element n to element n+1 but may not increase. The tensor must be fully packed.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptors in the array xDesc. The data are expected to be packed contiguously with the first element of iteration n+1 following directly from the last element of iteration n.
cudnnTensorDescriptor	hxDesc	Handle to a previously initialized tensor descriptor describing the initial hidden state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	hx	Data pointer to GPU memory associated with the tensor descriptor hxDesc. If a NULL pointer is passed, the initial hidden state of the network will be initialized to zero.
cudnnTensorDescriptor	cxDesc	A fully packed tensor descriptor describing the initial cell state for LSTM networks.The first dimension of the tensor depends on the direction argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc: ‣ If direction is CUDNN_UNIDIRECTIONAL the first dimension should match the numLayers argument passed to cudnnSetRNNDescriptor. ‣ If direction is CUDNN_BIDIRECTIONAL the first dimension should match double the numLayers argument passed to cudnnSetRNNDescriptor. The second dimension must match the first dimension of the tensors described in xDesc.The third dimension must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	cx	Data pointer to GPU memory associated with the tensor descriptor cxDesc.If a NULL pointer is passed, the initial cell state of the network will be initialized to zero.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor describing the weights for the RNN.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor wDesc.
cudnnTensorDescriptor[]	yDesc	A fully packed tensor descriptor describing the final cell state for LSTM networks.The first dimension of the tensor depends on the direction argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc: ‣ If direction is CUDNN_UNIDIRECTIONAL the first dimension should match the numLayers argument passed to cudnnSetRNNDescriptor. ‣ If direction is CUDNN_BIDIRECTIONAL the first dimension should match double the numLayers argument passed to cudnnSetRNNDescriptor. The second dimension must match the first dimension of the tensors described in xDesc.The third dimension must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc.The tensor must be fully packed.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor yDesc. The data are expected to be packed contiguously with the first element of iteration n+1 following directly from the last element of iteration n.
cudnnTensorDescriptor	hyDesc	Handle to a previously initialized tensor descriptor describing the final hidden state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	hy	Data pointer to GPU memory associated with the tensor descriptor hyDesc. If a NULL pointer is passed, the final hidden state of the network will not be saved.
cudnnTensorDescriptor	cyDesc	Handle to a previously initialized tensor descriptor describing the final cell state for LSTM networks. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	cy	Data pointer to GPU memory associated with the tensor descriptor cyDesc. If a NULL pointer is passed, the final cell state of the network will be not be saved.
CUdeviceptr	workspace	Data pointer to GPU memory to be used as a workspace for this call.
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workspace.

Returns

Type	Description
cudnnStatus

cudnnRNNForwardTraining(cudnnHandle, cudnnRNNDescriptor, cudnnTensorDescriptor[], CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnFilterDescriptor, CUdeviceptr, cudnnTensorDescriptor[], CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, SizeT, CUdeviceptr, SizeT)

This routine executes the recurrent neural network described by rnnDesc with inputs x, hx, cx, weights w and outputs y, hy, cy. workspace is required for intermediate storage. reserveSpace stores data required for training. The same reserveSpace data must be used for future calls to cudnnRNNBackwardData and cudnnRNNBackwardWeights if these execute on the same input data.

Declaration

public static cudnnStatus cudnnRNNForwardTraining(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, cudnnTensorDescriptor[] xDesc, CUdeviceptr x, cudnnTensorDescriptor hxDesc, CUdeviceptr hx, cudnnTensorDescriptor cxDesc, CUdeviceptr cx, cudnnFilterDescriptor wDesc, CUdeviceptr w, cudnnTensorDescriptor[] yDesc, CUdeviceptr y, cudnnTensorDescriptor hyDesc, CUdeviceptr hy, cudnnTensorDescriptor cyDesc, CUdeviceptr cy, CUdeviceptr workspace, SizeT workSpaceSizeInBytes, CUdeviceptr reserveSpace, SizeT reserveSpaceSizeInBytes)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnRNNDescriptor	rnnDesc	A previously initialized RNN descriptor.
cudnnTensorDescriptor[]	xDesc	An array of tensor descriptors describing the input to each recurrent iteration. Each tensor descriptor must have the same first dimension. The second dimension of the tensors may decrease from element n to element n+1 but may not increase. The tensor must be fully packed.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptors in the array xDesc.
cudnnTensorDescriptor	hxDesc	Handle to a previously initialized tensor descriptor describing the initial hidden state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	hx	Data pointer to GPU memory associated with the tensor descriptor hxDesc. If a NULL pointer is passed, the initial hidden state of the network will be initialized to zero.
cudnnTensorDescriptor	cxDesc	Handle to a previously initialized tensor descriptor describing the initial cell state for LSTM networks. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	cx	Data pointer to GPU memory associated with the tensor descriptor cxDesc. If a NULL pointer is passed, the initial cell state of the network will be initialized to zero.
cudnnFilterDescriptor	wDesc	Handle to a previously initialized filter descriptor describing the weights for the RNN.
CUdeviceptr	w	Data pointer to GPU memory associated with the filter descriptor wDesc.
cudnnTensorDescriptor[]	yDesc	An array of tensor descriptors describing the output from each recurrent iteration. The first dimension of the tensor depends on the direction argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc: If direction is CUDNN_UNIDIRECTIONAL the first dimension should match the hiddenSize argument passed to cudnnSetRNNDescriptor. If direction is CUDNN_BIDIRECTIONAL the first dimension should match double the hiddenSize argument passed to cudnnSetRNNDescriptor. The second dimension of the tensor n must match the second dimension of the tensor n in xDesc. The tensor must be fully packed.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor yDesc.
cudnnTensorDescriptor	hyDesc	Handle to a previously initialized tensor descriptor describing the final hidden state of the RNN. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	hy	Data pointer to GPU memory associated with the tensor descriptor hyDesc. If a NULL pointer is passed, the final hidden state of the network will not be saved.
cudnnTensorDescriptor	cyDesc	Handle to a previously initialized tensor descriptor describing the final cell state for LSTM networks. The first dimension of the tensor must match the hiddenSize argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The second dimension must match the second dimension of the first tensor described in xDesc. The third dimension must match the numLayers argument passed to the cudnnSetRNNDescriptor call used to initialize rnnDesc. The tensor must be fully packed.
CUdeviceptr	cy	Data pointer to GPU memory associated with the tensor descriptor cyDesc. If a NULL pointer is passed, the final cell state of the network will be not be saved.
CUdeviceptr	workspace	Data pointer to GPU memory to be used as a workspace for this call.
SizeT	workSpaceSizeInBytes	Specifies the size in bytes of the provided workspace.
CUdeviceptr	reserveSpace	Data pointer to GPU memory to be used as a reserve space for this call.
SizeT	reserveSpaceSizeInBytes	Specifies the size in bytes of the provided reserveSpace.

Returns

Type	Description
cudnnStatus

cudnnScaleTensor(cudnnHandle, cudnnTensorDescriptor, CUdeviceptr, ref Double)

This function scale all the elements of a tensor by a give factor.

Declaration

public static cudnnStatus cudnnScaleTensor(cudnnHandle handle, cudnnTensorDescriptor yDesc, CUdeviceptr y, ref double alpha)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Pointer to data of the tensor described by the srcDestDesc descriptor.
System.Double	alpha	Pointer in Host memory to a value that all elements of the tensor will be scaled with.

Returns

Type	Description
cudnnStatus

cudnnScaleTensor(cudnnHandle, cudnnTensorDescriptor, CUdeviceptr, ref Single)

This function scale all the elements of a tensor by a give factor.

Declaration

public static cudnnStatus cudnnScaleTensor(cudnnHandle handle, cudnnTensorDescriptor yDesc, CUdeviceptr y, ref float alpha)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Pointer to data of the tensor described by the srcDestDesc descriptor.
System.Single	alpha	Pointer in Host memory to a value that all elements of the tensor will be scaled with.

Returns

Type	Description
cudnnStatus

cudnnSetActivationDescriptor(cudnnActivationDescriptor, cudnnActivationMode, cudnnNanPropagation, Double)

This function initializes then previously created activation descriptor object.

Declaration

public static cudnnStatus cudnnSetActivationDescriptor(cudnnActivationDescriptor activationDesc, cudnnActivationMode mode, cudnnNanPropagation reluNanOpt, double coef)

Parameters

Type	Name	Description
cudnnActivationDescriptor	activationDesc	Handle to the previously created activation descriptor object.
cudnnActivationMode	mode	Enumerant to specify the activation mode.
cudnnNanPropagation	reluNanOpt	Nan propagation option for the relu.
System.Double	coef	floating point number to specify the clipping threashold when the activation mode is set to CUDNN_ACTIVATION_CLIPPED_RELU or to specify the alpha coefficient when the activation mode is set to CUDNN_ACTIVATION_ELU.

Returns

Type	Description
cudnnStatus

cudnnSetConvolution2dDescriptor(cudnnConvolutionDescriptor, Int32, Int32, Int32, Int32, Int32, Int32, cudnnConvolutionMode, cudnnDataType)

This function initializes a previously created convolution descriptor object into a 2D correlation. This function assumes that the tensor and filter descriptors corresponds to the forward convolution path and checks if their settings are valid. That same convolution descriptor can be reused in the backward path provided it corresponds to the same layer.

Declaration

public static cudnnStatus cudnnSetConvolution2dDescriptor(cudnnConvolutionDescriptor convDesc, int pad_h, int pad_w, int u, int v, int dilation_h, int dilation_w, cudnnConvolutionMode mode, cudnnDataType dataType)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc	Handle to a previously created convolution descriptor.
System.Int32	pad_h	zero-padding height: number of rows of zeros implicitly concatenated onto the top and onto the bottom of input images.
System.Int32	pad_w	zero-padding width: number of columns of zeros implicitly concatenated onto the left and onto the right of input images.
System.Int32	u	Vertical filter stride.
System.Int32	v	Horizontal filter stride.
System.Int32	dilation_h	Filter height dilation.
System.Int32	dilation_w	Filter width dilation.
cudnnConvolutionMode	mode	Selects between CUDNN_CONVOLUTION and CUDNN_CROSS_CORRELATION.
cudnnDataType	dataType	Selects the datatype in which the computation will be done.

Returns

Type	Description
cudnnStatus

cudnnSetConvolutionGroupCount(cudnnConvolutionDescriptor, Int32)

Declaration

public static cudnnStatus cudnnSetConvolutionGroupCount(cudnnConvolutionDescriptor convDesc, int groupCount)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc
System.Int32	groupCount

Returns

Type	Description
cudnnStatus

cudnnSetConvolutionMathType(cudnnConvolutionDescriptor, cudnnMathType)

Declaration

public static cudnnStatus cudnnSetConvolutionMathType(cudnnConvolutionDescriptor convDesc, cudnnMathType mathType)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc
cudnnMathType	mathType

Returns

Type	Description
cudnnStatus

cudnnSetConvolutionNdDescriptor(cudnnConvolutionDescriptor, Int32, Int32[], Int32[], Int32[], cudnnConvolutionMode, cudnnDataType)

This function initializes a previously created generic convolution descriptor object into a n-D correlation. That same convolution descriptor can be reused in the backward path provided it corresponds to the same layer. The convolution computation will done in the specified dataType, which can be potentially different from the input/output tensors.

Declaration

public static cudnnStatus cudnnSetConvolutionNdDescriptor(cudnnConvolutionDescriptor convDesc, int arrayLength, int[] padA, int[] filterStrideA, int[] dilationA, cudnnConvolutionMode mode, cudnnDataType computeType)

Parameters

Type	Name	Description
cudnnConvolutionDescriptor	convDesc	Handle to a previously created convolution descriptor.
System.Int32	arrayLength	Dimension of the convolution.
System.Int32[]	padA	Array of dimension arrayLength containing the zero-padding size for each dimension. For every dimension, the padding represents the number of extra zeros implicitly concatenated at the start and at the end of every element of that dimension.
System.Int32[]	filterStrideA	Array of dimension arrayLength containing the filter stride for each dimension. For every dimension, the fitler stride represents the number of elements to slide to reach the next start of the filtering window of the next point.
System.Int32[]	dilationA	Array of dimension arrayLength containing the dilation factor for each dimension.
cudnnConvolutionMode	mode	Selects between CUDNN_CONVOLUTION and CUDNN_CROSS_CORRELATION.
cudnnDataType	computeType	Selects the datatype in which the computation will be done.

Returns

Type	Description
cudnnStatus

cudnnSetCTCLossDescriptor(cudnnCTCLossDescriptor, cudnnDataType)

Declaration

public static cudnnStatus cudnnSetCTCLossDescriptor(cudnnCTCLossDescriptor ctcLossDesc, cudnnDataType compType)

Parameters

Type	Name	Description
cudnnCTCLossDescriptor	ctcLossDesc
cudnnDataType	compType

Returns

Type	Description
cudnnStatus

cudnnSetDropoutDescriptor(cudnnDropoutDescriptor, cudnnHandle, Single, CUdeviceptr, SizeT, UInt64)

This function initializes a previously created dropout descriptor object. If states argument is equal to NULL, random number generator states won't be initialized, and only dropout value will be set. No other function should be writing to the memory

Declaration

public static cudnnStatus cudnnSetDropoutDescriptor(cudnnDropoutDescriptor dropoutDesc, cudnnHandle handle, float dropout, CUdeviceptr states, SizeT stateSizeInBytes, ulong seed)

Parameters

Type	Name	Description
cudnnDropoutDescriptor	dropoutDesc	Previously created dropout descriptor object.
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Single	dropout	The probability with which the value from input would be propagated through the dropout layer.
CUdeviceptr	states	Pointer to user-allocated GPU memory that will hold random number generator states.
SizeT	stateSizeInBytes	Specifies size in bytes of the provided memory for the states.
System.UInt64	seed	Seed used to initialize random number generator states.

Returns

Type	Description
cudnnStatus

cudnnSetFilter4dDescriptor(cudnnFilterDescriptor, cudnnDataType, cudnnTensorFormat, Int32, Int32, Int32, Int32)

This function initializes a previously created filter descriptor object into a 4D filter. Filters layout must be contiguous in memory. v4 version of the function also has the format parameter.

Declaration

public static cudnnStatus cudnnSetFilter4dDescriptor(cudnnFilterDescriptor filterDesc, cudnnDataType dataType, cudnnTensorFormat format, int k, int c, int h, int w)

Parameters

Type	Name	Description
cudnnFilterDescriptor	filterDesc	Handle to a previously created filter descriptor.
cudnnDataType	dataType	Data type.
cudnnTensorFormat	format	Layout format.
System.Int32	k	Number of output feature maps.
System.Int32	c	Number of input feature maps.
System.Int32	h	Height of each filter.
System.Int32	w	Width of each filter.

Returns

Type	Description
cudnnStatus

cudnnSetFilterNdDescriptor(cudnnFilterDescriptor, cudnnDataType, cudnnTensorFormat, Int32, Int32[])

This function initializes a previously created filter descriptor object. Filters layout must be contiguous in memory. v4 version of the function also has the format parameter.

Declaration

public static cudnnStatus cudnnSetFilterNdDescriptor(cudnnFilterDescriptor filterDesc, cudnnDataType dataType, cudnnTensorFormat format, int nbDims, int[] filterDimA)

Parameters

Type	Name	Description
cudnnFilterDescriptor	filterDesc	Handle to a previously created filter descriptor.
cudnnDataType	dataType	Data type.
cudnnTensorFormat	format	Layout format.
System.Int32	nbDims	Dimension of the filter.
System.Int32[]	filterDimA	Array of dimension nbDims containing the size of the filter for each dimension.

Returns

Type	Description
cudnnStatus

cudnnSetLRNDescriptor(cudnnLRNDescriptor, UInt32, Double, Double, Double)

This function initializes a previously created LRN descriptor object.

Declaration

public static cudnnStatus cudnnSetLRNDescriptor(cudnnLRNDescriptor normDesc, uint lrnN, double lrnAlpha, double lrnBeta, double lrnK)

Parameters

Type	Name	Description
cudnnLRNDescriptor	normDesc	Handle to a previously created LRN descriptor.
System.UInt32	lrnN	Normalization window width in elements. LRN layer uses a window [center-lookBehind, center+lookAhead], where lookBehind = floor( (lrnN-1)/2 ), lookAhead = lrnN-lookBehind-1. So for n=10, the window is [k-4...k...k+5] with a total of 10 samples. For DivisiveNormalization layer the window has the same extents as above in all 'spatial' dimensions (dimA[2], dimA[3], dimA[4]). By default lrnN is set to 5 in cudnnCreateLRNDescriptor.
System.Double	lrnAlpha	Value of the alpha variance scaling parameter in the normalization formula. Inside the library code this value is divided by the window width for LRN and by (window width)^#spatialDimensions for DivisiveNormalization. By default this value is set to 1e-4 in cudnnCreateLRNDescriptor.
System.Double	lrnBeta	Value of the beta power parameter in the normalization formula. By default this value is set to 0.75 in cudnnCreateLRNDescriptor.
System.Double	lrnK	Value of the k parameter in normalization formula. By default this value is set to 2.0.

Returns

Type	Description
cudnnStatus

cudnnSetOpTensorDescriptor(cudnnOpTensorDescriptor, cudnnOpTensorOp, cudnnDataType, cudnnNanPropagation)

Declaration

public static cudnnStatus cudnnSetOpTensorDescriptor(cudnnOpTensorDescriptor opTensorDesc, cudnnOpTensorOp opTensorOp, cudnnDataType opTensorCompType, cudnnNanPropagation opTensorNanOpt)

Parameters

Type	Name	Description
cudnnOpTensorDescriptor	opTensorDesc
cudnnOpTensorOp	opTensorOp
cudnnDataType	opTensorCompType
cudnnNanPropagation	opTensorNanOpt

Returns

Type	Description
cudnnStatus

cudnnSetPersistentRNNPlan(cudnnRNNDescriptor, cudnnPersistentRNNPlan)

This function sets the persistent RNN plan to be executed when using rnnDesc and CUDNN_RNN_ALGO_PERSIST_DYNAMIC algo.

Declaration

public static cudnnStatus cudnnSetPersistentRNNPlan(cudnnRNNDescriptor rnnDesc, cudnnPersistentRNNPlan plan)

Parameters

Type	Name	Description
cudnnRNNDescriptor	rnnDesc
cudnnPersistentRNNPlan	plan

Returns

Type	Description
cudnnStatus

cudnnSetPooling2dDescriptor(cudnnPoolingDescriptor, cudnnPoolingMode, cudnnNanPropagation, Int32, Int32, Int32, Int32, Int32, Int32)

This function initializes a previously created generic pooling descriptor object into a 2D description.

Declaration

public static cudnnStatus cudnnSetPooling2dDescriptor(cudnnPoolingDescriptor poolingDesc, cudnnPoolingMode mode, cudnnNanPropagation maxpoolingNanOpt, int windowHeight, int windowWidth, int verticalPadding, int horizontalPadding, int verticalStride, int horizontalStride)

Parameters

Type	Name	Description
cudnnPoolingDescriptor	poolingDesc	Handle to a previously created pooling descriptor.
cudnnPoolingMode	mode	Enumerant to specify the pooling mode.
cudnnNanPropagation	maxpoolingNanOpt	Nan propagation option for max pooling.
System.Int32	windowHeight	Height of the pooling window.
System.Int32	windowWidth	Width of the pooling window.
System.Int32	verticalPadding	Size of vertical padding.
System.Int32	horizontalPadding	Size of horizontal padding
System.Int32	verticalStride	Pooling vertical stride.
System.Int32	horizontalStride	Pooling horizontal stride.

Returns

Type	Description
cudnnStatus

cudnnSetPoolingNdDescriptor(cudnnPoolingDescriptor, cudnnPoolingMode, cudnnNanPropagation, Int32, Int32[], Int32[], Int32[])

This function initializes a previously created generic pooling descriptor object.

Declaration

public static cudnnStatus cudnnSetPoolingNdDescriptor(cudnnPoolingDescriptor poolingDesc, cudnnPoolingMode mode, cudnnNanPropagation maxpoolingNanOpt, int nbDims, int[] windowDimA, int[] paddingA, int[] strideA)

Parameters

Type	Name	Description
cudnnPoolingDescriptor	poolingDesc	Handle to a previously created pooling descriptor.
cudnnPoolingMode	mode	Enumerant to specify the pooling mode.
cudnnNanPropagation	maxpoolingNanOpt	Nan propagation option for max pooling.
System.Int32	nbDims	Dimension of the pooling operation.
System.Int32[]	windowDimA	Array of dimension nbDims containing the window size for each dimension.
System.Int32[]	paddingA	Array of dimension nbDims containing the padding size for each dimension.
System.Int32[]	strideA	Array of dimension nbDims containing the striding size for each dimension.

Returns

Type	Description
cudnnStatus

cudnnSetReduceTensorDescriptor(cudnnReduceTensorDescriptor, cudnnReduceTensorOp, cudnnDataType, cudnnNanPropagation, cudnnReduceTensorIndices, cudnnIndicesType)

Declaration

public static cudnnStatus cudnnSetReduceTensorDescriptor(cudnnReduceTensorDescriptor reduceTensorDesc, cudnnReduceTensorOp reduceTensorOp, cudnnDataType reduceTensorCompType, cudnnNanPropagation reduceTensorNanOpt, cudnnReduceTensorIndices reduceTensorIndices, cudnnIndicesType reduceTensorIndicesType)

Parameters

Type	Name	Description
cudnnReduceTensorDescriptor	reduceTensorDesc
cudnnReduceTensorOp	reduceTensorOp
cudnnDataType	reduceTensorCompType
cudnnNanPropagation	reduceTensorNanOpt
cudnnReduceTensorIndices	reduceTensorIndices
cudnnIndicesType	reduceTensorIndicesType

Returns

Type	Description
cudnnStatus

cudnnSetRNNDescriptor(cudnnHandle, cudnnRNNDescriptor, Int32, Int32, cudnnDropoutDescriptor, cudnnRNNInputMode, cudnnDirectionMode, cudnnRNNMode, cudnnRNNAlgo, cudnnDataType)

This function initializes a previously created RNN descriptor object.

Declaration

public static cudnnStatus cudnnSetRNNDescriptor(cudnnHandle handle, cudnnRNNDescriptor rnnDesc, int hiddenSize, int numLayers, cudnnDropoutDescriptor dropoutDesc, cudnnRNNInputMode inputMode, cudnnDirectionMode direction, cudnnRNNMode mode, cudnnRNNAlgo algo, cudnnDataType dataType)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN library descriptor.
cudnnRNNDescriptor	rnnDesc	A previously created RNN descriptor.
System.Int32	hiddenSize	Size of the internal hidden state for each layer.
System.Int32	numLayers	Number of stacked layers.
cudnnDropoutDescriptor	dropoutDesc	Handle to a previously created and initialized dropout descriptor. Dropout will be applied between layers(eg.a single layer network will have no dropout applied).
cudnnRNNInputMode	inputMode	Specifies the behavior at the input to the first layer
cudnnDirectionMode	direction	Specifies the recurrence pattern. (eg. bidirectional)
cudnnRNNMode	mode	Specifies the type of RNN to compute.
cudnnRNNAlgo	algo	Specifies which RNN algorithm should be used to compute the results.
cudnnDataType	dataType	Compute precision.

Returns

Type	Description
cudnnStatus

cudnnSetRNNMatrixMathType(cudnnRNNDescriptor, cudnnMathType)

Declaration

public static cudnnStatus cudnnSetRNNMatrixMathType(cudnnRNNDescriptor desc, cudnnMathType math)

Parameters

Type	Name	Description
cudnnRNNDescriptor	desc
cudnnMathType	math

Returns

Type	Description
cudnnStatus

cudnnSetSpatialTransformerNdDescriptor(cudnnSpatialTransformerDescriptor, cudnnSamplerType, cudnnDataType, Int32, Int32[])

This function destroys a previously created spatial transformer descriptor object.

Declaration

public static cudnnStatus cudnnSetSpatialTransformerNdDescriptor(cudnnSpatialTransformerDescriptor stDesc, cudnnSamplerType samplerType, cudnnDataType dataType, int nbDims, int[] dimA)

Parameters

Type	Name	Description
cudnnSpatialTransformerDescriptor	stDesc	Previously created spatial transformer descriptor object.
cudnnSamplerType	samplerType	Enumerant to specify the sampler type.
cudnnDataType	dataType	Data type.
System.Int32	nbDims	Dimension of the transformed tensor.
System.Int32[]	dimA	Array of dimension nbDims containing the size of the transformed tensor for every dimension.

Returns

Type	Description
cudnnStatus

cudnnSetStream(cudnnHandle, CUstream)

This function sets the cuDNN library stream, which will be used to execute all subsequent calls to the cuDNN library functions with that particular handle. If the cuDNN library stream is not set, all kernels use the default (NULL) stream. In particular, this routine can be used to change the stream between kernel launches and then to reset the cuDNN library stream back to NULL.

Declaration

public static cudnnStatus cudnnSetStream(cudnnHandle handle, CUstream streamId)

Parameters

Type	Name	Description
cudnnHandle	handle
CUstream	streamId

Returns

Type	Description
cudnnStatus

cudnnSetTensor(cudnnHandle, cudnnTensorDescriptor, CUdeviceptr, ref Double)

This function sets all the elements of a tensor to a given value

Declaration

public static cudnnStatus cudnnSetTensor(cudnnHandle handle, cudnnTensorDescriptor yDesc, CUdeviceptr y, ref double value)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Pointer to data of the tensor described by the srcDestDesc descriptor.
System.Double	value	Pointer in Host memory to a value that all elements of the tensor will be set to.

Returns

Type	Description
cudnnStatus

cudnnSetTensor(cudnnHandle, cudnnTensorDescriptor, CUdeviceptr, ref Single)

This function sets all the elements of a tensor to a given value

Declaration

public static cudnnStatus cudnnSetTensor(cudnnHandle handle, cudnnTensorDescriptor yDesc, CUdeviceptr y, ref float value)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Pointer to data of the tensor described by the srcDestDesc descriptor.
System.Single	value	Pointer in Host memory to a value that all elements of the tensor will be set to.

Returns

Type	Description
cudnnStatus

cudnnSetTensor4dDescriptor(cudnnTensorDescriptor, cudnnTensorFormat, cudnnDataType, Int32, Int32, Int32, Int32)

This function initializes a previously created generic Tensor descriptor object into a 4D tensor. The strides of the four dimensions are inferred from the format parameter and set in such a way that the data is contiguous in memory with no padding between dimensions.

Declaration

public static cudnnStatus cudnnSetTensor4dDescriptor(cudnnTensorDescriptor tensorDesc, cudnnTensorFormat format, cudnnDataType dataType, int n, int c, int h, int w)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc	Handle to a previously created tensor descriptor.
cudnnTensorFormat	format	Type of format.
cudnnDataType	dataType	Data type.
System.Int32	n	Number of images.
System.Int32	c	Number of feature maps per image.
System.Int32	h	Height of each feature map.
System.Int32	w	Width of each feature map.

Returns

Type	Description
cudnnStatus

cudnnSetTensor4dDescriptorEx(cudnnTensorDescriptor, cudnnDataType, Int32, Int32, Int32, Int32, Int32, Int32, Int32, Int32)

This function initializes a previously created generic Tensor descriptor object into a 4D tensor, similarly to cudnnSetTensor4dDescriptor but with the strides explicitly passed as parameters. This can be used to lay out the 4D tensor in any order or simply to define gaps between dimensions.

Declaration

public static cudnnStatus cudnnSetTensor4dDescriptorEx(cudnnTensorDescriptor tensorDesc, cudnnDataType dataType, int n, int c, int h, int w, int nStride, int cStride, int hStride, int wStride)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc	Handle to a previously created tensor descriptor.
cudnnDataType	dataType	Data type.
System.Int32	n	Number of images.
System.Int32	c	Number of feature maps per image.
System.Int32	h	Height of each feature map.
System.Int32	w	Width of each feature map.
System.Int32	nStride	Stride between two consecutive images.
System.Int32	cStride	Stride between two consecutive feature maps.
System.Int32	hStride	Stride between two consecutive rows.
System.Int32	wStride	Stride between two consecutive columns.

Returns

Type	Description
cudnnStatus

cudnnSetTensorNdDescriptor(cudnnTensorDescriptor, cudnnDataType, Int32, Int32[], Int32[])

This function initializes a previously created generic Tensor descriptor object.

Declaration

public static cudnnStatus cudnnSetTensorNdDescriptor(cudnnTensorDescriptor tensorDesc, cudnnDataType dataType, int nbDims, int[] dimA, int[] strideA)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc	Handle to a previously created tensor descriptor.
cudnnDataType	dataType	Data type.
System.Int32	nbDims	Dimension of the tensor.
System.Int32[]	dimA	Array of dimension nbDims that contain the size of the tensor for every dimension.
System.Int32[]	strideA	Array of dimension nbDims that contain the stride of the tensor for every dimension.

Returns

Type	Description
cudnnStatus

cudnnSetTensorNdDescriptorEx(cudnnTensorDescriptor, cudnnTensorFormat, cudnnDataType, Int32, Int32[])

This function initializes a previously created generic Tensor descriptor object.

Declaration

public static cudnnStatus cudnnSetTensorNdDescriptorEx(cudnnTensorDescriptor tensorDesc, cudnnTensorFormat format, cudnnDataType dataType, int nbDims, int[] dimA)

Parameters

Type	Name	Description
cudnnTensorDescriptor	tensorDesc	Handle to a previously created tensor descriptor.
cudnnTensorFormat	format
cudnnDataType	dataType	Data type.
System.Int32	nbDims	Dimension of the tensor.
System.Int32[]	dimA	Array of dimension nbDims that contain the size of the tensor for every dimension.

Returns

Type	Description
cudnnStatus

cudnnSoftmaxBackward(cudnnHandle, cudnnSoftmaxAlgorithm, cudnnSoftmaxMode, ref Double, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This routine computes the gradient of the softmax function.

Declaration

public static cudnnStatus cudnnSoftmaxBackward(cudnnHandle handle, cudnnSoftmaxAlgorithm algorithm, cudnnSoftmaxMode mode, ref double alpha, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, ref double beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnSoftmaxAlgorithm	algorithm	Enumerant to specify the softmax algorithm.
cudnnSoftmaxMode	mode	Enumerant to specify the softmax mode.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor srcDiffData.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output differential tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor destDiffDesc.

Returns

Type	Description
cudnnStatus

cudnnSoftmaxBackward(cudnnHandle, cudnnSoftmaxAlgorithm, cudnnSoftmaxMode, ref Single, cudnnTensorDescriptor, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This routine computes the gradient of the softmax function.

Declaration

public static cudnnStatus cudnnSoftmaxBackward(cudnnHandle handle, cudnnSoftmaxAlgorithm algorithm, cudnnSoftmaxMode mode, ref float alpha, cudnnTensorDescriptor yDesc, CUdeviceptr y, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, ref float beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnSoftmaxAlgorithm	algorithm	Enumerant to specify the softmax algorithm.
cudnnSoftmaxMode	mode	Enumerant to specify the softmax mode.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor srcDiffData.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output differential tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor destDiffDesc.

Returns

Type	Description
cudnnStatus

cudnnSoftmaxForward(cudnnHandle, cudnnSoftmaxAlgorithm, cudnnSoftmaxMode, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This routine computes the softmax function.

Declaration

public static cudnnStatus cudnnSoftmaxForward(cudnnHandle handle, cudnnSoftmaxAlgorithm algorithm, cudnnSoftmaxMode mode, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnSoftmaxAlgorithm	algorithm	Enumerant to specify the softmax algorithm.
cudnnSoftmaxMode	mode	Enumerant to specify the softmax mode.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor destDesc.

Returns

Type	Description
cudnnStatus

cudnnSoftmaxForward(cudnnHandle, cudnnSoftmaxAlgorithm, cudnnSoftmaxMode, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This routine computes the softmax function.

Declaration

public static cudnnStatus cudnnSoftmaxForward(cudnnHandle handle, cudnnSoftmaxAlgorithm algorithm, cudnnSoftmaxMode mode, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnSoftmaxAlgorithm	algorithm	Enumerant to specify the softmax algorithm.
cudnnSoftmaxMode	mode	Enumerant to specify the softmax mode.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor srcDesc.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows: dstValue = alpha[0]result + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor destDesc.

Returns

Type	Description
cudnnStatus

cudnnSpatialTfGridGeneratorBackward(cudnnHandle, cudnnSpatialTransformerDescriptor, CUdeviceptr, CUdeviceptr)

This function computes the gradient of a grid generation operation.

Declaration

public static cudnnStatus cudnnSpatialTfGridGeneratorBackward(cudnnHandle handle, cudnnSpatialTransformerDescriptor stDesc, CUdeviceptr dgrid, CUdeviceptr dtheta)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnSpatialTransformerDescriptor	stDesc	Previously created spatial transformer descriptor object.
CUdeviceptr	dgrid	Data pointer to GPU memory contains the input differential data.
CUdeviceptr	dtheta	Data pointer to GPU memory contains the output differential data.

Returns

Type	Description
cudnnStatus

cudnnSpatialTfGridGeneratorForward(cudnnHandle, cudnnSpatialTransformerDescriptor, CUdeviceptr, CUdeviceptr)

This function generates a grid of coordinates in the input tensor corresponding to each pixel from the output tensor.

Declaration

public static cudnnStatus cudnnSpatialTfGridGeneratorForward(cudnnHandle handle, cudnnSpatialTransformerDescriptor stDesc, CUdeviceptr theta, CUdeviceptr grid)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnSpatialTransformerDescriptor	stDesc	Previously created spatial transformer descriptor object.
CUdeviceptr	theta	Affine transformation matrix. It should be of size n23 for a 2d transformation, where n is the number of images specified in stDesc.
CUdeviceptr	grid	A grid of coordinates. It is of size nhw*2 for a 2d transformation, where n, h, w is specified in stDesc. In the 4th dimension, the first coordinate is x, and the second coordinate is y.

Returns

Type	Description
cudnnStatus

cudnnSpatialTfSamplerBackward(cudnnHandle, cudnnSpatialTransformerDescriptor, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr)

Declaration

public static cudnnStatus cudnnSpatialTfSamplerBackward(cudnnHandle handle, cudnnSpatialTransformerDescriptor stDesc, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx, CUdeviceptr alphaDgrid, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, CUdeviceptr grid, CUdeviceptr betaDgrid, CUdeviceptr dgrid)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnSpatialTransformerDescriptor	stDesc
System.Double	alpha
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
System.Double	beta
cudnnTensorDescriptor	dxDesc
CUdeviceptr	dx
CUdeviceptr	alphaDgrid
cudnnTensorDescriptor	dyDesc
CUdeviceptr	dy
CUdeviceptr	grid
CUdeviceptr	betaDgrid
CUdeviceptr	dgrid

Returns

Type	Description
cudnnStatus

cudnnSpatialTfSamplerBackward(cudnnHandle, cudnnSpatialTransformerDescriptor, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, CUdeviceptr, CUdeviceptr)

This function computes the gradient of a sampling operation.

Declaration

public static cudnnStatus cudnnSpatialTfSamplerBackward(cudnnHandle handle, cudnnSpatialTransformerDescriptor stDesc, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor dxDesc, CUdeviceptr dx, CUdeviceptr alphaDgrid, cudnnTensorDescriptor dyDesc, CUdeviceptr dy, CUdeviceptr grid, CUdeviceptr betaDgrid, CUdeviceptr dgrid)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnSpatialTransformerDescriptor	stDesc	Previously created spatial transformer descriptor object.
System.Single	alpha	Pointer to scaling factor (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor xDesc.
System.Single	beta	Pointer to scaling factor (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue.
cudnnTensorDescriptor	dxDesc	Handle to the previously initialized output differential tensor descriptor.
CUdeviceptr	dx	Data pointer to GPU memory associated with the output tensor descriptor dxDesc.
CUdeviceptr	alphaDgrid	Pointer to scaling factor (in host memory) used to blend the gradient outputs dgrid with prior value in the destination pointer as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue.
cudnnTensorDescriptor	dyDesc	Handle to the previously initialized input differential tensor descriptor.
CUdeviceptr	dy	Data pointer to GPU memory associated with the tensor descriptor dyDesc.
CUdeviceptr	grid	A grid of coordinates generated by cudnnSpatialTfGridGeneratorForward.
CUdeviceptr	betaDgrid	Pointer to scaling factor (in host memory) used to blend the gradient outputs dgrid with prior value in the destination pointer as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue.
CUdeviceptr	dgrid	Data pointer to GPU memory contains the output differential data.

Returns

Type	Description
cudnnStatus

cudnnSpatialTfSamplerForward(cudnnHandle, cudnnSpatialTransformerDescriptor, ref Double, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnSpatialTfSamplerForward(cudnnHandle handle, cudnnSpatialTransformerDescriptor stDesc, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, CUdeviceptr grid, ref double beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle
cudnnSpatialTransformerDescriptor	stDesc
System.Double	alpha
cudnnTensorDescriptor	xDesc
CUdeviceptr	x
CUdeviceptr	grid
System.Double	beta
cudnnTensorDescriptor	yDesc
CUdeviceptr	y

Returns

Type	Description
cudnnStatus

cudnnSpatialTfSamplerForward(cudnnHandle, cudnnSpatialTransformerDescriptor, ref Single, cudnnTensorDescriptor, CUdeviceptr, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

This function performs a sampler operation and generates the output tensor using the grid given by the grid generator.

Declaration

public static cudnnStatus cudnnSpatialTfSamplerForward(cudnnHandle handle, cudnnSpatialTransformerDescriptor stDesc, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, CUdeviceptr grid, ref float beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
cudnnSpatialTransformerDescriptor	stDesc	Previously created spatial transformer descriptor object.
System.Single	alpha	Pointer to scaling factor (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue.
cudnnTensorDescriptor	xDesc	Handle to the previously initialized input tensor descriptor.
CUdeviceptr	x	Data pointer to GPU memory associated with the tensor descriptor xDesc.
CUdeviceptr	grid	A grid of coordinates generated by cudnnSpatialTfGridGeneratorForward.
System.Single	beta	Pointer to scaling factor (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue.
cudnnTensorDescriptor	yDesc	Handle to the previously initialized output tensor descriptor.
CUdeviceptr	y	Data pointer to GPU memory associated with the output tensor descriptor yDesc.

Returns

Type	Description
cudnnStatus

cudnnTransformTensor(cudnnHandle, ref Double, cudnnTensorDescriptor, CUdeviceptr, ref Double, cudnnTensorDescriptor, CUdeviceptr)

This function copies the scaled data from one tensor to another tensor with a different layout. Those descriptors need to have the same dimensions but not necessarily the same strides. The input and output tensors must not overlap in any way (i.e., tensors cannot be transformed in place). This function can be used to convert a tensor with an unsupported format to a supported one.

Declaration

public static cudnnStatus cudnnTransformTensor(cudnnHandle handle, ref double alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref double beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Double	alpha	Pointer to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Pointer to data of the tensor described by the srcDesc descriptor.
System.Double	beta	Pointer to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Pointer to data of the tensor described by the destDesc descriptor.

Returns

Type	Description
cudnnStatus

cudnnTransformTensor(cudnnHandle, ref Single, cudnnTensorDescriptor, CUdeviceptr, ref Single, cudnnTensorDescriptor, CUdeviceptr)

Declaration

public static cudnnStatus cudnnTransformTensor(cudnnHandle handle, ref float alpha, cudnnTensorDescriptor xDesc, CUdeviceptr x, ref float beta, cudnnTensorDescriptor yDesc, CUdeviceptr y)

Parameters

Type	Name	Description
cudnnHandle	handle	Handle to a previously created cuDNN context.
System.Single	alpha	Pointer to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	xDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	x	Pointer to data of the tensor described by the srcDesc descriptor.
System.Single	beta	Pointer to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows: dstValue = alpha[0]srcValue + beta[0]priorDstValue. Please refer to this section for additional details.
cudnnTensorDescriptor	yDesc	Handle to a previously initialized tensor descriptor.
CUdeviceptr	y	Pointer to data of the tensor described by the destDesc descriptor.

Returns

Type	Description
cudnnStatus