GPUMLib  0.2.2
GPU Machine Learning Library
Functions
Back-Propagation and Multiple Back-Propagation kernels

Functions

KERNEL FireLayer (cudafloat *inputs, cudafloat *weights, cudafloat *m, int mOffset, int totalNeuronsWithSelectiveActivation, cudafloat *outputs)
 
void KernelFireLayer (cudaStream_t stream, dim3 &gridSize, int blockSize, cudafloat *inputs, cudafloat *weights, cudafloat *m, int mOffset, int totalNeuronsWithSelectiveActivation, cudafloat *outputs, int numInputs)
 
KERNEL FireOutputLayer (cudafloat *inputs, cudafloat *weights, cudafloat *m, int mOffset, int totalNeuronsWithSelectiveActivation, cudafloat *desiredOutputs, cudafloat *outputs, cudafloat *localGradient, cudafloat *rms, cudafloat *localGradientSpaceNet)
 
void KernelFireOutputLayer (cudaStream_t stream, dim3 &gridSize, int blockSize, cudafloat *inputs, cudafloat *weights, cudafloat *m, int mOffset, int totalNeuronsWithSelectiveActivation, cudafloat *desiredOutputs, cudafloat *outputs, cudafloat *localGradient, cudafloat *rms, cudafloat *localGradientSpaceNet, int numInputs)
 
KERNEL CalculateLocalGradient (cudafloat *rmsF, cudafloat *bestRMS, cudafloat maxErrorGrowth, cudafloat *outputs, cudafloat *weights, cudafloat *m, int mOffset, int totalNeuronsWithSelectiveActivation, cudafloat *localGradientNextLayer, cudafloat *localGradient, cudafloat *localGradientSpaceNet)
 
void KernelCorrectLayerWeights (cudaStream_t stream, dim3 &gridSize, int blockSize, cudafloat *rmsF, cudafloat *bestRMS, cudafloat maxErrorGrowth, cudafloat *inputs, cudafloat *localGradient, cudafloat *weights, cudafloat *learningRate, cudafloat *lastDeltaWithoutLearningMomentum, cudafloat *lastDelta, cudafloat maxStepSize, cudafloat u, cudafloat d, cudafloat r, cudafloat momentum, int numberPatterns)
 
void KernelCalculateRMS (cudaStream_t stream, int blockSize, cudafloat *rms, cudafloat *rmsOut, int numberPatterns, cudafloat numberPatternsNeurons)
 
KERNEL RobustLearning (cudafloat *rmsF, cudafloat *bestRMS, cudafloat maxErrorGrowth, int layers, int *numberWeights, cudafloat **weights, cudafloat **bestWeights, cudafloat **learningRate, cudafloat r, cudafloat **lastDeltaWithoutLearningMomentum, cudafloat **lastDelta)
 
KERNEL FireSelectiveInputs (cudafloat *inputs, cudafloat *weights, cudafloat *bias, cudafloat *outputs, int numNeurons)
 
KERNEL CalcLocalGradSelectiveInputs (cudafloat *rmsF, cudafloat *bestRMS, cudafloat maxErrorGrowth, cudafloat *inputs, cudafloat *selectiveNeuronsWeights, cudafloat *selectiveNeuronsBias, cudafloat *weights, cudafloat *localGradientNextLayer, cudafloat *localGradient)
 
void KernelCorrectWeightsSelectiveInputs (cudaStream_t stream, int neurons, int patterns, cudafloat *rmsF, cudafloat *bestRMS, cudafloat maxErrorGrowth, cudafloat *inputs, cudafloat *localGradient, cudafloat *selectiveNeuronsWeights, cudafloat *selectiveNeuronsBias, cudafloat *learningRateWeights, cudafloat *learningRateBias, cudafloat *lastDeltaWithoutLearningMomentumWeights, cudafloat *lastDeltaWithoutLearningMomentumBias, cudafloat *lastDeltaWeights, cudafloat *lastDeltaBias, cudafloat u, cudafloat d, cudafloat r, cudafloat maxStepSize, cudafloat momentum, int numberPatterns)
 

Detailed Description

Function Documentation

KERNEL CalcLocalGradSelectiveInputs ( cudafloat rmsF,
cudafloat bestRMS,
cudafloat  maxErrorGrowth,
cudafloat inputs,
cudafloat selectiveNeuronsWeights,
cudafloat selectiveNeuronsBias,
cudafloat weights,
cudafloat localGradientNextLayer,
cudafloat localGradient 
)

Kernel to calculate the local gradient of the selective input neurons.

Parameters
[in]rmsFaddress of the variable containg the current root mean square error. Should be nullptr if robust training is not used.
[in]bestRMSaddress of the variable containg the best root mean square error found so far. Should be nullptr if robust training is not used.
[in]maxErrorGrowthHow much the root mean square error is allowed to grow relatively to bestRMS before applying the robusteness techique (see the RobustLearning kernel).
[in]inputsInput data for all patterns (numInputs * numPatterns). Position [p * numInputs + i] should contain the input i for pattern p.
[in]selectiveNeuronsWeightsInput weights of selective input neurons (one per neuron).
[in]selectiveNeuronsBiasBias of selective input neurons.
[in]weightsInput weights of the next layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the weight of the connection between input (neuron) i and neuron n.
[in]localGradientNextLayerLocal gradients of the next layer (numOutputs * numPatterns). Position [p * numOutputs + o] will contain the local gradients of neuron o for pattern p.
[out]localGradientLocal gradients of the layer (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the local gradients of neuron n for pattern p.
See also
RobustLearning

Definition at line 36 of file CalcLocalGradSelectiveInputs.cu.

KERNEL CalculateLocalGradient ( cudafloat rmsF,
cudafloat bestRMS,
cudafloat  maxErrorGrowth,
cudafloat outputs,
cudafloat weights,
cudafloat m,
int  mOffset,
int  totalNeuronsWithSelectiveActivation,
cudafloat localGradientNextLayer,
cudafloat localGradient,
cudafloat localGradientSpaceNet 
)

Kernel to calculate the local gradient of all neurons in a hidden layer. For selective activation neurons, the local gradients of the corresponding space network neurons are also calculated.

Attention
used only for hidden layers. The local gradients of output layers are automatically calculated by FireOutputLayer or by KernelFireOutputLayer.
Parameters
[in]rmsFaddress of the variable containg the current root mean square error. Should be nullptr if robust training is not used.
[in]bestRMSaddress of the variable containg the best root mean square error found so far. Should be nullptr if robust training is not used.
[in]maxErrorGrowthHow much the root mean square error is allowed to grow relatively to bestRMS before applying the robusteness techique (see the RobustLearning kernel).
[in]outputsNetwork outputs (numNeurons * numPatterns). Position [p * numNeurons + n] contains the output of neuron n for pattern p.
[in]weightsInput weights of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the weight of the connection between input (neuron) i and neuron n.
[in]mImportance of the neurons of the layer (numNeurons * numPatterns). Should be nullptr for neurons without selective activation. Position [p * numNeurons + n] should contain the importance of neuron n for pattern p.
[in]mOffsetOffset of the importance of this layers neurons whithin the importance values. Tipically identical to the number of neurons in the previous layers.
[in]totalNeuronsWithSelectiveActivationNumber of neurons with selective activation in all layers (total).
[in]localGradientNextLayerLocal gradients of the next layer (numOutputs * numPatterns). Position [p * numOutputs + o] will contain the local gradients of neuron o for pattern p.
[out]localGradientLocal gradients of the layer (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the local gradients of neuron n for pattern p.
[out]localGradientSpaceNetLocal gradients of the associated neurons in space network output layer (numNeurons * numPatterns). Should be nullptr for neurons without selective activation. Position [p * numNeurons + n] will contain the local gradients, for pattern p, of the neuron that calculates the importance of neuron n.
Examples:
dim3 dimOutputsNeurons(numOutputs, numNeurons);
CalculateLocalGradient<<<patterns, dimOutputsNeurons, (outputs * (neurons + 1)) * sizeof(cudafloat)>>>(rms, bestRMS, maxErrorGrowthToApplyRobustLearning, outputs, nextLayerWeights, m, nextLayerlocalGradient, localGradient, lgSpaceNet);
dim3 dimOutputsNeurons(numOutputs, numNeurons);
CalculateLocalGradient<<<patterns, dimOutputsNeurons, (outputs * (neurons + 1)) * sizeof(cudafloat)>>>(rms, bestRMS, maxErrorGrowthToApplyRobustLearning, outputs, nextLayerWeights, nullptr, nextLayerlocalGradient, localGradient, nullptr);
See also
FireOutputLayer
KernelFireOutputLayer
RobustLearning

Definition at line 36 of file LocalGradientKernel.cu.

KERNEL FireLayer ( cudafloat inputs,
cudafloat weights,
cudafloat m,
int  mOffset,
int  totalNeuronsWithSelectiveActivation,
cudafloat outputs 
)

Kernel to calculate the outputs of all neurons in a given layer. For the output layer use FireOutputLayer or KernelFireOutputLayer instead.

Attention
used only if the number of connections (weights) is NOT greather than MAX_THREADS_PER_BLOCK.
Parameters
[in]inputsInput data for all patterns (numInputs * numPatterns). Position [p * numInputs + i] should contain the input i for pattern p.
[in]weightsInput weights of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the weight of the connection between input (neuron) i and neuron n.
[in]mImportance of the neurons of the layer (numNeurons * numPatterns). Should be nullptr for neurons without selective activation. Position [p * numNeurons + n] should contain the importance of neuron n for pattern p.
[in]mOffsetOffset of the importance of this layers neurons whithin the importance values. Tipically identical to the number of neurons in the previous layers.
[in]totalNeuronsWithSelectiveActivationNumber of neurons with selective activation in all layers (total).
[out]outputsNeuron outputs of the layer (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the output of neuron n for pattern p.
Examples:
FireLayer<<<numPatterns, (numInputs + 1, numNeurons), numWeights * sizeof(cudafloat), stream>>>(inputs, weights, m, outputs);
FireLayer<<<numPatterns, (numInputs + 1, numNeurons), numWeights * sizeof(cudafloat), stream>>>(inputs, weights, nullptr, outputs);
See also
KernelFireLayer
FireOutputLayer
KernelFireOutputLayer
MAX_THREADS_PER_BLOCK

Definition at line 65 of file FireLayerKernel.cu.

KERNEL FireOutputLayer ( cudafloat inputs,
cudafloat weights,
cudafloat m,
int  mOffset,
int  totalNeuronsWithSelectiveActivation,
cudafloat desiredOutputs,
cudafloat outputs,
cudafloat localGradient,
cudafloat rms,
cudafloat localGradientSpaceNet 
)

Kernel to calculate the outputs of the network output layer and the local gradients of its neurons. If the layer contains selective activation neurons, the local gradients of the corresponding space network neurons are also calculated. This kernel also calculates part of the RMS.

Attention
used only if the number of connections (weights) is NOT greather than MAX_THREADS_PER_BLOCK.
Parameters
[in]inputsInput data for all patterns (numInputs * numPatterns). Position [p * numInputs + i] should contain the input i for pattern p.
[in]weightsInput weights of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the weight of the connection between input (neuron) i and neuron n.
[in]mImportance of the neurons of the layer (numNeurons * numPatterns). Should be nullptr for neurons without selective activation. Position [p * numNeurons + n] should contain the importance of neuron n for pattern p.
[in]mOffsetOffset of the importance of this layers neurons whithin the importance values. Tipically identical to the number of neurons in the previous layers.
[in]totalNeuronsWithSelectiveActivationNumber of neurons with selective activation in all layers (total).
[in]desiredOutputsThe network desired outputs for all patterns (numNeurons * numPatterns). Position [p * numNeurons + n] should contain the desired output of neuron n for pattern p.
[out]outputsNetwork outputs (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the output of neuron n for pattern p.
[out]localGradientLocal gradients of the network output layer (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the local gradients of neuron n for pattern p.
[out]rmspartial calculation of the root mean square error (numPatterns).
[out]localGradientSpaceNetLocal gradients of the associated neurons in space network output layer (numNeurons * numPatterns). Should be nullptr for neurons without selective activation. Position [p * numNeurons + n] will contain the local gradients, for pattern p, of the neuron that calculates the importance of neuron n.
Examples:
FireOutputLayer<<<NumPatterns, (numInputs + 1, numNeurons), (numWeights + numNeurons) * sizeof(cudafloat), stream>>>(inputs, weights, m, desiredOutputs, outputs, localGradient, rms, localGradientSpaceNet);
FireOutputLayer<<<NumPatterns, (numInputs + 1, numNeurons), (numWeights + numNeurons) * sizeof(cudafloat), stream>>>(inputs, weights, nullptr, desiredOutputs, outputs, localGradient, rms, nullptr);
See also
KernelFireOutputLayer
MAX_THREADS_PER_BLOCK

Definition at line 85 of file FireLayerKernel.cu.

KERNEL FireSelectiveInputs ( cudafloat inputs,
cudafloat weights,
cudafloat bias,
cudafloat outputs,
int  numNeurons 
)

Kernel to calculate the outputs of the selective input neurons.

Parameters
[in]inputsInput data for all patterns (numInputs * numPatterns). Position [p * numInputs + i] should contain the input i for pattern p.
[in]weightsInput weights of selective input neurons (one per neuron).
[in]biasBias of selective input neurons.
[out]outputsNeuron outputs of the layer (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the output of neuron n for pattern p.
[in]numNeuronsNumber of neurons (identical to the number of inputs of the network).

Definition at line 27 of file SelectiveInputs.cu.

void KernelCalculateRMS ( cudaStream_t  stream,
int  blockSize,
cudafloat rms,
cudafloat rmsOut,
int  numberPatterns,
cudafloat  numberPatternsNeurons 
)

Kernel to calculate the Root Mean Square (RMS) error of the neural network.

Parameters
[in]streamCUDA stream.
[in]blockSizeBlock size. Must be a multiple of 2 and not exceed MAX_THREADS_PER_BLOCK.
[in]rmsThe partial calculations of the root mean square error (see FireOutputLayer and KernelFireOutputLayer).
[out]rmsOutAddress of the variable that will contain the root mean square error.
[in]numberPatternsNumber of patterns.
[in]numberPatternsNeuronsProduct between the number of patterns and the number of neurons.
Examples:
KernelCalculateRMS(stream, patternsBlockSize, rms, rmsOut, numberPatterns, (cudafloat) numberPatterns * numberNeurons);
See also
KernelFireOutputLayer
FireOutputLayer

Definition at line 46 of file CalculateRMS.cu.

void KernelCorrectLayerWeights ( cudaStream_t  stream,
dim3 &  gridSize,
int  blockSize,
cudafloat rmsF,
cudafloat bestRMS,
cudafloat  maxErrorGrowth,
cudafloat inputs,
cudafloat localGradient,
cudafloat weights,
cudafloat learningRate,
cudafloat lastDeltaWithoutLearningMomentum,
cudafloat lastDelta,
cudafloat  maxStepSize,
cudafloat  u,
cudafloat  d,
cudafloat  r,
cudafloat  momentum,
int  numberPatterns 
)

Kernel to adjust the weights of a given layer. The step sizes are also updated.

Parameters
[in]streamCUDA stream.
[in]gridSizeSize of the grid (numInputs + 1, numNeurons).
[in]blockSizeBlock size. Must be a multiple of 2 and not exceed MAX_THREADS_PER_BLOCK.
[in]rmsFaddress of the variable containg the current root mean square error. Should be nullptr if robust training is not used.
[in]bestRMSaddress of the variable containg the best root mean square error found so far. Should be nullptr if robust training is not used.
[in]maxErrorGrowthHow much the root mean square error is allowed to grow relatively to bestRMS before applying the robusteness techique (see the RobustLearning kernel).
[in]inputsInput data for all patterns (numInputs * numPatterns). Position [p * numInputs + i] should contain the input i for pattern p.
[in]localGradientLocal gradients of the layer (numNeurons * numPatterns). Position [p * numNeurons + n] should contain the local gradients of neuron n for pattern p.
[in,out]weightsInput weights of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the weight of the connection between input (neuron) i and neuron n.
[in,out]learningRateLearning rate (step size) of each input connections of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the step size of the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the step size of the connection between input (neuron) i and neuron n.
[in,out]lastDeltaWithoutLearningMomentumLast delta without learning or momentum of each input connections of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the last delta associated with the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the last delta associated with the connection between input (neuron) i and neuron n.
[in,out]lastDeltaLast delta of each input connections of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the last delta associated with the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the last delta associated with the connection between input (neuron) i and neuron n.
[in]maxStepSizeMaximum step size.
[in]uIncrement factor (step size).
[in]dDecrement factor (step size).
[in]rReducing factor (step size / robust training).
[in]momentumMomentum factor.
[in]numberPatternsNumber of patterns.
Examples:
dim3 dimInputsNeurons(numInputs + 1, numNeurons);
KernelCorrectLayerWeights(stream, dimInputsNeurons, patternsBlockSize, rms, bestRMS, maxErrorGrowth, inputValues, localGradient, weights, learnRate, lastDeltaWithoutLearningMomentum, lastDelta, maxStepSize, u, d, robustFactor, momentum, numPatterns);

Definition at line 92 of file CorrectWeightsKernel.cu.

void KernelCorrectWeightsSelectiveInputs ( cudaStream_t  stream,
int  neurons,
int  patterns,
cudafloat rmsF,
cudafloat bestRMS,
cudafloat  maxErrorGrowth,
cudafloat inputs,
cudafloat localGradient,
cudafloat selectiveNeuronsWeights,
cudafloat selectiveNeuronsBias,
cudafloat learningRateWeights,
cudafloat learningRateBias,
cudafloat lastDeltaWithoutLearningMomentumWeights,
cudafloat lastDeltaWithoutLearningMomentumBias,
cudafloat lastDeltaWeights,
cudafloat lastDeltaBias,
cudafloat  u,
cudafloat  d,
cudafloat  r,
cudafloat  maxStepSize,
cudafloat  momentum,
int  numberPatterns 
)

Kernel to adjust the weights of a given layer. The step sizes are also updated.

Parameters
[in]streamCUDA stream.
[in]neuronsNumber of selective input neurons.
[in]patternsNumber of samples.
[in]rmsFaddress of the variable containg the current root mean square error. Should be nullptr if robust training is not used.
[in]bestRMSaddress of the variable containg the best root mean square error found so far. Should be nullptr if robust training is not used.
[in]maxErrorGrowthHow much the root mean square error is allowed to grow relatively to bestRMS before applying the robusteness techique (see the RobustLearning kernel).
[in]inputsInput data for all patterns (numInputs * numPatterns). Position [p * numInputs + i] should contain the input i for pattern p.
[in]localGradientLocal gradients of the layer (numNeurons * numPatterns). Position [p * numNeurons + n] should contain the local gradients of neuron n for pattern p.
[in,out]selectiveNeuronsWeightsInput weights of selective input neurons (one per neuron).
[in,out]selectiveNeuronsBiasBias of selective input neurons.
[in,out]learningRateWeightsLearning rate (step size) of the input connection of each neuron.
[in,out]learningRateBiasLearning rate (step size) of the bias of each neuron.
[in,out]lastDeltaWithoutLearningMomentumWeightsLast delta without learning or momentum of the input connections of each neuron.
[in,out]lastDeltaWithoutLearningMomentumBiasLast delta without learning or momentum of the bias of each neuron.
[in,out]lastDeltaWeightsLast delta of the input connections of each neuron.
[in,out]lastDeltaBiasLast delta of the bias of each neuron.
[in]uIncrement factor (step size).
[in]dDecrement factor (step size).
[in]rReducing factor (step size / robust training).
[in]maxStepSizeMaximum step size.
[in]momentumMomentum factor.
[in]numberPatternsNumber of samples (patterns).

Definition at line 180 of file CorrectWeightsSelInputs.cu.

void KernelFireLayer ( cudaStream_t  stream,
dim3 &  gridSize,
int  blockSize,
cudafloat inputs,
cudafloat weights,
cudafloat m,
int  mOffset,
int  totalNeuronsWithSelectiveActivation,
cudafloat outputs,
int  numInputs 
)

Kernel to calculate the outputs of all neurons in a given layer. For the output layer use FireOutputLayer or KernelFireOutputLayer instead.

Attention
Used when the number of connections (weights) is greather than MAX_THREADS_PER_BLOCK. Use FireLayer otherwise.
Parameters
[in]streamCUDA stream.
[in]gridSizeSize of the grid (numNeurons, numPatterns).
[in]blockSizeBlock size. Must be a multiple of 2 and not exceed MAX_THREADS_PER_BLOCK.
[in]inputsInput data for all patterns (numInputs * numPatterns). Position [p * numInputs + i] should contain the input i for pattern p.
[in]weightsInput weights of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the weight of the connection between input (neuron) i and neuron n.
[in]mImportance of the neurons of the layer (numNeurons * numPatterns). Should be nullptr for neurons without selective activation. Position [p * numNeurons + n] should contain the importance of neuron n for pattern p.
[in]mOffsetOffset of the importance of this layers neurons whithin the importance values. Tipically identical to the number of neurons in the previous layers.
[in]totalNeuronsWithSelectiveActivationNumber of neurons with selective activation in all layers (total).
[out]outputsNeuron outputs of the layer (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the output of neuron n for pattern p.
[in]numInputsNumber of inputs.
Examples:
dim3 dimNeuronsPatterns(numNeurons, numPatterns);
KernelFireLayer(stream, dimNeuronsPatterns, 512, inputs, weights, m, outputs, numInputs);
dim3 dimNeuronsPatterns(numNeurons, numPatterns);
KernelFireLayer(stream, dimNeuronsPatterns, 512, inputs, weights, nullptr, outputs, numInputs);
See also
FireLayer
FireOutputLayer
KernelFireOutputLayer
MAX_THREADS_PER_BLOCK

Definition at line 57 of file FireLayerNeuronsKernel.cu.

void KernelFireOutputLayer ( cudaStream_t  stream,
dim3 &  gridSize,
int  blockSize,
cudafloat inputs,
cudafloat weights,
cudafloat m,
int  mOffset,
int  totalNeuronsWithSelectiveActivation,
cudafloat desiredOutputs,
cudafloat outputs,
cudafloat localGradient,
cudafloat rms,
cudafloat localGradientSpaceNet,
int  numInputs 
)

Kernel to calculate the outputs of the network output layer and the local gradients of its neurons. If the layer contains selective activation neurons, the local gradients of the corresponding space network neurons are also calculated. This kernel also calculates part of the RMS.

Attention
Used when the number of connections (weights) is greather than MAX_THREADS_PER_BLOCK. Use FireOutputLayer otherwise.
Parameters
[in]streamCUDA stream.
[in]gridSizeSize of the grid (numNeurons, numPatterns).
[in]blockSizeBlock size. Must be a multiple of 2 and not exceed MAX_THREADS_PER_BLOCK.
[in]inputsInput data for all patterns (numInputs * numPatterns). Position [p * numInputs + i] should contain the input i for pattern p.
[in]weightsInput weights of the layer (numNeurons * (numInputs + 1)). Position [n * (numInputs + 1)] should contain the bias of neuron n. Position [n * (numInputs + 1) + i + 1] should contain the weight of the connection between input (neuron) i and neuron n.
[in]mImportance of the neurons of the layer (numNeurons * numPatterns). Should be nullptr for neurons without selective activation. Position [p * numNeurons + n] should contain the importance of neuron n for pattern p.
[in]mOffsetOffset of the importance of this layers neurons whithin the importance values. Tipically identical to the number of neurons in the previous layers.
[in]totalNeuronsWithSelectiveActivationNumber of neurons with selective activation in all layers (total).
[in]desiredOutputsThe network desired outputs for all patterns (numNeurons * numPatterns). Position [p * numNeurons + n] should contain the desired output of neuron n for pattern p.
[out]outputsNetwork outputs (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the output of neuron n for pattern p.
[out]localGradientLocal gradients of the network output layer (numNeurons * numPatterns). Position [p * numNeurons + n] will contain the local gradients of neuron n for pattern p.
[out]rmspartial calculation of the root mean square error (numPatterns).
[out]localGradientSpaceNetLocal gradients of the associated neurons in space network output layer (numNeurons * numPatterns). Should be nullptr for neurons without selective activation. Position [p * numNeurons + n] will contain the local gradients, for pattern p, of the neuron that calculates the importance of neuron n.
[in]numInputsNumber of inputs.
Examples:
dim3 dimNeuronsPatterns(numNeurons, numPatterns);
KernelFireOutputLayer(stream, dimNeuronsPatterns, 512, inputs, weights, m, desiredOutputs, outputs, localGradient, rms, localGradientSpaceNet, numInputs);
dim3 dimNeuronsPatterns(numNeurons, numPatterns);
KernelFireOutputLayer(stream, dimNeuronsPatterns, 512, inputs, weights, nullptr, desiredOutputs, outputs, localGradient, rms, nullptr, numInputs);
See also
FireOutputLayer
MAX_THREADS_PER_BLOCK

Definition at line 134 of file FireLayerNeuronsKernel.cu.

KERNEL RobustLearning ( cudafloat rmsF,
cudafloat bestRMS,
cudafloat  maxErrorGrowth,
int  layers,
int *  numberWeights,
cudafloat **  weights,
cudafloat **  bestWeights,
cudafloat **  learningRate,
cudafloat  r,
cudafloat **  lastDeltaWithoutLearningMomentum,
cudafloat **  lastDelta 
)

This kernel Checks if the RMS is lower than the minimum obtained so far. If so, the minimum RMS is updated and the NN weights are stored. Otherwise, the kernel checks whether the RMS exceeded the best RMS by a given tolerance and in affirmative case:

  • the best weights are restored;
  • the step sizes reduced by a given factor;
  • the momentum memories set to zero.
    Parameters
    [in]rmsFaddress of the variable containg the current root mean square error. Should be nullptr if robust training is not used.
    [in]bestRMSaddress of the variable containg the best root mean square error found so far. Should be nullptr if robust training is not used.
    [in]maxErrorGrowthHow much the root mean square error is allowed to grow relatively to bestRMS before applying the robusteness techique (see the RobustLearning kernel).
    [in]layersNumber of layers of the network (not including the input layers). Must include the layers of both the main and the space network.
    [in]numberWeightsNumber of input weights of each layer.
    [in]weightsWeights of each layer.
    [in,out]bestWeightsBest weights of each layer.
    [in,out]learningRateLearning rate (step sizes) of the input connections of each layer.
    [in]rReducing factor (step size).
    [in,out]lastDeltaWithoutLearningMomentumLast delta without learning or momentum of the input connections of each layer.
    [in,out]lastDeltaLast delta of the input connections of each layer.
    Examples:
    RobustLearning<<<1, maxNumberWeightsPerLayer, 0>>>(rms, bestRMS, maxErrorGrowth, numLayers, numberWeightsPerLayer, weightsLayers, bestWeightsLayers, learnRatesLayers, robustFactor, lastDeltaWithoutLMlayers, lastDeltaLayers);

Definition at line 25 of file RobustLearning.cu.