I have some problems with training CNN 🙁
For example, 3D data (width, height, depth):
Input 6x6x3, 1 core 3x3x3, output = 4x4x1 => pool: 2x2x1
This and other tutors are explain to calc deltas for weights and Input only for 2D (without depth):
input * output=deltas for 2D weights
filter * out = input delta
But how I can to calc weights deltas for 3D filters?
I must to multiply each input by output as below?
FilterLayer1Delta = OutputDelta * InputLayer1
FilterLayer2Delta = OutputDelta * InputLayer2
FilterLayer3Delta = OutputDelta * InputLayer3
FilterLayerNDelta is delta for layer of the current filter
InputLayerN is input image for layer of the current filter
This is not a 3d filter, this is a standard 2d filter. A 3d filter looks like this: https://i.imgur.com/jriyCTU.png?1 (orange box). This is a common misconception. A 2D convolution is actually more of a collection of filters (represented by depth) that each have their own depth equal to the depth of the input layer (aka channels).
However, I think what you’re trying to ask is how to calculate the final value for each filter. I have already answered question like this on ai.stackexhange, see here for the link: https://ai.stackexchange.com/a/13591/26726
Essentially, you multiply each channel of the filter with the corresponding input channel at a specific location and add these to get a single value, then you sum each value at the output location for all the outputs for every channel to get the final value at that location for that specific filter in that specific convolutional layer.
Here is an image of the backward pass, with random gradients assigned for the output, and back propagated to find the gradient for each weight in filter 1 channel 1:
Answered By – Recessive
Answer Checked By – Senaida (AngularFixing Volunteer)