First, it is difficult to find the actual formula for this error type. The one that I like the best is how MathWorks tells users how to user define it since it is not in their toolbox. http://www.mathworks.com/support/soluti ... on=1-1A4F6
NOTE: There is one requirement for this calculation and that is SoftMax Activation must be used on the output nodes. (This is a map into (0,1) with the additional constraint that all the outputs add up to 1. ...something about getting probability outputs so you have the right units for entropy (??? this is beyond my pay grade) )
From their definition:
The cross-entropy error, C is then expressed as follows:
C = - sum [all cases and outputs] (d*log(y) + (1-d)*log(1-y) )
The derivative of this error function for a given output and training example (this is the value we actually back-propagate) is as follows:
dC/dy = - d/y + (1-d)(1-y)
_____________________________
Definition of C in Membrain's new Scripted Elements would define the function CalcaulteNetErrorSummand as
return = - d*log(y) - (1-d)*log(1-y) where d = targetActivation and y = Activation
This appears to be the only required input which begs the question, do we also need user defined partial derivatives?
- Does Membrain use them for BP?
- Does Membrain calculate them numerically from the function itself?
Tom
PS. Cross Entropy Error seems to be everywhere starting in the early 2000's. Would it be possible to build it in and give it a Radio Button?
