Neuron Output = Input

You work on a certain topic or data set and don't know how to start off with it using MemBrain? Not sure if your net design matches your problem or if there is room for optimization? Is it reasonable at all to approach your problem with NNs? Is MemBrain the correct tool to accomplish your task and to match your infrastructure?

These questions are best placed here!
Post Reply
klubow
Posts: 14
Joined: Thu 31. May 2012, 13:16

Neuron Output = Input

Post by klubow »

Hi there,

I want to use output value back in my network when the next pattern is applied.
But I need clear value, so how to set up a hidden neuron to basically pass it's input to output ?

From help file:
"Act = Act * ActSustainFactor + ActFunc ( InputSum - ActivationThreshold )"

So setting ActSustainFactor=0 and ActivationThreshold=0 makes

Act = ActFunc(InputSum)
but how to get rid off ActFunction ? Each function changes value of Input.

Any ideas hot to get Output = Input ?
User avatar
TJetter
Posts: 346
Joined: Sat 13. Oct 2012, 12:04

Re: Neuron Output = Input

Post by TJetter »

klubow wrote:but how to get rid off ActFunction ? Each function changes value of Input.

Any ideas hot to get Output = Input ?
Act func = IDENTICAL produces output = input as long as ActivationThreshold and ActSustainFactor are set to 0.

Also ensure the following:

- Leave the Output Fire Level settings at their defaults.
- Lock the Activation Threshold (so that it won't be changed during teach)
- Set the weight of the incoming link of the neuron to 1 and lock it, too.

See example attached to this post.

Please post more information in case this doesn't work as you expect it to.

Kind regards
Attachments
SimpleHiddenReproducer.mbn
Example showing an input, a hidden and and output neuron and how they simply reproduce the input neuron's activation
(561 Bytes) Downloaded 949 times
Thomas Jetter
klubow
Posts: 14
Joined: Thu 31. May 2012, 13:16

Re: Neuron Output = Input

Post by klubow »

Thank you Thomas for the replay, also in the other post.

I am aware of IDENTICAl activation function but it works only for inputs from -1 to 1 so it clips below and above values.
I can work around by dividing inputs so they are in the range -1 .. 1.

Now I have a trained net and I want it to employ outside Membrain.
I am aware of exporting to csv or C code, where you get all the detail.
Is there any tutorial how to use C code ?

Is there an easy way to convert trained net to mathematical equation/function ?

Beacuse at the end it is basically a function - with variables as inputs and lot of other variables but in trained net they are considered to be constant.
That would be a very nice feature.

--
Krzysztof
User avatar
TJetter
Posts: 346
Joined: Sat 13. Oct 2012, 12:04

Re: Neuron Output = Input

Post by TJetter »

klubow wrote:I am aware of IDENTICAl activation function but it works only for inputs from -1 to 1 so it clips below and above values.
I can work around by dividing inputs so they are in the range -1 .. 1.
You don't have to perform this manually: MemBrain supports the so called 'Normalization' setting for input and output neurons. There you can specify a user defined value range for each neuron which is then automatically mapped by MemBrain to the applicable internal range of the neuron according to its activation function.
You can access the Normalization feature of a neuron via its property dialog.

Hidden neurons do not support this feature.

Activation values of neurons used internally by MemBrain are always limited to -1 and 1, that's true. Can you tell me in which situations this might be a poroblem for you? Then I can possibly suggest a solution.
klubow wrote:Is there any tutorial how to use C code ?
See section 'Source Code Generation' in the MemBrain help. Please let me know if you need more information.
klubow wrote:Is there an easy way to convert trained net to mathematical equation/function ?
No, there isn't. And by definition, there can't be an 'easy one' since it would have as many terms as there are neurons and links with their thresholds, weights, activation functions and additional parameters.
A formula would simply be put all these calculations into one large series of mathemetical operations but this would be no better than using the C-Code anyway.

Regards
Thomas Jetter
klubow
Posts: 14
Joined: Thu 31. May 2012, 13:16

Re: Neuron Output = Input

Post by klubow »

You don't have to perform this manually: MemBrain supports the so called 'Normalization' setting for input and output neurons. There you can specify a user defined value range for each neuron which is then automatically mapped by MemBrain to the applicable internal range of the neuron according to its activation function.
You can access the Normalization feature of a neuron via its property dialog.
I have data in range 1.2 .. 1.5 and I did normalize it to min 1.1 max 1.6.

How the mapping is done - could you write the formula? Which function do you use?
As I understand it is done on input values before any other calculations.
I am not sure if output value of the same neuron is mapped back or output neuron value is mapped back?
When neuron displays it Activation (neuron properties-neuron appereance) it is scaled back, at least it seems so.

Could you explain all the steps and calculations for neuron and where the mapping is done:
1) InputSum (weighted in case of hidden and output neurons)
2) Act = Act * ActSustainFactor + ActFunc ( InputSum - ActivationThreshold )
3) if Act >= Upper Fire Threshold and OutputFireLevel=Act and # >= Output Recovery Time then Output=Act

Or maybe you could paste that part of the source code ?
klubow
Posts: 14
Joined: Thu 31. May 2012, 13:16

Re: Neuron Output = Input

Post by klubow »

Below I pasted part of the code I wrote,
could you have a look and tell me if it is ok ?

How the numbering of neurons is done ?
0 is the first neuron in Lesson Editor ?

Edit: changed the code, now seems to work,
but the result I get are not exactly the same,
there is a small difference in fraction part.

Code: Select all

int i, j, k;
	int NoInNeurons = 10;
	int NoOutNeurons = 2;
	double TabInputValue [10]; 

	T_ACT_FLOAT TabOutputActValue[2];
	T_ACT_FLOAT TabOutputValue[2]; 

	T_ACT_FLOAT OutputActValue;
	T_ACT_FLOAT OutputValue; 

	T_ACT_FLOAT *ptrOutputActValue = &OutputActValue;
	T_ACT_FLOAT *ptrOutputValue = &OutputValue;
	


	printf( "Test SIECI \n" );
	printf( "____________________\n" );
	printf( "\n" );

	//last training lesson record
	TabInputValue[0] = 1.3323;
	TabInputValue[1] = 1.3334;
	TabInputValue[2] = 1.3323;
	TabInputValue[3] = 1.3331;
	TabInputValue[4] = 0.236;
	TabInputValue[5] = 0.382;
	TabInputValue[6] = 0.618;
	TabInputValue[7] = 1.618;
	TabInputValue[8] = 2.618;
	TabInputValue[9] = 4.236;


	NeuralNetInitAndReset();
		

	for(i=0; i<NoInNeurons; i++)
	{
		NeuralNetApplyInputAct(i, TabInputValue[i]);

		printf( "InputActValue: ");
		printf( "%d", i);
		printf(" = "); 
		printf("%6.6f", TabInputValue[i]);
		printf( "\n");
	}


	NeuralNetThinkStep();



	for(j=0; j<NoOutNeurons; j++)
	{
	NeuralNetGetOutputAct(j, ptrOutputActValue); 
	
		printf( "OutputActValue: ");
		printf( "%d", j);
		printf(" = "); 
		printf( "%6.6f", *ptrOutputActValue);
		printf( "\n");
	}



   
User avatar
TJetter
Posts: 346
Joined: Sat 13. Oct 2012, 12:04

Re: Neuron Output = Input

Post by TJetter »

klubow wrote:Edit: changed the code, now seems to work,
but the result I get are not exactly the same,
there is a small difference in fraction part.
The code looks fine so far and yes, index 0 ist the left most one in the lesson editor.
With respect to the differences: Check out the menu option <Code-Generation><Configuration...> and ensure that everything is adjusted to 64 bit resolution.
Then the calculations should exactly match.

Regards
Thomas Jetter
Post Reply