Cascade Correlation with Hidden Node Networks

[MrCreosote]

Just to see what would happen, I tried CC on a custom built network with hidden nodes and connections which were not full layer to layer.

First a quick description of my custom network:

• Imagine two 3-layer networks (In,Hid,Out) where they are combined, in parallel, to produce a single output: the output nodes of the two networks would become another hidden layer and the new output node would be an additional, 4th layer.
• The theory is that if inputs come from 2 different classes, allow the nnet to basically do a reduction in dimensionality before they are combined netwise to get an output.

What I thought I saw was that CC would basically set the output of the Hiddens to ZERO and then start working at turning them on, one at a time. (This is a guess on my part since I was not able to make sense out of the process.)

If that is true, however, that would be nice because one could build an architecture with an abundance of Hidden nodes, connected as desired, and CC would simply arrive at some solution where a number of these nodes would be unused.

QUESTION: Is this how CC would operate on such a network?

[Admin]

Hi Tom,

won't have the time today to reply to this in more depth. Would suggest that you download the new MemBrain version that has the bug fix on the Cascade Correlation teachers (just went online) and give it a try first. This might simplify things, i.e. discussing about real observations is easier than talking about 'what would happen'

Regards,
Thomas

[MrCreosote]

Explored behavior a bit and it looks like this probably isn't the way to do things.

What appears to happen is that newly generated nodes connect everything together: Inputs, Hiddens, Outputs.

The vast majority of original hiddens had values of 0 so it seems they were removed so to speak.

I believe I've seen enough to conclude that this is not the way to do things with CC.

Thanks
Tom

[Admin]

There's one thing you could try additionally:

Neurons in MemBrain have a property flag 'Allow Teacher to Connect Output'. If you deactivate this flag for the neurons you don't want to receive new output connections then the CC teacher won't do so.

Don't know if this helps with your approach, though.

Regards,
Thomas