Time variant vs invariant network

You work on a certain topic or data set and don't know how to start off with it using MemBrain? Not sure if your net design matches your problem or if there is room for optimization? Is it reasonable at all to approach your problem with NNs? Is MemBrain the correct tool to accomplish your task and to match your infrastructure?

These questions are best placed here!
Post Reply
klubow
Posts: 14
Joined: Thu 31. May 2012, 13:16

Time variant vs invariant network

Post by klubow » Mon 26. Nov 2012, 19:31

Hello,
Can anyone point me in the right direction ?

I have data set with (4 inputs and 1 output) x 1500 records.
Is it better to deal with it in time variant network:
4 inputs each one with 1500 delay neurons connected to all hidden and output neurons

or

use data set prepared so that each record is like that:
A(0) .. D(0), A(-1)..D(-1) ............. A(-1500)..D(-1500)
(that makes another problem - what is learning data limit ? my data set is 525Mb)

How does it influence on time spent on learning ?
Does Membrain have any limits on number of neurons, links ?

What does it mean when trying to load lesson in csv format "File Access Error" - too big file ?

By the way - great piece of software,
I would propose only one improvement - estimated time when learning is finished.

Regards
Krzysztof

User avatar
TJetter
Posts: 306
Joined: Sat 13. Oct 2012, 12:04

Re: Time variant vs invariant network

Post by TJetter » Wed 5. Dec 2012, 19:46

Hi Krzysztof,

sorry for the late reply, I didn't receive notification from the board about this post and just stumbled upon it by accident. Sometimes phpBB doesn't work like it should ;-)
klubow wrote:Can anyone point me in the right direction ?

I have data set with (4 inputs and 1 output) x 1500 records.
Is it better to deal with it in time variant network:
4 inputs each one with 1500 delay neurons connected to all hidden and output neurons

or

use data set prepared so that each record is like that:
A(0) .. D(0), A(-1)..D(-1) ............. A(-1500)..D(-1500)
Neither of the two approaches seems to be plausible to me. MemBrain can work with large data sets and huge amounts of links and neurons but this brings extremely long training times and the huge number of neurons you plan makes the network much too powerful with respect to its memorization capacities. It will not be able to generalize anymore in a satisfactory way.

Have you worked through the Mackey Glass time series prediction example that is available in the examples from the homepage? There you could get ideas on how to approach a time series prediction problem like yours.
In addition to the net examples given in the Mackey Glass example you could also work with feedback links (I assume you are just about to test this according to your other post here in the forum?).
klubow wrote:What does it mean when trying to load lesson in csv format "File Access Error" - too big file ?
Yes, the size could be the reason. However, if you really need to have such a huge input file you could split it into smaller pieces, import these into MemBrain and save them as .mbl files (the MemBrain internal lesson file type) using the Lesson Editor's 'Lesson Files' menu. Once you have these files available you can use the lesson editor to load the first one of these files and then use the function 'Append Lesson to Current Lesson' in the Lesson Editor's menu.
Note that CSV reading is drastically slower than reading in the mbl format. Also, you should then be able to save the full lesson as mbl file and also load it back into MemBrain later.

Kind regards
Thomas Jetter

Post Reply