Problem with time series prediction

You work on a certain topic or data set and don't know how to start off with it using MemBrain? Not sure if your net design matches your problem or if there is room for optimization? Is it reasonable at all to approach your problem with NNs? Is MemBrain the correct tool to accomplish your task and to match your infrastructure?

These questions are best placed here!
Post Reply
fabbie
Posts: 2
Joined: Tue 5. Mar 2013, 10:25

Problem with time series prediction

Post by fabbie » Tue 5. Mar 2013, 10:53

Hello there!

I'm a totally newbie on ANN and Membrain stuff, that's why i'm asking for your help...

So, I'm doing a project for the developing of an ANN for the prediction of the price of the copper.
I tried a first ANN with a forecast horizon of 1 month, with 5 input neurons t-4,t-3,t-2,t-2,t-1,t , output t+1 and one hidden neuron ( I also compared other ANNs with more hidden neurons but the result doesn't improve).Training set of 23 patterns and test set of 20 patterns. I think i got quite good result. An error on the training set of 0.00121453 and on the validation set of 0.00191616.

Then i decided to extend the time-frame of the prediction to three months, i tried different layouts and different input format like:

- Input: t-4,t-3,t-2,t-2,t-1,t Output: t+3
- Input: t-12,t-9,t-6,t-3,t Output: t+3

But the error is still large, in the first case something like 0.006 on the validation set and in the second case the trending of the error increases without getting stable so there is something of totally wrong.

So do you guys have any suggestions? Which is the best approach? I also tried by using the delay,decay and differential neurons but i really didn't get how do they should work...

Thank you in advance, hope you can help me...

User avatar
TJetter
Posts: 306
Joined: Sat 13. Oct 2012, 12:04

Re: Problem with time series prediction

Post by TJetter » Wed 6. Mar 2013, 07:00

Hi there!

have you already played through the time series prediction example from the MemBrain homepage? There are very important pieces of information in there with respect to time dependent networks. The example also shows how to build up networks with decay and delay neurons/links.

With respect to your problem:

The main issue probably is that something like the price of copper can not be predicted just from its past values: The required information to determine the next approximate price just isn't represented solely in its past. There are many many more relevant inputs to this prediction problem. That's the case for all stock/economy related things I suppose.

Your experiments show that you can quite well predict the next value (month) of your time series. That's probably only because the next value (t+1) in most cases isn't too far away from the last value (t). This is something quite easy to learn for a neural net: The net learns to produce an output that is quite close to one of its inputs (i.e. the input 't') and this will be a successful net with respect to produce a low net error. However, it is of no use for predicting the real price change.
You could try the following experiment: Do not predict the value '(t+1)' but the delta between (t+1) and (t). This is something that should allow you to better see if your net really predicts something useful or not.

Another thing is that you shouldn't look at the absolute values of the net error. Use it only for trend analysis or to compare different nets (as you did). Use the pattern error viewer to check if the output is really close to its expected value and where the deviations are.

Regards
Thomas Jetter

fabbie
Posts: 2
Joined: Tue 5. Mar 2013, 10:25

Re: Problem with time series prediction

Post by fabbie » Wed 6. Mar 2013, 09:50

Thank you for the quick reply!

Yes, I also think that the main problem is that the price of copper depends on other variables like demand, offer etc... I'm working on it.
And about the error, one thing that is not clear to me is how to estimate it. I mean, as you told me I should look at the pattern error viewer to evaluate the capability of learn/generalize of the ANN, but how can I quantify it?

Thank you again..

User avatar
TJetter
Posts: 306
Joined: Sat 13. Oct 2012, 12:04

Re: Problem with time series prediction

Post by TJetter » Thu 7. Mar 2013, 07:44

fabbie wrote:but how can I quantify it?
One approach is to consider what the net error is and how to interpret it. The MemBrain help file contains details about the net error if you want to learn more about this.
However, as a quick guide, be sure to check the following setting:
<Teach><Configure Net Error Function...>
Left side selection : "Squared Deviation"
Right side selection: "Root Mean of Sum"

This will cause your net error to reflect the average deviation over all validation patterns which might already be a good option.

If you want to go into more details you will need to apply your own analysis and criteria: MemBrain certainly can't tell you if your net is good or bad since it doesn't know anything about its purpose or about copper ;-)
So you need to export your result data into CSV, import it into a spread sheet program an perform your own delta analysis.
To simplify the generation of the result file I would strongly recommend to use a script like given here:
viewtopic.php?f=14&t=233
The script does all the MemBrain work for you and you can concentrate on the data analysis and post processing in your favourite spread sheet program. If you want to you can expand the script to automatically open the exported file in your spreadsheet program - just one idea for a script enhancement.

Finally one important thing in general when dealing with time variant nets: Set the 'Lesson Pattern Selection' method of the used teacher to 'Ordered' and enable the check box 'Reset Net Before Every Lesson' for the used teacher!
You can do this once manually or even implement it in the script, too. Note that the script might select a teacher (probably RPROP) so be sure to adjust the mentioned settings for the teacher the script is using!

Regards
Thomas Jetter

Post Reply