What Is Out-Of-Sample Testing And Why Is It Critical For Success In Forex?


What Is Out-Of-Sample Testing And Why Is It Critical For Success In Forex?

Assuming you have a manual system, an automatic robot or EA (expert advisor), you first need to check how this system would have been doing in the past if you had actually used it. This gives a vital view of the system performance and let you know what to expect and whether this system should deserve any further attention from you.

Indeed one of the main issues you are going to face is ensuring a system has a high probability of its future profitability. This issue is the most important problem in automated robots because the most important question is not about what will be the returns but about will the system survive. Out of sample testing gives us an idea about how our system performs even out side of its designed period.

Out of sample testing means doing stress tests that can tell us how a strategy behaves when it is not being optimized to specific market conditions. You develop a strategy across a certain set of market conditions and then run it across a set where no optimization was done. How to do it? Take a system you have optimized from date A to B and then run it from C to D, then evaluate the system performance on the C to D period to evaluate if the system did ok. Any strategy evaluated in an out of sample must behave exactly as in the original period. This means that after doing the optimization you should run an out of sample test. If the result is doing worse then the system is useless and you should not spend more time on it, and surely not trade it.

If the results of the out of sample period is looking OK, the next step is that you should continue the testing process in real time until you are fully confident with the system performance. You only then can put in your real money.

One of the great advantages to this method of system testing and trading is that your system will better adapt to changes in market behavior that will happen over time. Markets do change with time. Many systems have made profits for several years and then stopped, when some of the market conditions have changed.

Out of sample testing is one of the most important tools, which I as a trader use on powertrendsystem.info. The forum and community I am part of depend on this tool very much and we never release a system without it first passing this test.

To summarize, out of sample testing is a powerful way in which you can test a strategy, make sure the system does what you expect it to do even when it meets new market conditions which it have never seen before.