The curse of the curve

Curve fitting. It's supposed to be a trading system developper worst enemy: when you develop your system and you change its settings in such a way that the system performs the best in the past, you're lying to yourself and you know it.

The market changes all the time: with every tick you have an arrival of new info, and this changes the dynamics of the system (hence dynamic nonlinear system). Projecting the curve-fitted system (curve-fitted to the past) into the future may cause a problem because the market's changing. But what if you keep on doing this all the time? And what about optimization period?

I think the answers to these questions are remarkably simple: there's nothing wrong with curve fitting, unless you do it continuously: you can't predict the market - I can't - but you can follow it very closely. By coding a continuous optimization algorithm into your program you might even do better than historical optimized performance.

The optimization period is also easily obtainable: look at your equity curve of your trading system and where it starts bending (put a moving average of N periods on the chart), that's where the market's fading memory starts impacting the performance: much like the knack-criterium in principal component analysis: where you see a clear change in direction (like a broken branch of a tree) you place the stop. You should repeat this a couple of times to see the average period after which the system diverges from its (optimal) profitability.

The best link on continuous optimization is this article. It contains an example of an optimization project of one of the standard expert advisors that come with the MetaTrader terminal: the macd expert.

You want a rule of thumb? Just compare your in-sample forecasting error with your out-of-sample forecasting error: it shouldn't be much greater, when there is a significant difference, you have over-curve-fitted.