Sunday, February 3, 2008
Entropy rate
Entropy rate is the time density of the average information in a stochastic process. The recurrence period density entropy's goal is to determine the periodicity in the phase space of a system. Check Wikipedia to find more about it because it has a lot of applications, like financial timeseries: I'll elaborate on the application to technical analysis later.
Wednesday, January 16, 2008
The curse of the curve
Curve fitting. It's supposed to be a trading system developper worst enemy: when you develop your system and you change its settings in such a way that the system performs the best in the past, you're lying to yourself and you know it.
The market changes all the time: with every tick you have an arrival of new info, and this changes the dynamics of the system (hence dynamic nonlinear system). Projecting the curve-fitted system (curve-fitted to the past) into the future may cause a problem because the market's changing. But what if you keep on doing this all the time? And what about optimization period?
I think the answers to these questions are remarkably simple: there's nothing wrong with curve fitting, unless you do it continuously: you can't predict the market - I can't - but you can follow it very closely. By coding a continuous optimization algorithm into your program you might even do better than historical optimized performance.
The optimization period is also easily obtainable: look at your equity curve of your trading system and where it starts bending (put a moving average of N periods on the chart), that's where the market's fading memory starts impacting the performance: much like the knack-criterium in principal component analysis: where you see a clear change in direction (like a broken branch of a tree) you place the stop. You should repeat this a couple of times to see the average period after which the system diverges from its (optimal) profitability.
The best link on continuous optimization is this article. It contains an example of an optimization project of one of the standard expert advisors that come with the MetaTrader terminal: the macd expert.
You want a rule of thumb? Just compare your in-sample forecasting error with your out-of-sample forecasting error: it shouldn't be much greater, when there is a significant difference, you have over-curve-fitted.
The market changes all the time: with every tick you have an arrival of new info, and this changes the dynamics of the system (hence dynamic nonlinear system). Projecting the curve-fitted system (curve-fitted to the past) into the future may cause a problem because the market's changing. But what if you keep on doing this all the time? And what about optimization period?
I think the answers to these questions are remarkably simple: there's nothing wrong with curve fitting, unless you do it continuously: you can't predict the market - I can't - but you can follow it very closely. By coding a continuous optimization algorithm into your program you might even do better than historical optimized performance.
The optimization period is also easily obtainable: look at your equity curve of your trading system and where it starts bending (put a moving average of N periods on the chart), that's where the market's fading memory starts impacting the performance: much like the knack-criterium in principal component analysis: where you see a clear change in direction (like a broken branch of a tree) you place the stop. You should repeat this a couple of times to see the average period after which the system diverges from its (optimal) profitability.
The best link on continuous optimization is this article. It contains an example of an optimization project of one of the standard expert advisors that come with the MetaTrader terminal: the macd expert.
You want a rule of thumb? Just compare your in-sample forecasting error with your out-of-sample forecasting error: it shouldn't be much greater, when there is a significant difference, you have over-curve-fitted.
Friday, January 11, 2008
The ideal trading system
I was wondering lately, what would the -almost- perfect trading system look like?
-A high % of winners
-A good win-loss ration (like 3-to-1)
-relative drawdowns of less than ..% (let's take 10)
-small % of the account at risk per trade the system makes
Most equity curves of individual trading systems that seem to shoot to the moon usually have large drawdowns: the stable system (high % winners) is usually combined with an aggressive money management strategy (e.g. set number of lots = 50% of my account * leverage ratio / 10.000(0): depending on whether you have a mini account or not). The high % of winners lets you do that but it would be wiser to set your asset allocation % of your AUM (assets under management, or for the non-professional: your total account value) to a 1 or 0.5 times Kelly (you see I'm a really big fan of the Kelly Criterion).
If you don't use a money management stoploss in your trading system of about 2% you should incorporate this in your money management strategy and be really conservative if you're in it for the long haul. I personally think gambling is for people who don't know statistics and/or have a latent urge to become poor, but if you do have the Vegas mentality, just go for it and place number of lots equal to account * leverage ratio / 10.000 (something tells me you'll have a mini account and not a regular account, if you do this).
An example of growth-optimal strategies who allocate money according to the Kelly Criterion should not be confused with exagerated conservativity: it's just the only smart way to trade: let me give an example:
f: what we seek: the % of the account to place in this system
b: the odds received on the winner: 3-to-1
p: probability of winning: let's take 70%: I've seen this more than once
q: probability of losing: 30%
f would in this case be equal to:
f = ((b*p)-q)/b
f = ((3*0.7)-O.3)/3
f = 0.6
60% of the account should be allocated to this trading system to optimize the growth of your account. The gambler will pick 100% because he loves that 70% (and will probably boast about it to everyone who wants to hear), the bonus pater familias type will be overly concerned with the 30% losers and will place 20% of his account in this system. Both of them are wrong!
The fun part of this is that the b,p and q's are dynamic: they change with every trade you make and so should you asset allocation: that's why you need to code it into your EA! The more you win, the more money you want to put in it, and when there's is a "loosing streak": a number of consecutive losses, you want to scale down your exposure to this "moneyburner", or to this temporary "money burning phase" the system goes through. Vice versa, when things go well you want them to go better. If you know that your system's consecutive losses are clustered together (maybe because the EA opens more than 1 lot in the same bar with the same "logic" behind it) you could even anticipate a loosing streak and stop live trading for a while, but I'm not a big fan of doing that: I'm a big fan of coding the Kelly logic into your EA, putting it on a server and letting it do its thing for the next couple of months.
So there you have it: optimize, don't be conservative or a gambling man!
(or woman)!
-A high % of winners
-A good win-loss ration (like 3-to-1)
-relative drawdowns of less than ..% (let's take 10)
-small % of the account at risk per trade the system makes
Most equity curves of individual trading systems that seem to shoot to the moon usually have large drawdowns: the stable system (high % winners) is usually combined with an aggressive money management strategy (e.g. set number of lots = 50% of my account * leverage ratio / 10.000(0): depending on whether you have a mini account or not). The high % of winners lets you do that but it would be wiser to set your asset allocation % of your AUM (assets under management, or for the non-professional: your total account value) to a 1 or 0.5 times Kelly (you see I'm a really big fan of the Kelly Criterion).
If you don't use a money management stoploss in your trading system of about 2% you should incorporate this in your money management strategy and be really conservative if you're in it for the long haul. I personally think gambling is for people who don't know statistics and/or have a latent urge to become poor, but if you do have the Vegas mentality, just go for it and place number of lots equal to account * leverage ratio / 10.000 (something tells me you'll have a mini account and not a regular account, if you do this).
An example of growth-optimal strategies who allocate money according to the Kelly Criterion should not be confused with exagerated conservativity: it's just the only smart way to trade: let me give an example:
f: what we seek: the % of the account to place in this system
b: the odds received on the winner: 3-to-1
p: probability of winning: let's take 70%: I've seen this more than once
q: probability of losing: 30%
f would in this case be equal to:
f = ((b*p)-q)/b
f = ((3*0.7)-O.3)/3
f = 0.6
60% of the account should be allocated to this trading system to optimize the growth of your account. The gambler will pick 100% because he loves that 70% (and will probably boast about it to everyone who wants to hear), the bonus pater familias type will be overly concerned with the 30% losers and will place 20% of his account in this system. Both of them are wrong!
The fun part of this is that the b,p and q's are dynamic: they change with every trade you make and so should you asset allocation: that's why you need to code it into your EA! The more you win, the more money you want to put in it, and when there's is a "loosing streak": a number of consecutive losses, you want to scale down your exposure to this "moneyburner", or to this temporary "money burning phase" the system goes through. Vice versa, when things go well you want them to go better. If you know that your system's consecutive losses are clustered together (maybe because the EA opens more than 1 lot in the same bar with the same "logic" behind it) you could even anticipate a loosing streak and stop live trading for a while, but I'm not a big fan of doing that: I'm a big fan of coding the Kelly logic into your EA, putting it on a server and letting it do its thing for the next couple of months.
So there you have it: optimize, don't be conservative or a gambling man!
(or woman)!
Monday, January 7, 2008
Are bigger timeframes really less noisy?
If you select your timeframe for a trading system there are many things to consider: is it noisy, does it give good trading signals, how far should I put my stoploss and can my strategy take this if I only want to risk a certain % of my account per trade. I think it makes sense not to risk more than 2% per trade. You should calculate this how many pips that are for you (depending on the €'s or $'s in your account and depending on how much leverage you use).
There are many methods you can use for stoploss. The turtles used 2 ATR's, other people use a set number (like 15 pips for example), but I think the stoploss method should incorporate two things: volatility and p&l. So after you've done this and you have extensively backtested and forward-tested your trading system on demo, you can go ahead and launch. But we're getting off topic. The question here was: are longer timeframes better for trading or not?
So I came up with a small tool to calculate the easiest statistic you could imagine (no Hurst for now). If a bar is a red bar (the close is below the open), how big a chance do you have (according to the timeseries you put the tool on) that the next bar will also be a red bar, and if a bar is a green bar (close is above the open), how much % chance do you have that the next bar will be a green one too? If you put this in Metatrader, and you start flipping through the different timeframes, and you go from high to low (from 1month, to 1 week, to daily etc. to 1 minute) you'll see something peculiar appearing: I call it the complexity structure (I'm sure there's a scientific name for this but I just don't care). I added both %'s and called the result 2 bar Trend probability. Normally you would think that the real small timeframes (like the 1minute chart or the '5) would exhibit low 2 bar-trending behaviour, and that this would linearly increase: a bigger timeframe gives less 2 bar noise, and more 2 bar trends. But this is not always the case!
Some currency pairs (this changes so don't expect me to name them) exhibit a complexity structure that is not linear: the maximum 2 bar trend % increases all the way up to e.g. the 4h chart, but then it is flat or it decreases gradually. In that case there is no merit in trading the larger timeframes (because you think they'll give better signals), and you're better off trading the smaller ones: it becomes a trade-off between decrease in 2 bar trending probability and smaller timeframes. Smaller timeframes are better because they give you more trading signals, but they are worse because they have a tendency to be noisy, but it is always dangerous to generalize and this is a nice example.
p.s.: imagine a casino with a roulette wheel with 50 boxes, no double 0 and 27 red boxes and 23 green ones, would you take the bet? When looking at the large timeframes, some of the 2 bar probability %'s are above 50 %, some even at 54%: the simple trading strategy would be to sell when a bar closes down and buy when a bar closes up, with a payoff % of 54, and a per-trade payoff of ... I would take the bet; divide my dollars into 1 cent pieces and scale my betting money according to the Kelly Criterion.
There are many methods you can use for stoploss. The turtles used 2 ATR's, other people use a set number (like 15 pips for example), but I think the stoploss method should incorporate two things: volatility and p&l. So after you've done this and you have extensively backtested and forward-tested your trading system on demo, you can go ahead and launch. But we're getting off topic. The question here was: are longer timeframes better for trading or not?
So I came up with a small tool to calculate the easiest statistic you could imagine (no Hurst for now). If a bar is a red bar (the close is below the open), how big a chance do you have (according to the timeseries you put the tool on) that the next bar will also be a red bar, and if a bar is a green bar (close is above the open), how much % chance do you have that the next bar will be a green one too? If you put this in Metatrader, and you start flipping through the different timeframes, and you go from high to low (from 1month, to 1 week, to daily etc. to 1 minute) you'll see something peculiar appearing: I call it the complexity structure (I'm sure there's a scientific name for this but I just don't care). I added both %'s and called the result 2 bar Trend probability. Normally you would think that the real small timeframes (like the 1minute chart or the '5) would exhibit low 2 bar-trending behaviour, and that this would linearly increase: a bigger timeframe gives less 2 bar noise, and more 2 bar trends. But this is not always the case!
Some currency pairs (this changes so don't expect me to name them) exhibit a complexity structure that is not linear: the maximum 2 bar trend % increases all the way up to e.g. the 4h chart, but then it is flat or it decreases gradually. In that case there is no merit in trading the larger timeframes (because you think they'll give better signals), and you're better off trading the smaller ones: it becomes a trade-off between decrease in 2 bar trending probability and smaller timeframes. Smaller timeframes are better because they give you more trading signals, but they are worse because they have a tendency to be noisy, but it is always dangerous to generalize and this is a nice example.
p.s.: imagine a casino with a roulette wheel with 50 boxes, no double 0 and 27 red boxes and 23 green ones, would you take the bet? When looking at the large timeframes, some of the 2 bar probability %'s are above 50 %, some even at 54%: the simple trading strategy would be to sell when a bar closes down and buy when a bar closes up, with a payoff % of 54, and a per-trade payoff of ... I would take the bet; divide my dollars into 1 cent pieces and scale my betting money according to the Kelly Criterion.
Saturday, January 5, 2008
Hurst exponent, the holy grail of trading?
The Hurst exponent truly is a remarkable thing. I won't bother you with the stuff you can find on Wikipedia about it, but it is an incredibly interesting statistic for all of us who are in the trendfollowing camp. It's a way of evaluating the persistence in timeseries. Actually I can't think of a single situation where you wouldn't want your timeseries to be persistent (unless maybe if you're someone with a trading system that can be defined as "sell when RSI drops below 80 and buy when it goes back above 20"). But some guy called Darwin came up with a law that quickly lets us ignore these people and their "system".
So I came up with a tool for Metatrader that shows you the highest Hurst exponent of all currency pairs of all timeframes. Anyone interested?
So I came up with a tool for Metatrader that shows you the highest Hurst exponent of all currency pairs of all timeframes. Anyone interested?
The ideal timeseries to trade
Let's imagine if you were the god of the markets and you could do whatever you wanted, what would you make of EUR/USD? Which characteristics would you give your timeseries?
My ideal timeseries would be:
- a lot less noise than what is currently present (thank god for digital signal processing)
- a normal probability distribution function (I hate those heavy tails)
- a Hurst exponent of 0.9: very persistent trends
- at least one positive Lyapunov exponent
- all timeseries would be stationary (ADF & PP)
- long Lyapunov time
- low dimensionality & easily embeddable
My ideal timeseries would be:
- a lot less noise than what is currently present (thank god for digital signal processing)
- a normal probability distribution function (I hate those heavy tails)
- a Hurst exponent of 0.9: very persistent trends
- at least one positive Lyapunov exponent
- all timeseries would be stationary (ADF & PP)
- long Lyapunov time
- low dimensionality & easily embeddable
Disclaimer
All ideas, opinions, and/or forecasts, expressed or implied, are for informational purposes only and should not be construed as a recommendations to invest, trade and/or speculate in the markets. Any investments, trades and/or speculations made in light of the ideas, opinions and/or forecasts expressed or implied herein, are committed at your own risk, financial or otherwise.
The best idea of 2008
Let me make 1 exception to confirm my rule of never wanting to predict nonlinear dynamic systems (financial markets) unless I've done an adequate analysis.
The best investment of 2008 very well could be:
long the ABX index, the higher tranches
Goldman made most of its earnings where the other IB's lost because it was short the ABX last year. I think given the short analysis I ran on the relevant data there is a pretty large possibility that this could be the reversal of the year. But don't expect me to explain the trade because I'm not an investor in cdo's, it's just a very general hunch/guess.
The best investment of 2008 very well could be:
long the ABX index, the higher tranches
Goldman made most of its earnings where the other IB's lost because it was short the ABX last year. I think given the short analysis I ran on the relevant data there is a pretty large possibility that this could be the reversal of the year. But don't expect me to explain the trade because I'm not an investor in cdo's, it's just a very general hunch/guess.
why?
This blog will be 100% about system trading: this is an automated way of trading the financial markets which doesn't require any human interaction (in the trading process): the computer makes the buy or sell decisions, not the trader. The trader is the one who makes the trading system. I use the Metatrader terminal for trading fx. The Metatrader terminal is hands down the best product on the market and it's free. There are numerous forums about automated trading but the thing I'm missing on there is people who are systematic fx traders with a sound background in economics, econometrics, chaos theory etc.
Subscribe to:
Posts (Atom)