Backtesting with feedback

[Johnny]

I imagine that this question is a no-brainer for high frequency traders and designers of efficient execution systems.



When back testing medium and long term strategies it seems reasonable (even for large players) to assume that the strategy has no effect on prices. i.e. that the agent is a price taker. However, for high frequency strategies this assumption doesn't stand up. Even small traders can have an effect on the order book, which feeds back to inference to order placement. How do people model this feed back effect? Is this a game theory thing? Specifically, how do people test trading strategies when the effect of the trading strategy upon itself (via order placement) is unknown a priori?

[Tradenator]

We use actual fill prices whenever we can (about 7-8 years of futures history), and would probably fall into your long term category with holding periods longer than a day.  It's probably tough to get similar data for a high frequency system unless you have another process to cannibalize data from.



Does Oanda cover this anywhere on their site?

[Johnny]

So your suggestion is to back-test one strategy by cannibalising fill data from another similar strategy. Seems reasonable. Care to say more about how you would cannibalise?



btw, I couldn't see anything relevant on the Oanda site on a quick skim.

[FDAXHunter]

The impact is not unknown a priori. There are assumptions that one can make. Just as one can assume that long term position won't have an impact on prices (which is not true, by the way. Long term traders regularly account for the slippage caused by their positions in backtesting, although they will usually assume that prices will return to a pre-trade equilibrium before continuing to evolve). Backtesting always makes assumptions about the state of market and the effect of trading in the market. You can never get around that.



Via micro-structural models, we know what our impact is on price given price, time and size constraints on average, for a given market state. We can use that knowledge to account for the cost and the price movement in our trading system. We can account for the behavior of the order book after our trade from our structural model and update the model accordingly with our trades, then feed it to the next trading system to update.



Backtesting in high frequency is different than backtesting on longer time frames, where we can simply do an accounting trick by adding slippage which will guarantee our position and hopefully not impacting the evolution of price too severely to significantly alter the trade (a tenuous assumption... if your size is big enough!).

High frequency systems that are not price takers will inevitably end up with a probability of not executing. So the easiest thing to do then, is to assign a price, a quantity and a probability of being filled and crank all that through a netting engine (in backtesting), updating your probabilities as you go. There are feedback effects and you will need to iterate many times as different systems add and pull orders or modify prices.



Backtesting a significant portfolio of high frequency systems, where cross-effects are predicted all the time are very complex and VERY computationally expensive and therefore it's usually convenient to just run with without looking for those cross effects. Being pragmatic and all that. You will never get your tickets to match your hypothetical trades in high frequency trading, so you have to stop thinking in terms of a "trade" and start thinking in terms of "the probability of a trade". This is analogous to quantuum physics, where you don't think of the particle at a particular location, you think of the probability of the particle being in a location.



By the way, speaking of long term traders and assumptions: Monroe Trout did account for the impact on prices effect in his trading, back to the 80s. He would systematically pre-empt his signals by considering the price impact his trading would have on the signals themselves (something large discretionary traders do anyway... remember Jesse Livermore and painting the tape?). Anyway, enough history...

[Johnny]

Yes, this is the kind of thing I want to hear about. Let me think about this for a bit and I'll see if I can ask some intelligent questions. In the meantime, glad to hear any other thoughts.



EDIT: btw, just to clarify this bit:



[i]Just as one can assume that long term position won't have an impact on prices (which is not true, by the way. Long term traders regularly account for the slippage caused by their positions in backtesting, although they will usually assume that prices will return to a pre-trade equilibrium before continuing to evolve).[/i]



Exactly, long term traders do have an impact on prices, but as you say, it's a matter of slippage without particularly effecting the price evolution, rather than a feedback loop in which the trade or even the order affects the price evolution. I'm specifically asking about this kind of feedback and not about slippage.

[Tradenator]

Johnny, I don't know where you can get any data for this.  But I just thought I'd throw the emprical approach out there since it works well for us at low frequencies.



When I think of positive feedback, I am reminded of Sornette, but that probably doesn't help here either.



Following on FDAX's remarks, maybe the probability avenue is a good one but suggesting that the other aspects of quantum mechanics are there would be a bit of a stretch.



EDIT: Maybe you can use fills from a lower frequency system if it is known that they trade on the open, or the close.  Then you can at least estimate for these time periods.  I suppose you could then scale up with some heroic assumptions for the intraday situation where volumes are lower.  Basically, to use my suggestion you need to tap into someone's portfolio accounting system.  I doubt if anyone on the sell side would have a database for monitoring execution efficiency against the trade order, but if they did then that might help you.

[Johnny]

I'd like to focus on this paragraph by FDAX:



[i]Via micro-structural models, we know what our impact is on price given price, time and size constraints on average, for a given market state. We can use that knowledge to account for the cost and the price movement in our trading system. We can account for the behavior of the order book after our trade from our structural model and update the model accordingly with our trades, then feed it to the next trading system to update.[/i]



If I understand correctly, there are two things going on here. One is a price impact model and the other is a structural model of the order book. Am I right in thinking that the price impact model estimates a slippage estimate as a function of {market state, time, size, etc} calibrated on recent data for that particular asset? If so, this is similar to something I do already for long term trading.



Wrt the structural model, it's clear that you could create an image of the order book after you have placed an order. But does your model then go further and attempt to say something about how other orders react to the placement of your order? Or does the structural model assume no reaction to your orders? This is the bit that I'm particularly interested in.



To clarify (although I think this is already understood) I'm certainly not trying to prise out proprietary information. I'm interested to hear FDAX's thoughts but also anyone else's too. It think this is an interesting area that NP can discuss in principle without crossing proprietary lines.

[HankScorpio]

Johnny,



I happened to stumble accross this.  Don't know if it is of any use to you, I guess the keywords "Order Book" and "Feedback" reminded me of this thread...



[i]"The price impact function describes how prices change if stocks are bought or sold. Using order book data, we explain the shape of the average price impact function by a feedback mechanism due to a strong anticorrelation between price changes and limit order flow. We find that the average price impact function has only weak explanatory power for large price changes. Hence, we study the time dependence of liquidity and find it to be a necessary prerequisite for the explanation of extreme price fluctuation.[/i]"



[Edit] A peek through Google books... here

[Johnny]

Thanks HS, I'll check it out.

[HankScorpio]

No worries - although I'm not sure it is what you are after. A while back (years ago) I remember printing and reading a rather lengthy paper on the longer term effects of a trade - the study was conducted on the Paris Bourse if memory serves. I can't find the phreaking pdf at the moment - let me have a look at my printed papers, I might be able to dig it up.