Datamining and the hunt for returns

[Steve Castle]

Hey everyone. I'm working on this strategy and I have minute closes and a signal generator.



The signal generator has 6 or so factors, each factor has set of finite values, and I'm trying to find the factor values which produce the best returns.



We're looking over a range of periods as well, i.e., we're trading minute bars, so we're looking to trade at signal time + x minute and exit at y > x minute, finding both the optimal minute in (x) and the optimal holding period (y - x).



We're probably searching for a piece of hay in a haystack, but I'm enjoying the exercise.





I'm having a hard time summarizing the results so we can actually understand what we're looking at.



I have the returns, sharpe ratio, and a found a few other tests to run in this paper:



Optimization of Technical trading strategy



My problem is that for each factor I have 5 statistics which all determine some kind of ranking, and they don't produce the same ranking.



I could weight the rankings, but how do I determine the appropriate weights? Normalize? is it art rather than science or is there an appropriate way to do it?



Any hints / help appreciated. Thanks!

[Cheng]

I know this issue from scoring systems (which are basically a mechanical rating system which a bunch of factors giving different results).



First thing is to normalize the factors (for example mean 0 and stddev 1) in some way so that you compare apples with apples. Weighting is more an art than anything else. I personally prefer equal weights (being equivalent to no additional information available) until I have some strong proof that one is better than the others. How you adjust the weight is still open to discussion, though.



I don't like any ad-hoc weighting schemes based on so called "expert opinion". As a matter of fact experts tend to be wrong quite often (if you compare their opinion with results from data where possible) so I rather acknowledge that I don't know.

[Steve Castle]

Thanks Cheng, that makes perfect sense.

[YukaRedux]

I agree with Cheng in that, in the absence of data or intuition to the contrary, equal weighting is the way to go. But it's been my experience that nobody is equally ambivalent as to the characteristics of their system that matter most.



You presumably have some notion of the ideal performance/risk characteristic of the final system, and what you want it to do? If Sharpe is important to you, then that should command a higher weight within the list of statistics. Or is peak-trough drawdown the key element? Or win:loss ratio? And what positive statistics are nice-to-have rather than must-have? If you can say that X is twice as important to you as Y, and that A, B and C are equally important in aggregate as Y, then you've got the basis for a rough weighting system. You can, and should, compare the performance of this system to the baseline equally weighted version.

[cjforex]

Also consider not choosing the best. Run them in simulation mode forward. Then apply a basic filter with any of your favorite metrics.



Allocate funds to the top 10 in a weighted manner.



I've had pretty good success this way and avoid fitting. Every week remeasure, rebalance.

[intradaybill]

"finding both the optimal minute in (x) and the optimal holding period (y - x)."



These are not constants but random variables, imo.

[Steve Castle]

intradaybill: you're probably right, I'm assuming our optimal minute in will generally be the first, and holding period would, in practice, be determined by other factors. The person who's guiding the research actively trades these bars so he's expecting to see some kind of relationship here, so I have to produce this. I'm more interested in seeing a signal which is strongly correlated with a return/ sharpe ratio / score /etc. and leave the rest up to optimizing the execution.





Yuka, cj, thanks for your notes. We've since found a few signals which produce equally weighted scored strength well above the others, and we'll use your advice to drill down and see if changing the weights to reflect our view impacts the results. In the end though, trading all the signals with strong performance makes the most sense, and we can use our view on the score to weight the signals or whatever else we wish.



Thanks again

[iasmath]

Since it is a system that trades minute bars, I imagine there are going to be lots of trades in a month. So perhaps it is a good idea to consider Kelly's optimal fraction as a ranking method. In that case you would have two main statistics: #winning_trades/#losing_trades and avg(winning_return)/avg(losing_return) that can be used to calculate the optimal fraction. A third statistics would be the max drawdown.

[Steve Castle]

iasmath: thanks for suggestion, I'll be including those as well, much appreciated.

[Steve Castle]

Anyone want to comment on this? Is this clear as a teaser plot?



Deliberately not saying much, would like the plot to be obvious given what's already in the thread.



Fake Title is actually a short description of signal, something like 'See's Shadow'.



[img]/User%20Files/7195/figure_1.png[/img]



thanks for any comments/criticism