trade optimization

meteor · Post by **meteor** » Fri Dec 21, 2007 7:09 pm

I have a set up where a trading scheme is based on two paramters. I want to optmize the trade performance wrt these two paramters.

The question is then: what is a good measure of performance. I have browsed the forum and people tend to point me toward the Sortino ratio.

Is there other stat I should consider?

Thanks

nodoodahs · Post by **nodoodahs** » Fri Dec 21, 2007 9:24 pm

There's gonna have to be a subjective judgment made on what's "better." There's not a *perfect* ratio that I know of. I like Sortino better than Sharpe, but the selection of MAR can be a bugaboo when comparing systems.

Assume two output streams that are to be tested, both with similar average returns. Sortino will give different results for those output streams, depending on where your MAR (min. acceptable return) is set. If the MAR is well below the average gain for both of your streams, the one with the least incursion below the MAR will win, which will probably be the one with the least variability. If the MAR is close to the average gain of the two systems, and their variability is even relatively close to each other, then the one with the average highest above the MAR will become the "winner." So the Sortino output will vary based on the MAR. Similarly, the Sharpe will vary based on the risk-free rate chosen, but I assume most good systems will have MAR much above the risk-free rate and that nobody wants to penalize outperformance so I would prefer Sortino to Sharpe.

Other things to consider in making a decision (tie-breakers?) might be maximum drawdown, months spent in drawdown as a ratio to total months, percentage of months above a target monthly return ...

If I wasn't going to use Sortino for some reason, I might have used:

Avg Gain/ Std Dev – with a min. acceptable average gain

CAGR / Std Dev – with a min. acceptable CAGR

CAGR / Max DD – with a min. acceptable CAGR and max. acceptable DD

meteor · Post by **meteor** » Fri Dec 21, 2007 9:48 pm

thanks a lot for spending some time and sharing your mind.

what do you mean by CAGR?

nodoodahs · Post by **nodoodahs** » Fri Dec 21, 2007 10:38 pm

I use it as "cumulative (sometimes written as compounded) annualized growth rate" = [ point b (ending equity) / point a (starting equity)] ^ (1 / # of years tested) - 1.00

If you do the numerical average annual gain for the S&P 500, you get one number. If you do the CAGR, you get another number (lower), because drawdowns influence the compounding of equity. This is why I say that CAGR is risk-adjusted to some degree, and there's a relationship between CAGR, average annual gain, and volatility of return.

I'm thicker-skinned than some, but I still am a little wimpier than others with drawdowns, so if I'm pulling from a benchmark, say a stock strategy in the U.S., I may also compare the winning percentage (times it outpeformed vs. total data points) on a monthly basis versus a benchmark. It's nice to look at outperformance and maybe worth a few ticks off of the total over a long time period, keeps me from getting too emotional about it. I don't know what you're trading, but if it has a benchmark, having a good winning percentage on a month-to-month basis versus that benchmark may be a good thing for your clients (if you have any, I dunno, just supposing).

meteor · Post by **meteor** » Fri Dec 21, 2007 11:31 pm

Ok, it make sense. Thanks a bunch much for the precision.

Last question: my trade scheme is not trading everyday. When I compute the performance statistics should I include this non trading day or not?

nodoodahs · Post by **nodoodahs** » Sat Dec 22, 2007 12:40 am

I asked once on this forum, and didn't get a definitive answer as to what the "industry standard" is. I believe the standard is annual data for Sharpe, Sortino, etc., but I'm not sure.

Anything with a CAGR would default to annual, but you could in theory use any timeframe you wanted, **as long as** all your comparisons are in the same timeframe.

For example, the same data stream of returns would have a different Sortino at 9 data points of annual returns against MAR of 12%, than it would a Sortino at 9*12 data points of monthly returns against an MAR of (1.12^(1/12)-1). So when you're doing a comparison for your own needs, just be consistent with timeframes and return increments.

If you're going to show it off in the industry, it would help to confirm that they like annual data points.

*The above discussions by me all assume you're converting the raw trade data into a time stream of returns.*

If you're looking for a "by trade" performance statistic, perhaps you should look into expectancy.

Expectancy = win% * avg win amount + lose% * avg loss amount (loss amounts are negative numbers)

The bugger here is that you need a position sizing algorithm and trade count per unit time to compare an expectancy to an ROE over time, but you could easily compare one expectancy to another.

sammus · Post by **sammus** » Sat Dec 22, 2007 4:33 am

Those performance measures (MAR, CAGR etc) are very sensitive to the timeframe of each test. In fact, Curtis Faith in his new book argues that a more robust solution is use regressed annual return (RAR%) and average max DD.

Tradenator · Post by **Tradenator** » Sun Dec 23, 2007 12:32 am

I like Sharpe or Info Ratio to capture the body of the distribution, and adjust for the integrated drawdown (a la Seykota's Lake Ratio) to include consideration of both the depth and duration of events in the downside tail. Perhaps the duration is more important than the depth, though, as a fast recovery is what I'd like to see regardless of the depth. Sortino ratio alone can be dangerous for its use of the downside deviation, which doesn't fully capture those big drawdowns . The same applies to Sharpe, it should be combined with something that captures the full sting of losses.

nodoodahs · Post by **nodoodahs** » Sun Dec 23, 2007 3:47 pm

Instead of the MM (mathematical masturbation) on duration of downside events, how about a simple percent of time spent in drawdown? With a monthly (or weekly) return stream in backtests and some other metric capturing return-to-volatility, use the percentage of months (weeks) in drawdown as a "tiebreaker" between relatively comparable systems. To a large degree, percent of time in DD will be determinant of the streakiness of DD events, and can even be used to assess whether the streakiness is more than one would expect if DD were "randomly distributed."

Ultimately, the question becomes totally subjective (what's the best measure for ME?) and even the issue of DD depth vs duration is a personal preference. Even trying to combine several metrics quantitatively implies a subjective weighting algorithm between metrics.

My heart goes out for those OPMMs (other people's money managers) who most likely have to use measures they personally find suboptimal but which their investors are more comforted by.

I'm still digging into all the various risk-adjusted metrics in the literature - I've got an opinion of what I like best and a definite bias towards simpler measures, but that might change with my investigation of the "next big thing" in portfolio optimization. And I guess that might be the case with you, too.

zinmaster · Post by **zinmaster** » Wed Dec 26, 2007 6:57 pm

Any canonical way of trading off, say, negative skew against increased mean? In fact, better yet, any canonical pricing function for a return stream that takes in moments & maybe autocorrelation & spits out its value?