The possibilities for quantifying risk in portfolio analytics seems to be limited only by the imagination of researchers. Indeed, you can find dictionaries that wade through an ever-lengthening list of indicators. But any shortlist of robust metrics surely deserves to include drawdown, which offers a powerful combination of relevance and simplicity. A new research paper reminds, however, that drawdown comes in several flavors and so investors need to think carefully when deploying this metric in the quest to identify genuinely skillful portfolio results.

“Over the years, a diverse range of drawdown measures has evolved to guide asset management,” advise the authors of “Drawdown Measures: Are They All the Same?” , a recent working paper by Olaf Korn (University of Goettingen) and two co-authors. They go on to advise in the study’s abstract:

Conceptual differences between drawdown measures translate into different rankings of portfolios, which we document in a simulation study. Our research also shows that all drawdown measures can (to some degree) discriminate between skillful and unskillful portfolio managers, but differ in terms of accuracy. However, the ability to detect skill does not easily improve performance ratios where drawdown measures serve as the denominator. In conclusion, our study shows that the choice of an adequate drawdown measure is vital to the assessment of investments because different measures emphasize different aspects of risk.

For the casual investor, the notion that there’s more than one measure of drawdown may be surprising. After all, drawdown is often described as a simple peak-to-trough calculation. For example, here’s how the US stock market’s drawdown history stacks up since 2005 via the S&P 500.

As a simple, intuitive measure of risk, reviewing drawdown in this way has obvious appeal. But the question is how to quantify the data? One could start with looking at the maximum drawdown, which in the chart above is a bit more than -55%. In turn, the deepest peak-to-trough decline can be compared with the equivalent for other markets to assess relative risk profiles.

But as Olaf Korn and company point out, there are more ambitious ways to evaluate drawdown. The main point, they explain, is that “almost all drawdown measures can be subsumed under a common framework, which we refer to as the weighted drawdown (wDD) framework because its main idea is to attach weights to different elements” of drawdown. “By choosing a set of weights, new drawdown measures can be developed and tailored to a client’s conception of risk.”

For example, the paper considers several variations, including the average drawdown for a sample period, a linearly weighed drawdown, an average squared drawdown, and a trend weighted drawdown.

Which measure of drawdown excels? In search of an answer, the authors simulate portfolios that select stocks from the MSCI World Index, allowing some portfolios to exhibit skillful management. Filtering the results through various drawdown variations reveals that some are better than others in identifying skill. For example, average drawdown (ADD) fares best for historical one-year periods in the test “and exhibits significant skill detection abilities.” A close runner-up: linearly weighted drawdown (lwDD).

The authors also note that all the drawdown measures tested deliver “markedly better than the expected shortfall [a.k.a. conditional value at risk or CVaR] and the standard deviation, which only have little power to discriminate between skillful and unskillful managers.”

The paper also reviews how drawdown fares when used as a performance measure (as opposed to a risk measure, as discussed above). That is, dividing excess return over drawdown as the denominator. The standard approach is to use volatility in denominator, a.k.a. the Sharpe ratio. On this front, however, drawdown results are similar. “In detecting skill, drawdown-based performance ratios perform well on average but poorly in periods of negative returns.”

Is this the last word on selecting the best drawdown metric? No, in part because the test focuses on selecting stocks. Further research is needed for comparable tests on multi-asset class portfolios using ETFs, for instance.

In any case, this paper provides a valuable review of a little-discussed topic in matters of drawdown: there’s more than one way to define the metric for portfolio analytics and, more importantly, the results vary. The goal, then, is to identify which variation of drawdown is best suited for a particular investment strategy. Olaf Korn and company’s research study lays out a useful roadmap for further analysis.